SparseRegression is a Julia package which combines JuliaML primitives to implement high-performance algorithms for fitting linear models.

Objective Function

The objective function that SparseRegression can solve takes the form:

\[\frac{1}{n}\sum_{i=1}^n w_i f(y_i, x_i^T\beta) + \sum_{j=1}^p \lambda_j J(\beta_j),\]

where $f$ is a loss function, $J$ is a penalty or regularization function, the $w_i$'s are nonnegative observation weights and the $\lambda_j$'s are nonnegative element-wise regularization parameters. Many models take this form:

Model$f(y_i, x_i^T\beta)$$g(\beta_j)$
Lasso Regression$\frac{1}{2}(y_i - x_i^T\beta)^2$$|\beta_j|$
Ridge Regression$\frac{1}{2}(y_i - x_i^T\beta)^2$$\beta_j^2$
SVM$max(0, 1 - y_i x_i^T\beta)$$\beta_j^2$


The three core JuliaML packages that SparseRegression brings together are: