# Introduction

**SparseRegression** is a Julia package which combines JuliaML primitives to implement high-performance algorithms for fitting linear models.

## Objective Function

The objective function that SparseRegression can solve takes the form:

\[\frac{1}{n}\sum_{i=1}^n w_i f(y_i, x_i^T\beta) + \sum_{j=1}^p \lambda_j J(\beta_j),\]

where $f$ is a loss function, $J$ is a penalty or regularization function, the $w_i$'s are nonnegative observation weights and the $\lambda_j$'s are nonnegative element-wise regularization parameters. Many models take this form:

Model | $f(y_i, x_i^T\beta)$ | $g(\beta_j)$ |
---|---|---|

Lasso Regression | $\frac{1}{2}(y_i - x_i^T\beta)^2$ | $|\beta_j|$ |

Ridge Regression | $\frac{1}{2}(y_i - x_i^T\beta)^2$ | $\beta_j^2$ |

SVM | $max(0, 1 - y_i x_i^T\beta)$ | $\beta_j^2$ |

## JuliaML

The three core JuliaML packages that SparseRegression brings together are: