algorithms.sqrt_lasso

Module: algorithms.sqrt_lasso

Inheritance diagram for selectinf.algorithms.sqrt_lasso:

digraph inheritance335b2be946 { rankdir=LR; size="8.0, 12.0"; "algorithms.sqrt_lasso.l2norm_saturated" [URL="#selectinf.algorithms.sqrt_lasso.l2norm_saturated",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="A little wrapper so that sqrt_lasso view can be bootstrapped"]; "regreg.smooth.smooth_atom" -> "algorithms.sqrt_lasso.l2norm_saturated" [arrowsize=0.5,style="setlinewidth(0.5)"]; "algorithms.sqrt_lasso.sqlasso_objective" [URL="#selectinf.algorithms.sqrt_lasso.sqlasso_objective",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="The square-root LASSO objective. Essentially"]; "regreg.smooth.smooth_atom" -> "algorithms.sqrt_lasso.sqlasso_objective" [arrowsize=0.5,style="setlinewidth(0.5)"]; "algorithms.sqrt_lasso.sqlasso_objective_skinny" [URL="#selectinf.algorithms.sqrt_lasso.sqlasso_objective_skinny",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="The square-root LASSO objective on larger parameter space:"]; "regreg.smooth.smooth_atom" -> "algorithms.sqrt_lasso.sqlasso_objective_skinny" [arrowsize=0.5,style="setlinewidth(0.5)"]; "problems.composite.composite" [fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",tooltip="A generic way to specify a problem in composite form."]; "problems.composite.smooth" [fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",tooltip="A composite subclass that has 0 as "]; "problems.composite.composite" -> "problems.composite.smooth" [arrowsize=0.5,style="setlinewidth(0.5)"]; "regreg.smooth.smooth_atom" [fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",tooltip="A class for representing a smooth function and its gradient"]; "problems.composite.smooth" -> "regreg.smooth.smooth_atom" [arrowsize=0.5,style="setlinewidth(0.5)"]; }

Module to solve sqrt-LASSO convex program using regreg.

Classes

l2norm_saturated

class selectinf.algorithms.sqrt_lasso.l2norm_saturated(shape, response, coef=1.0, offset=None, quadratic=None, initial=None)

Bases: regreg.smooth.smooth_atom

A little wrapper so that sqrt_lasso view can be bootstrapped like a glm.

Mainly needs the saturated_loss.hessian method.

__init__(shape, response, coef=1.0, offset=None, quadratic=None, initial=None)

Initialize self. See help(type(self)) for accurate signature.

smooth_objective(natural_param, mode='both', check_feasibility=False)

Evaluate the smooth objective, computing its value, gradient or both.

Parameters

natural_param : ndarray

The current parameter values.

mode : str

One of [‘func’, ‘grad’, ‘both’].

check_feasibility : bool

If True, return np.inf when point is not feasible, i.e. when natural_param is not in the domain.

Returns

If mode is ‘func’ returns just the objective value

at natural_param, else if mode is ‘grad’ returns the gradient

else returns both.

hessian(natural_param)

Hessian of the loss.

Parameters

natural_param : ndarray

Parameters where Hessian will be evaluated.

Returns

hess : ndarray

A 1D-array representing the diagonal of the Hessian evaluated at natural_param.

get_data()

set_data(data)

property data

mean_function(eta)

classmethod affine(linear_operator, offset, coef=1, diag=False, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property conjugate

get_conjugate()

get_lipschitz()

get_offset()

get_quadratic()

Get the quadratic part of the composite.

latexify(var=None, idx='')

classmethod linear(linear_operator, coef=1, diag=False, offset=None, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

property lipschitz

nonsmooth_objective(x, check_feasibility=False)

objective(x, check_feasibility=False)

objective_template = 'f(%(var)s)'

objective_vars = {'coef': 'C', 'offset': '\\alpha+', 'shape': 'p', 'var': '\\beta'}

property offset

proximal(quadratic)

proximal_optimum(quadratic)

proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

scale(obj, copy=False)

set_lipschitz(value)

set_offset(value)

set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, coef=1, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)

sqlasso_objective

class selectinf.algorithms.sqrt_lasso.sqlasso_objective(X, Y, quadratic=None, initial=None, offset=None)

Bases: regreg.smooth.smooth_atom

The square-root LASSO objective. Essentially smooth, but singular on \(\{eta: y=Xeta\}\).

This singularity is ignored in solving the problem. It might be a problem sometimes?

__init__(X, Y, quadratic=None, initial=None, offset=None)

Initialize self. See help(type(self)) for accurate signature.

get_data()

set_data(data)

property data

Data for the sqrt LASSO objective.

smooth_objective(x, mode='both', check_feasibility=False)

Parameters

arg : ndarray

The current parameter values.

mode : str

One of [‘func’, ‘grad’, ‘both’].

check_feasibility : bool

If True, return np.inf when point is not feasible, i.e. when beta is not in the domain.

Returns

If mode is ‘func’ returns just the objective value

at beta, else if mode is ‘grad’ returns the gradient

else returns both.

hessian(beta)

Compute the Hessian of the loss :math:`

abla^2 ell(Xeta)`.

Parameters

beta : ndarray

Parameters.

Returns

hess : ndarray

Hessian of the loss at \(eta\), defined everywhere the residual is not 0.

classmethod affine(linear_operator, offset, coef=1, diag=False, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property conjugate

get_conjugate()

get_lipschitz()

get_offset()

get_quadratic()

Get the quadratic part of the composite.

latexify(var=None, idx='')

classmethod linear(linear_operator, coef=1, diag=False, offset=None, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

property lipschitz

nonsmooth_objective(x, check_feasibility=False)

objective(x, check_feasibility=False)

objective_template = 'f(%(var)s)'

objective_vars = {'coef': 'C', 'offset': '\\alpha+', 'shape': 'p', 'var': '\\beta'}

property offset

proximal(quadratic)

proximal_optimum(quadratic)

proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

scale(obj, copy=False)

set_lipschitz(value)

set_offset(value)

set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, coef=1, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)

sqlasso_objective_skinny

class selectinf.algorithms.sqrt_lasso.sqlasso_objective_skinny(X, Y)

Bases: regreg.smooth.smooth_atom

The square-root LASSO objective on larger parameter space:

\[(eta, \sigma) \mapsto \]

rac{|y-Xeta|_2^2}{sigma} + sigma

__init__(X, Y)

Initialize self. See help(type(self)) for accurate signature.

smooth_objective(x, mode='both', check_feasibility=False)

Parameters

arg : ndarray

The current parameter values.

mode : str

One of [‘func’, ‘grad’, ‘both’].

check_feasibility : bool

If True, return np.inf when point is not feasible, i.e. when beta is not in the domain.

Returns

If mode is ‘func’ returns just the objective value

at beta, else if mode is ‘grad’ returns the gradient

else returns both.

classmethod affine(linear_operator, offset, coef=1, diag=False, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property conjugate

get_conjugate()

get_lipschitz()

get_offset()

get_quadratic()

Get the quadratic part of the composite.

latexify(var=None, idx='')

classmethod linear(linear_operator, coef=1, diag=False, offset=None, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

property lipschitz

nonsmooth_objective(x, check_feasibility=False)

objective(x, check_feasibility=False)

objective_template = 'f(%(var)s)'

objective_vars = {'coef': 'C', 'offset': '\\alpha+', 'shape': 'p', 'var': '\\beta'}

property offset

proximal(quadratic)

proximal_optimum(quadratic)

proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

scale(obj, copy=False)

set_lipschitz(value)

set_offset(value)

set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, coef=1, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)

Functions

selectinf.algorithms.sqrt_lasso.choose_lambda(X, quantile=0.95, ndraw=10000)

Choose a value of lam for the square-root LASSO based on an empirical quantile of the distribution of

$$

rac{|X^Tepsilon|_{infty}}{|epsilon|_2}.

$$

Parameters

X : np.float((n, p))

Design matrix.

quantilefloat

What quantile should we use?

ndrawint

How many draws?

selectinf.algorithms.sqrt_lasso.estimate_sigma(observed, truncated_df, lower_bound, upper_bound, untruncated_df=0, factor=3, npts=50, nsample=2000)

Produce an estimate of \(\sigma\) from a constrained error sum of squares. The relevant distribution is a scaled \(\chi^2\) restricted to \([0,U]\) where \(U\) is upper_bound.

Parameters

observed : float

The observed sum of squares.

truncated_df : float

Degrees of freedom of the truncated \(\chi^2\) in the sum of squares. The observed sum is assumed to be the sum of an independent untruncated \(\chi^2\) and the truncated one.

lower_bound : float

Lower limit of truncation interval.

upper_bound : float

Upper limit of truncation interval.

untruncated_df : float

Degrees of freedom of the untruncated \(\chi^2\) in the sum of squares.

factor : float

Range of candidate values is [observed/factor, observed*factor]

npts : int

How many candidate values for interpolator.

nsample : int

How many samples for each expected value of the truncated sum of squares.

Returns

sigma_hat : np.float

Estimate of \(\sigma\).

selectinf.algorithms.sqrt_lasso.goodness_of_fit(lasso_obj, statistic, force=False, alternative='twosided', ndraw=5000, burnin=2000, sample=None)

Compute a goodness of fit test based on a given statistic applied to

\[U_{-E}(y) = (I-P_E)y / \|(I-P_E)y\|_2\]

which is ancillary for the selected model.

Parameters

lasso_obj : lasso

Instance of selection.algorithms.lasso.lasso instantiated with a gaussian loglikelihood (instance of regreg.smooth.glm.gaussian_loglike

statistic : callable

Statistic to compute on observed \(U_{-E}\) as well as sample from null distribution.

alternative : str

One of [‘greater’, ‘less’, ‘twosided’]. Determines how pvalue is computed, based on upper tail, lower tail or equal tail.

force : bool

Resample from \(U_{-E}\) under the null even if the instance already has a null sample.

ndraw : int (optional)

If a null sample is to be drawn, how large a sample? Defaults to 1000.

burnin : int (optional)

If a null sample is to be drawn, how long a burnin? Defaults to 1000.

sample : multiparameter_family (optional)

If not None, this is used as sample instead of generating a new sample.

Returns

pvalue : np.float

Two-tailed p-value.

selectinf.algorithms.sqrt_lasso.l2norm_glm(X, Y, quadratic=None, initial=None, offset=None)

selectinf.algorithms.sqrt_lasso.solve_sqrt_lasso(X, Y, weights=None, initial=None, quadratic=None, solve_args={}, force_fat=False)

Solve the square-root LASSO optimization problem:

$$

ext{minimize}_{eta} |y-Xeta|_2 + D |eta|,

$$ where \(D\) is the diagonal matrix with weights on its diagonal.

Parameters

y : np.float((n,))

The target, in the model \(y = Xeta\)

X : np.float((n, p))

The data, in the model \(y = Xeta\)

weights : np.float

Coefficients of the L-1 penalty in optimization problem, note that different coordinates can have different coefficients.

initial : np.float(p)

Initial point for optimization.

solve_args : dict

Arguments passed to regreg solver.

quadratic : regreg.identity_quadratic

A quadratic term added to objective function.

selectinf.algorithms.sqrt_lasso.solve_sqrt_lasso_fat(X, Y, weights=None, initial=None, quadratic=None, solve_args={})

Solve the square-root LASSO optimization problem:

$$

ext{minimize}_{eta} |y-Xeta|_2 + D |eta|,

$$ where \(D\) is the diagonal matrix with weights on its diagonal.

Parameters

y : np.float((n,))

The target, in the model \(y = Xeta\)

X : np.float((n, p))

The data, in the model \(y = Xeta\)

weights : np.float

Coefficients of the L-1 penalty in optimization problem, note that different coordinates can have different coefficients.

initial : np.float(p)

Initial point for optimization.

solve_args : dict

Arguments passed to regreg solver.

quadratic : regreg.identity_quadratic

A quadratic term added to objective function.

selectinf.algorithms.sqrt_lasso.solve_sqrt_lasso_skinny(X, Y, weights=None, initial=None, quadratic=None, solve_args={})

Solve the square-root LASSO optimization problem:

$$

ext{minimize}_{eta} |y-Xeta|_2 + D |eta|,

$$ where \(D\) is the diagonal matrix with weights on its diagonal.

Parameters

y : np.float((n,))

The target, in the model \(y = Xeta\)

X : np.float((n, p))

The data, in the model \(y = Xeta\)

weights : np.float

Coefficients of the L-1 penalty in optimization problem, note that different coordinates can have different coefficients.

initial : np.float(p)

Initial point for optimization.

solve_args : dict

Arguments passed to regreg solver.

quadratic : regreg.identity_quadratic

A quadratic term added to objective function.