constraints.estimation

Module: constraints.estimation

Inheritance diagram for selectinf.constraints.estimation:

digraph inheritance337f579918 { rankdir=LR; size="8.0, 12.0"; "constraints.estimation.gaussian_cumulant" [URL="#selectinf.constraints.estimation.gaussian_cumulant",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="Cumulant generating function for Gaussian"]; "regreg.smooth.smooth_atom" -> "constraints.estimation.gaussian_cumulant" [arrowsize=0.5,style="setlinewidth(0.5)"]; "constraints.estimation.gaussian_cumulant_conjugate" [URL="#selectinf.constraints.estimation.gaussian_cumulant_conjugate",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="Conjugate of cumulant generating function "]; "regreg.smooth.smooth_atom" -> "constraints.estimation.gaussian_cumulant_conjugate" [arrowsize=0.5,style="setlinewidth(0.5)"]; "constraints.estimation.gaussian_cumulant_conjugate_known" [URL="#selectinf.constraints.estimation.gaussian_cumulant_conjugate_known",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="Cumulant generating function for Gaussian"]; "regreg.smooth.smooth_atom" -> "constraints.estimation.gaussian_cumulant_conjugate_known" [arrowsize=0.5,style="setlinewidth(0.5)"]; "constraints.estimation.gaussian_cumulant_known" [URL="#selectinf.constraints.estimation.gaussian_cumulant_known",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="Cumulant generating function for Gaussian"]; "regreg.smooth.smooth_atom" -> "constraints.estimation.gaussian_cumulant_known" [arrowsize=0.5,style="setlinewidth(0.5)"]; "constraints.estimation.optimal_tilt" [URL="#selectinf.constraints.estimation.optimal_tilt",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="An objective used to find an"]; "regreg.smooth.smooth_atom" -> "constraints.estimation.optimal_tilt" [arrowsize=0.5,style="setlinewidth(0.5)"]; "constraints.estimation.softmax" [URL="#selectinf.constraints.estimation.softmax",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="Softmax function"]; "regreg.smooth.smooth_atom" -> "constraints.estimation.softmax" [arrowsize=0.5,style="setlinewidth(0.5)"]; "constraints.estimation.softmax_conjugate" [URL="#selectinf.constraints.estimation.softmax_conjugate",fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",target="_top",tooltip="Objective function that computes the value of "]; "regreg.smooth.smooth_atom" -> "constraints.estimation.softmax_conjugate" [arrowsize=0.5,style="setlinewidth(0.5)"]; "problems.composite.composite" [fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",tooltip="A generic way to specify a problem in composite form."]; "problems.composite.smooth" [fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",tooltip="A composite subclass that has 0 as "]; "problems.composite.composite" -> "problems.composite.smooth" [arrowsize=0.5,style="setlinewidth(0.5)"]; "regreg.smooth.smooth_atom" [fontname="Vera Sans, DejaVu Sans, Liberation Sans, Arial, Helvetica, sans",fontsize=10,height=0.25,shape=box,style="setlinewidth(0.5)",tooltip="A class for representing a smooth function and its gradient"]; "problems.composite.smooth" -> "regreg.smooth.smooth_atom" [arrowsize=0.5,style="setlinewidth(0.5)"]; }

Classes

gaussian_cumulant

class selectinf.constraints.estimation.gaussian_cumulant(X, offset=None, quadratic=None, initial=None)[source]

Bases: regreg.smooth.smooth_atom

Cumulant generating function for Gaussian likelihood with unknown variance in the regression model with design matrix \(X\).

The cumulant generating function is determined by

\[egin{aligned} e^{\Lambda(\gamma,\eta)} &= (2\pi)^{-n/2} \int e^{\gamma^T(X^Ty) - \]
rac{eta}{2}|y|^2_2} ; dy

&= e^{

rac{1}{2eta}|Xgamma|^2_2}.

end{aligned}

__init__(X, offset=None, quadratic=None, initial=None)[source]
Parameters

X : np.ndarray

Design matrix.

regression_parameters(natural_param)[source]

From the natural parameters, form the usual \((eta, \sigma^2)\) parameters.

Parameters

natural_param : np.ndarray

Natural parameters

Returns

The usual estimates of \((eta, \sigma^2)\) derived

from the natural parameters.

smooth_objective(natural_param, mode='both', check_feasibility=False)[source]
Parameters

arg : ndarray

The current parameter values.

mode : str

One of [‘func’, ‘grad’, ‘both’].

check_feasibility : bool

If True, return np.inf when point is not feasible, i.e. when beta is not in the domain.

Returns

If mode is ‘func’ returns just the objective value

at beta, else if mode is ‘grad’ returns the gradient

else returns both.

classmethod affine(linear_operator, offset, coef=1, diag=False, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property conjugate
get_conjugate()
get_lipschitz()
get_offset()
get_quadratic()

Get the quadratic part of the composite.

latexify(var=None, idx='')
classmethod linear(linear_operator, coef=1, diag=False, offset=None, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

property lipschitz
nonsmooth_objective(x, check_feasibility=False)
objective(x, check_feasibility=False)
objective_template = 'f(%(var)s)'
objective_vars = {'coef': 'C', 'offset': '\\alpha+', 'shape': 'p', 'var': '\\beta'}
property offset
proximal(quadratic)
proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

scale(obj, copy=False)
set_lipschitz(value)
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, coef=1, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)

gaussian_cumulant_conjugate

class selectinf.constraints.estimation.gaussian_cumulant_conjugate(X, offset=None, quadratic=None, initial=None)[source]

Bases: regreg.smooth.smooth_atom

Conjugate of cumulant generating function for Gaussian likelihood with unknown variance in the regression model with design matrix \(X\).

The cumulant generating function is determined by

\[egin{aligned} e^{\Lambda(\gamma,\eta)} &= (2\pi)^{-n/2} \int e^{\gamma^T(X^Ty) - \]
rac{eta}{2}|y|^2_2} ; dy

&= e^{

rac{1}{2eta}|Xgamma|^2_2}.

end{aligned}

The convex conjugate of this function is

\[\Lambda^*(\delta, s) = -\]

rac{n}{2}(1 - log(n)) - rac{n}{2}

log(-2 s - delta^T(X^TX)^{-1}delta)

plus the constraint \(\delta \in ext{row}(X)\).

__init__(X, offset=None, quadratic=None, initial=None)[source]
Parameters

X : np.ndarray

Design matrix.

smooth_objective(mean_param, mode='both', check_feasibility=False)[source]
Parameters

arg : ndarray

The current parameter values.

mode : str

One of [‘func’, ‘grad’, ‘both’].

check_feasibility : bool

If True, return np.inf when point is not feasible, i.e. when beta is not in the domain.

Returns

If mode is ‘func’ returns just the objective value

at beta, else if mode is ‘grad’ returns the gradient

else returns both.

classmethod affine(linear_operator, offset, coef=1, diag=False, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property conjugate
get_conjugate()
get_lipschitz()
get_offset()
get_quadratic()

Get the quadratic part of the composite.

latexify(var=None, idx='')
classmethod linear(linear_operator, coef=1, diag=False, offset=None, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

property lipschitz
nonsmooth_objective(x, check_feasibility=False)
objective(x, check_feasibility=False)
objective_template = 'f(%(var)s)'
objective_vars = {'coef': 'C', 'offset': '\\alpha+', 'shape': 'p', 'var': '\\beta'}
property offset
proximal(quadratic)
proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

scale(obj, copy=False)
set_lipschitz(value)
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, coef=1, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)

gaussian_cumulant_conjugate_known

class selectinf.constraints.estimation.gaussian_cumulant_conjugate_known(X, sigma, offset=None, quadratic=None, initial=None)[source]

Bases: regreg.smooth.smooth_atom

Cumulant generating function for Gaussian likelihood with known variance in the regression model with design matrix \(X\).

The cumulant generating function is

\[\Lambda(\gamma) = \]

rac{sigma^2}{2} |Xgamma|^2_2

Its conjugate is

\[\Lambda^*(\delta) = \]

rac{1}{2sigma^2} delta^T(X^TX)^{-1}delta

with the constraint \(\delta \in ext{row}(X)\).

__init__(X, sigma, offset=None, quadratic=None, initial=None)[source]
Parameters

X : np.ndarray

Design matrix.

sigma : float

Known standard deviation.

smooth_objective(mean_param, mode='both', check_feasibility=False)[source]
Parameters

arg : ndarray

The current parameter values.

mode : str

One of [‘func’, ‘grad’, ‘both’].

check_feasibility : bool

If True, return np.inf when point is not feasible, i.e. when beta is not in the domain.

Returns

If mode is ‘func’ returns just the objective value

at beta, else if mode is ‘grad’ returns the gradient

else returns both.

classmethod affine(linear_operator, offset, coef=1, diag=False, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property conjugate
get_conjugate()
get_lipschitz()
get_offset()
get_quadratic()

Get the quadratic part of the composite.

latexify(var=None, idx='')
classmethod linear(linear_operator, coef=1, diag=False, offset=None, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

property lipschitz
nonsmooth_objective(x, check_feasibility=False)
objective(x, check_feasibility=False)
objective_template = 'f(%(var)s)'
objective_vars = {'coef': 'C', 'offset': '\\alpha+', 'shape': 'p', 'var': '\\beta'}
property offset
proximal(quadratic)
proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

scale(obj, copy=False)
set_lipschitz(value)
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, coef=1, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)

gaussian_cumulant_known

class selectinf.constraints.estimation.gaussian_cumulant_known(X, sigma, offset=None, quadratic=None, initial=None)[source]

Bases: regreg.smooth.smooth_atom

Cumulant generating function for Gaussian likelihood with known variance in the regression model with design matrix \(X\).

The cumulant generating function is

\[\Lambda(\gamma) = \]

rac{sigma^2}{2} |Xgamma|^2_2

__init__(X, sigma, offset=None, quadratic=None, initial=None)[source]
Parameters

X : np.ndarray

Design matrix.

sigma : float

Known standard deviation.

regression_parameters(natural_param)[source]

From the natural parameters, form the usual \(eta\) parameters.

smooth_objective(natural_param, mode='both', check_feasibility=False)[source]
Parameters

arg : ndarray

The current parameter values.

mode : str

One of [‘func’, ‘grad’, ‘both’].

check_feasibility : bool

If True, return np.inf when point is not feasible, i.e. when beta is not in the domain.

Returns

If mode is ‘func’ returns just the objective value

at beta, else if mode is ‘grad’ returns the gradient

else returns both.

classmethod affine(linear_operator, offset, coef=1, diag=False, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property conjugate
get_conjugate()
get_lipschitz()
get_offset()
get_quadratic()

Get the quadratic part of the composite.

latexify(var=None, idx='')
classmethod linear(linear_operator, coef=1, diag=False, offset=None, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

property lipschitz
nonsmooth_objective(x, check_feasibility=False)
objective(x, check_feasibility=False)
objective_template = 'f(%(var)s)'
objective_vars = {'coef': 'C', 'offset': '\\alpha+', 'shape': 'p', 'var': '\\beta'}
property offset
proximal(quadratic)
proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

scale(obj, copy=False)
set_lipschitz(value)
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, coef=1, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)

optimal_tilt

class selectinf.constraints.estimation.optimal_tilt(affine_con, direction_of_interest, offset=None, quadratic=None, initial=None)[source]

Bases: regreg.smooth.smooth_atom

An objective used to find an approximately best tilt for a given affine constraint and a given direction of interest.

We approximately solve the problem

..math:

ext{min.}_{c,z:A(z + c\eta + \gamma) \leq b} \|z + c \eta\|^2_{\Sigma}

where the objective is Mahalanobis distance for the constraint’s covariance, \(\gamma\) is the constraint’s mean and the set \(\{w:Aw \leq b\}\) is the affine constraint.

__init__(affine_con, direction_of_interest, offset=None, quadratic=None, initial=None)[source]

Initialize self. See help(type(self)) for accurate signature.

smooth_objective(z, mode='both', check_feasibility=False)[source]
Parameters

arg : ndarray

The current parameter values.

mode : str

One of [‘func’, ‘grad’, ‘both’].

check_feasibility : bool

If True, return np.inf when point is not feasible, i.e. when beta is not in the domain.

Returns

If mode is ‘func’ returns just the objective value

at beta, else if mode is ‘grad’ returns the gradient

else returns both.

fit(**regreg_args)[source]
classmethod affine(linear_operator, offset, coef=1, diag=False, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property conjugate
get_conjugate()
get_lipschitz()
get_offset()
get_quadratic()

Get the quadratic part of the composite.

latexify(var=None, idx='')
classmethod linear(linear_operator, coef=1, diag=False, offset=None, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

property lipschitz
nonsmooth_objective(x, check_feasibility=False)
objective(x, check_feasibility=False)
objective_template = 'f(%(var)s)'
objective_vars = {'coef': 'C', 'offset': '\\alpha+', 'shape': 'p', 'var': '\\beta'}
property offset
proximal(quadratic)
proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

scale(obj, copy=False)
set_lipschitz(value)
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, coef=1, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)

softmax

class selectinf.constraints.estimation.softmax(affine_con, sigma=1.0, feasible_point=None, offset=None, quadratic=None, initial=None)[source]

Bases: regreg.smooth.smooth_atom

Softmax function

rac{1}{2sigma^2} |mu|^2 +

sum_{i=1}^n log(1 + sigma /(b_i - a_i^Tmu))

__init__(affine_con, sigma=1.0, feasible_point=None, offset=None, quadratic=None, initial=None)[source]

Initialize self. See help(type(self)) for accurate signature.

smooth_objective(mean_param, mode='func', check_feasibility=False)[source]
Parameters

arg : ndarray

The current parameter values.

mode : str

One of [‘func’, ‘grad’, ‘both’].

check_feasibility : bool

If True, return np.inf when point is not feasible, i.e. when beta is not in the domain.

Returns

If mode is ‘func’ returns just the objective value

at beta, else if mode is ‘grad’ returns the gradient

else returns both.

classmethod affine(linear_operator, offset, coef=1, diag=False, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property conjugate
get_conjugate()
get_lipschitz()
get_offset()
get_quadratic()

Get the quadratic part of the composite.

latexify(var=None, idx='')
classmethod linear(linear_operator, coef=1, diag=False, offset=None, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

property lipschitz
nonsmooth_objective(x, check_feasibility=False)
objective(x, check_feasibility=False)
objective_template = 'f(%(var)s)'
objective_vars = {'coef': 'C', 'offset': '\\alpha+', 'shape': 'p', 'var': '\\beta'}
property offset
proximal(quadratic)
proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

scale(obj, copy=False)
set_lipschitz(value)
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, coef=1, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)

softmax_conjugate

class selectinf.constraints.estimation.softmax_conjugate(affine_con, feasible_point, sigma=1.0, offset=None, quadratic=None, initial=None)[source]

Bases: regreg.smooth.smooth_atom

Objective function that computes the value of

rac{1}{2} |y-z|^2_2 + sum_{i=1}^n log(1 + 1 /(b_i - a_i^Tmu))

__init__(affine_con, feasible_point, sigma=1.0, offset=None, quadratic=None, initial=None)[source]

Initialize self. See help(type(self)) for accurate signature.

smooth_objective(natural_param, mode='func', check_feasibility=False)[source]
Parameters

arg : ndarray

The current parameter values.

mode : str

One of [‘func’, ‘grad’, ‘both’].

check_feasibility : bool

If True, return np.inf when point is not feasible, i.e. when beta is not in the domain.

Returns

If mode is ‘func’ returns just the objective value

at beta, else if mode is ‘grad’ returns the gradient

else returns both.

classmethod affine(linear_operator, offset, coef=1, diag=False, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

apply_offset(x)

If self.offset is not None, return x-self.offset, else return x.

property conjugate
get_conjugate()
get_lipschitz()
get_offset()
get_quadratic()

Get the quadratic part of the composite.

latexify(var=None, idx='')
classmethod linear(linear_operator, coef=1, diag=False, offset=None, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

property lipschitz
nonsmooth_objective(x, check_feasibility=False)
objective(x, check_feasibility=False)
objective_template = 'f(%(var)s)'
objective_vars = {'coef': 'C', 'offset': '\\alpha+', 'shape': 'p', 'var': '\\beta'}
property offset
proximal(quadratic)
proximal_optimum(quadratic)
proximal_step(quadratic, prox_control=None)

Compute the proximal optimization

Parameters

prox_control: [None, dict]

If not None, then a dictionary of parameters for the prox procedure

property quadratic

Quadratic part of the object, instance of regreg.identity_quadratic.identity_quadratic.

scale(obj, copy=False)
set_lipschitz(value)
set_offset(value)
set_quadratic(quadratic)

Set the quadratic part of the composite.

classmethod shift(offset, coef=1, quadratic=None, **kws)

Keywords given in kws are passed to cls constructor along with other arguments

smoothed(smoothing_quadratic)

Add quadratic smoothing term

solve(quadratic=None, return_optimum=False, **fit_args)