randomized.group_lasso¶
Module: randomized.group_lasso¶
Inheritance diagram for selectinf.randomized.group_lasso:
Classes¶
group_lasso¶
-
class
selectinf.randomized.group_lasso.group_lasso(loglike, groups, weights, ridge_term, randomizer, perturb=None)[source]¶ Bases:
selectinf.randomized.query.queryA class for the randomized LASSO for post-selection inference. The problem solved is
\[\text{minimize}_{\beta} \ell(\beta) + \sum_{i=1}^p \lambda_i |\beta_i\| - \omega^T\beta + \frac{\epsilon}{2} \|\beta\|^2_2\]where \(\lambda\) is lam, \(\omega\) is a randomization generated below and the last term is a small ridge penalty. Each static method forms \(\ell\) as well as the \(\ell_1\) penalty. The generic class forms the remaining two terms in the objective.
-
__init__(loglike, groups, weights, ridge_term, randomizer, perturb=None)[source]¶ Create a new post-selection object for the LASSO problem
- Parameters
loglike : regreg.smooth.glm.glm
A (negative) log-likelihood as implemented in regreg.
feature_weights : np.ndarray
Feature weights for L-1 penalty. If a float, it is brodcast to all features.
ridge_term : float
How big a ridge term to add?
randomizer : object
Randomizer – contains representation of randomization density.
perturb : np.ndarray
Random perturbation subtracted as a linear term in the objective function.
-
fit(solve_args={'min_its': 50, 'tol': 1e-12}, perturb=None)[source]¶ Fit the randomized lasso using regreg.
- Parameters
solve_args : keyword args
Passed to regreg.problems.simple_problem.solve.
- Returns
signs : np.float
Support and non-zero signs of randomized lasso solution.
-
summary(observed_target, group_assignments, target_cov, target_score_cov, alternatives, parameter=None, level=0.9, ndraw=10000, compute_intervals=False)[source]¶ Produce p-values and confidence intervals for targets of model including selected features
- Parameters
target : one of [‘selected’, ‘full’]
features : np.bool
Binary encoding of which features to use in final model and targets.
parameter : np.array
Hypothesized value for parameter – defaults to 0.
level : float
Confidence level.
ndraw : int (optional)
Defaults to 1000.
burnin : int (optional)
Defaults to 1000.
compute_intervals : bool
Compute confidence intervals?
dispersion : float (optional)
Use a known value for dispersion, or Pearson’s X^2?
-
static
gaussian(X, Y, groups, weights, sigma=1.0, quadratic=None, ridge_term=None, randomizer_scale=None)[source]¶ Squared-error LASSO with feature weights. Objective function is (before randomization)
\[\beta \mapsto \frac{1}{2} \|Y-X\beta\|^2_2 + \sum_{i=1}^p \lambda_i |\beta_i|\]where \(\lambda\) is feature_weights. The ridge term is determined by the Hessian and np.std(Y) by default, as is the randomizer scale.
- Parameters
X : ndarray
Shape (n,p) – the design matrix.
Y : ndarray
Shape (n,) – the response.
feature_weights: [float, sequence]
Penalty weights. An intercept, or other unpenalized features are handled by setting those entries of feature_weights to 0. If feature_weights is a float, then all parameters are penalized equally.
sigma : float (optional)
Noise variance. Set to 1 if covariance_estimator is not None. This scales the loglikelihood by sigma**(-2).
quadratic : regreg.identity_quadratic.identity_quadratic (optional)
An optional quadratic term to be added to the objective. Can also be a linear term by setting quadratic coefficient to 0.
ridge_term : float
How big a ridge term to add?
randomizer_scale : float
Scale for IID components of randomizer.
randomizer : str
One of [‘laplace’, ‘logistic’, ‘gaussian’]
- Returns
L : selection.randomized.lasso.lasso
-
static
logistic(X, successes, groups, weights, trials=None, quadratic=None, ridge_term=None, randomizer_scale=None)[source]¶ Logistic LASSO with feature weights (before randomization)
\[\beta \mapsto \ell(X\beta) + \sum_{i=1}^p \lambda_i |\beta_i|\]where \(\ell\) is the negative of the logistic log-likelihood (half the logistic deviance) and \(\lambda\) is feature_weights.
- Parameters
X : ndarray
Shape (n,p) – the design matrix.
successes : ndarray
Shape (n,) – response vector. An integer number of successes. For data that is proportions, multiply the proportions by the number of trials first.
feature_weights: [float, sequence]
Penalty weights. An intercept, or other unpenalized features are handled by setting those entries of feature_weights to 0. If feature_weights is a float, then all parameters are penalized equally.
trials : ndarray (optional)
Number of trials per response, defaults to ones the same shape as Y.
quadratic : regreg.identity_quadratic.identity_quadratic (optional)
An optional quadratic term to be added to the objective. Can also be a linear term by setting quadratic coefficient to 0.
ridge_term : float
How big a ridge term to add?
randomizer_scale : float
Scale for IID components of randomizer.
randomizer : str
One of [‘laplace’, ‘logistic’, ‘gaussian’]
- Returns
L : selection.randomized.lasso.lasso
-
static
coxph(X, times, status, groups, weights, quadratic=None, ridge_term=None, randomizer_scale=None)[source]¶ Cox proportional hazards LASSO with feature weights. Objective function is (before randomization)
\[\beta \mapsto \ell^{\text{Cox}}(\beta) + \sum_{i=1}^p \lambda_i |\beta_i|\]where \(\ell^{\text{Cox}}\) is the negative of the log of the Cox partial likelihood and \(\lambda\) is feature_weights. Uses Efron’s tie breaking method.
- Parameters
X : ndarray
Shape (n,p) – the design matrix.
times : ndarray
Shape (n,) – the survival times.
status : ndarray
Shape (n,) – the censoring status.
feature_weights: [float, sequence]
Penalty weights. An intercept, or other unpenalized features are handled by setting those entries of feature_weights to 0. If feature_weights is a float, then all parameters are penalized equally.
covariance_estimator : optional
If None, use the parameteric covariance estimate of the selected model.
quadratic : regreg.identity_quadratic.identity_quadratic (optional)
An optional quadratic term to be added to the objective. Can also be a linear term by setting quadratic coefficient to 0.
ridge_term : float
How big a ridge term to add?
randomizer_scale : float
Scale for IID components of randomizer.
randomizer : str
One of [‘laplace’, ‘logistic’, ‘gaussian’]
- Returns
L : selection.randomized.lasso.lasso
-
static
poisson(X, counts, groups, weights, quadratic=None, ridge_term=None, randomizer_scale=None)[source]¶ Poisson log-linear LASSO with feature weights. Objective function is (before randomization)
\[\beta \mapsto \ell^{\text{Poisson}}(\beta) + \sum_{i=1}^p \lambda_i |\beta_i|\]where \(\ell^{\text{Poisson}}\) is the negative of the log of the Poisson likelihood (half the deviance) and \(\lambda\) is feature_weights.
- Parameters
X : ndarray
Shape (n,p) – the design matrix.
counts : ndarray
Shape (n,) – the response.
feature_weights: [float, sequence]
Penalty weights. An intercept, or other unpenalized features are handled by setting those entries of feature_weights to 0. If feature_weights is a float, then all parameters are penalized equally.
quadratic : regreg.identity_quadratic.identity_quadratic (optional)
An optional quadratic term to be added to the objective. Can also be a linear term by setting quadratic coefficient to 0.
ridge_term : float
How big a ridge term to add?
randomizer_scale : float
Scale for IID components of randomizer.
randomizer : str
One of [‘laplace’, ‘logistic’, ‘gaussian’]
- Returns
L : selection.randomized.lasso.lasso
-
static
sqrt_lasso(X, Y, groups, weights, quadratic=None, ridge_term=None, randomizer_scale=None, solve_args={'min_its': 200}, perturb=None)[source]¶ Use sqrt-LASSO to choose variables. Objective function is (before randomization)
\[\beta \mapsto \|Y-X\beta\|_2 + \sum_{i=1}^p \lambda_i |\beta_i|\]where \(\lambda\) is feature_weights. After solving the problem treat as if gaussian with implied variance and choice of multiplier. See arxiv.org/abs/1504.08031 for details.
- Parameters
X : ndarray
Shape (n,p) – the design matrix.
Y : ndarray
Shape (n,) – the response.
feature_weights: [float, sequence]
Penalty weights. An intercept, or other unpenalized features are handled by setting those entries of feature_weights to 0. If feature_weights is a float, then all parameters are penalized equally.
quadratic : regreg.identity_quadratic.identity_quadratic (optional)
An optional quadratic term to be added to the objective. Can also be a linear term by setting quadratic coefficient to 0.
covariance : str
One of ‘parametric’ or ‘sandwich’. Method used to estimate covariance for inference in second stage.
solve_args : dict
Arguments passed to solver.
ridge_term : float
How big a ridge term to add?
randomizer_scale : float
Scale for IID components of randomizer.
randomizer : str
One of [‘laplace’, ‘logistic’, ‘gaussian’]
- Returns
L : selection.randomized.lasso.lasso
Notes
Unlike other variants of LASSO, this solves the problem on construction as the active set is needed to find equivalent gaussian LASSO. Assumes parametric model is correct for inference, i.e. does not accept a covariance estimator.
-
get_sampler()¶
-
randomize(perturb=None)¶ The actual randomization step.
- Parameters
perturb : ndarray, optional
Value of randomization vector, an instance of \(\omega\).
-
property
sampler¶ Sampler of optimization (augmented) variables.
-
selective_MLE(observed_target, target_cov, target_score_cov, level=0.9, solve_args={'tol': 1e-12})¶ - Parameters
observed_target : ndarray
Observed estimate of target.
target_cov : ndarray
Estimated covaraince of target.
target_score_cov : ndarray
Estimated covariance of target and score of randomized query.
level : float, optional
Confidence level.
solve_args : dict, optional
Arguments passed to solver.
-
set_sampler(sampler)¶
-
setup_sampler()¶ Setup query to prepare for sampling. Should set a few key attributes:
observed_score_state
observed_opt_state
opt_transform
-
solve()¶
-
polynomial_gaussian_sampler¶
-
class
selectinf.randomized.group_lasso.polynomial_gaussian_sampler(implied_mean, implied_covariance, initial_point, observed_score_state, log_cond_density, log_det, logdens_transform, selection_info=None, useC=False)[source]¶ Bases:
selectinf.randomized.query.affine_gaussian_sampler1-dimensional Gaussian density restricted to [0,infty) times a polynomial
-
__init__(implied_mean, implied_covariance, initial_point, observed_score_state, log_cond_density, log_det, logdens_transform, selection_info=None, useC=False)[source]¶ - Parameters
affine_con : selection.constraints.affine.constraints
Affine constraints
- initial_pointndarray
Feasible point for affine constraints.
- observed_score_statendarray
Observed score of convex loss (slightly modified). Essentially (asymptotically) equivalent to :math:`
abla ell(eta^*) +
Q(eta^*)eta^*` where \(eta^*\) is population minimizer. For linear regression, it is always \(-X^Ty\).
- log_cond_densitycallable
Density of optimization variables given score
- logdens_transformtuple
Description of how conditional mean of optimization variables depends on score.
- selection_infooptional
Function of optimization variables that will be conditioned on.
- useCbool, optional
Use python or C solver.
-
sample(ndraw)[source]¶ Sample from a Gaussian truncated at zero with our mean and covariance, but give weight based on self.log_det
- Parameters
ndraw : int
How long a chain to return?
-
selective_MLE(observed_target, target_cov, target_score_cov, init_soln, solve_args={'tol': 1e-12}, level=0.9)[source]¶ Selective MLE based on approximation of CGF.
- Parameters
observed_target : ndarray
Observed estimate of target.
target_cov : ndarray
Estimated covaraince of target.
target_score_cov : ndarray
Estimated covariance of target and score of randomized query.
init_soln : ndarray
Feasible point for optimization problem.
level : float, optional
Confidence level.
solve_args : dict, optional
Arguments passed to solver.
-
reparam_map(parameter_target, observed_target, target_cov, target_score_cov, init_soln, solve_args={'tol': 1e-12}, useC=True)[source]¶
-
coefficient_pvalues(observed_target, target_cov, score_cov, parameter=None, sample_args=(), sample=None, normal_sample=None, alternatives=None)¶ Construct selective p-values for each parameter of the target.
- Parameters
observed : np.float
A vector of parameters with shape self.shape, representing coordinates of the target.
parameter : np.float (optional)
A vector of parameters with shape self.shape at which to evaluate p-values. Defaults to np.zeros(self.shape).
sample_args : sequence
Arguments to self.sample if sample is None.
sample : np.array (optional)
If not None, assumed to be a sample of shape (-1,) + self.shape representing a sample of the target from parameters self.reference. Allows reuse of the same sample for construction of confidence intervals, hypothesis tests, etc.
alternatives : list of [‘greater’, ‘less’, ‘twosided’]
What alternative to use.
- Returns
pvalues : np.float
-
confidence_intervals(observed_target, target_cov, score_cov, sample_args=(), sample=None, normal_sample=None, level=0.9, initial_guess=None)¶ - Parameters
observed : np.float
A vector of parameters with shape self.shape, representing coordinates of the target.
sample_args : sequence
Arguments to self.sample if sample is None.
sample : np.array (optional)
If not None, assumed to be a sample of shape (-1,) + self.shape representing a sample of the target from parameters self.reference. Allows reuse of the same sample for construction of confidence intervals, hypothesis tests, etc.
level : float (optional)
Specify the confidence level.
initial_guess : np.float
Initial guesses at upper and lower limits, optional.
- Returns
intervals : [(float, float)]
List of confidence intervals.
Notes
Construct selective confidence intervals for each parameter of the target.
-
hypothesis_test(test_stat, observed_value, target_cov, score_cov, sample_args=(), sample=None, parameter=0, alternative='twosided')¶ Sample target from selective density using sampler with gradient map self.gradient and projection map self.projection.
- Parameters
test_stat : callable
Test statistic to evaluate on sample from selective distribution.
observed_value : float
Observed value of test statistic. Used in p-value calculation.
sample_args : sequence
Arguments to self.sample if sample is None.
sample : np.array (optional)
If not None, assumed to be a sample of shape (-1,) + self.shape representing a sample of the target from parameters. Allows reuse of the same sample for construction of confidence intervals, hypothesis tests, etc. If not None, ndraw, burnin, stepsize are ignored.
parameter : np.float (optional)
alternative : [‘greater’, ‘less’, ‘twosided’]
What alternative to use.
- Returns
pvalue : float
-
log_cond_density(opt_sample, target_sample, transform=None)¶ Density of opt_sample | target_sample
-
Functions¶
-
selectinf.randomized.group_lasso.debiased_targets(loglike, W, active_groups, penalty, sign_info={}, dispersion=None, approximate_inverse='JM', debiasing_args={})[source]¶
-
selectinf.randomized.group_lasso.form_targets(target, loglike, W, active_groups, penalty, **kwargs)[source]¶