econirl.TDCCP

class econirl.TDCCP(n_states=90, n_actions=2, discount=0.9999, utility='linear_cost', se_method='asymptotic', method='semigradient', cross_fitting=True, robust_se=True, basis_dim=8, basis_type='polynomial', basis_include_rewards=False, basis_ridge=1e-08, basis_pinv_rcond=None, hidden_dim=64, num_hidden_layers=2, avi_iterations=20, epochs_per_avi=30, learning_rate=0.001, batch_size=8192, ccp_method='frequency', n_policy_iterations=1, verbose=False)[source]

Bases: object

Sklearn-style TD-CCP estimator for dynamic discrete choice models.

TD-CCP (Temporal Difference CCP) estimates utility parameters using neural network-based approximate value iteration combined with CCP decomposition. Instead of matrix inversion (as in standard CCP), it trains per-feature EV component networks via semi-gradient TD learning, then uses the learned components in a partial MLE for structural parameters.

This is particularly useful for large state spaces where matrix-based CCP methods are computationally infeasible.

Parameters:

n_states (int, default=90) – Number of discrete states (e.g., mileage bins).
n_actions (int, default=2) – Number of discrete actions (e.g., keep/replace).
discount (float, default=0.9999) – Time discount factor (beta).
utility (str or RewardSpec, default="linear_cost") – Utility specification. Pass "linear_cost" for the classic Rust bus model (u = -theta_c * s * (1-a) - RC * a), or a RewardSpec for custom features.
se_method (str, default="asymptotic") – Method for computing standard errors. Options: “robust”, “asymptotic”.
hidden_dim (int, default=64) – Number of hidden units per layer in the EV component networks.
num_hidden_layers (int, default=2) – Number of hidden layers in the EV component networks.
avi_iterations (int, default=20) – Number of approximate value iteration rounds.
epochs_per_avi (int, default=30) – Number of SGD epochs per AVI iteration.
learning_rate (float, default=1e-3) – Learning rate for neural network training.
batch_size (int, default=8192) – Mini-batch size for SGD training.
n_policy_iterations (int, default=3) – Number of NPL-style policy iterations.
verbose (bool, default=False) – Whether to print progress messages during estimation.
method (Literal['semigradient', 'neural'])
cross_fitting (bool)
robust_se (bool)
basis_dim (int)
basis_type (Literal['polynomial', 'encoded', 'tabular'])
basis_include_rewards (bool)
basis_ridge (float)
basis_pinv_rcond (float | None)
ccp_method (Literal['frequency', 'logit'])

Variables:

params (dict) – Estimated parameters after fitting. Keys are parameter names (e.g., “theta_c”, “RC”) and values are point estimates.
se (dict) – Standard errors for each parameter.
coef (numpy.ndarray) – Coefficients as a numpy array (sklearn convention).
log_likelihood (float) – Maximized log-likelihood value.
pvalues (dict) – P-values for each parameter (from Wald t-test).
policy (numpy.ndarray) – Estimated choice probabilities P(a|s) of shape (n_states, n_actions).
value (numpy.ndarray) – Estimated value function V(s) of shape (n_states,).
ev_features (numpy.ndarray or None) – Per-feature EV component values of shape (n_states, n_features). Shows how much of the continuation value comes from each structural feature. Available after fitting if the underlying estimator includes them in metadata.
converged (bool) – Whether the optimization converged.
reward_spec (RewardSpec) – The reward specification used for estimation.

Examples

>>> from econirl.estimators import TDCCP
>>> import pandas as pd
>>>
>>> df = pd.DataFrame({
...     "bus_id": [0, 0, 1, 1],
...     "mileage": [10, 20, 15, 30],
...     "replaced": [0, 0, 0, 1],
... })
>>>
>>> model = TDCCP(n_states=90, hidden_dim=64, avi_iterations=20)
>>> model.fit(df, state="mileage", action="replaced", id="bus_id")
>>> print(model.params_)

__init__(n_states=90, n_actions=2, discount=0.9999, utility='linear_cost', se_method='asymptotic', method='semigradient', cross_fitting=True, robust_se=True, basis_dim=8, basis_type='polynomial', basis_include_rewards=False, basis_ridge=1e-08, basis_pinv_rcond=None, hidden_dim=64, num_hidden_layers=2, avi_iterations=20, epochs_per_avi=30, learning_rate=0.001, batch_size=8192, ccp_method='frequency', n_policy_iterations=1, verbose=False)[source]

Initialize the TD-CCP estimator.

Parameters:

n_states (int, default=90) – Number of discrete states.
n_actions (int, default=2) – Number of discrete actions.
discount (float, default=0.9999) – Time discount factor (beta).
utility (str or RewardSpec, default="linear_cost") – Utility specification to use.
se_method (str, default="asymptotic") – Method for computing standard errors.
method (str, default="semigradient") – TD method: “semigradient” (fast closed-form, eq 3.5) or “neural” (AVI with neural networks, Algorithm 1).
cross_fitting (bool, default=True) – Use 2-fold cross-fitting (Algorithm 2) for valid inference.
robust_se (bool, default=True) – Compute locally robust standard errors (Section 4).
basis_dim (int, default=8) – Number of polynomial basis functions for semi-gradient method.
basis_type (str, default="polynomial") – Semi-gradient basis: “polynomial”, “encoded”, or “tabular”.
basis_include_rewards (bool, default=False) – Include reward features in the encoded semi-gradient basis.
basis_ridge (float, default=1e-8) – Ridge stabilization for the semi-gradient normal equation.
basis_pinv_rcond (float, optional) – Pseudoinverse cutoff for nearly singular semi-gradient bases.
hidden_dim (int, default=64) – Hidden units per layer in EV component networks.
num_hidden_layers (int, default=2) – Number of hidden layers in EV component networks.
avi_iterations (int, default=20) – Number of approximate value iteration rounds.
epochs_per_avi (int, default=30) – Number of SGD epochs per AVI iteration.
learning_rate (float, default=1e-3) – Learning rate for neural network training.
batch_size (int, default=8192) – Mini-batch size for SGD training.
ccp_method (str, default="frequency") – CCP estimation: “frequency” or “logit”.
n_policy_iterations (int, default=1) – Number of NPL-style policy iterations. Paper uses 1.
verbose (bool, default=False) – Whether to print progress messages.

fit(data, state=None, action=None, id=None, transitions=None, reward=None)[source]

Fit the TD-CCP estimator to data.

Parameters:

data (pandas.DataFrame or Panel or TrajectoryPanel) – Panel data with observations. When a DataFrame is passed, state, action, and id column names are required. When a Panel/TrajectoryPanel is passed, column names are ignored.
state (str, optional) – Column name for the state variable (required for DataFrame input).
action (str, optional) – Column name for the action variable (required for DataFrame input).
id (str, optional) – Column name for the individual identifier (required for DataFrame input).
transitions (numpy.ndarray, optional) – Pre-estimated transition matrix of shape (n_states, n_states). If None, transitions are estimated from the data.
reward (RewardSpec, optional) – Reward/utility specification. If provided, overrides the utility parameter passed at construction time.

Returns:

self – Returns self for method chaining.

Return type:

TDCCP

property reward_matrix_: ndarray | None

Structural reward matrix R(s,a) of shape (n_states, n_actions).

Computes the utility matrix from the fitted parameters and the feature specification. Returns None if the model has not been fitted.

summary()[source]

Generate a formatted summary of estimation results.

Returns:: Human-readable summary of the estimation.
Return type:: str

conf_int(alpha=0.05)[source]

Compute confidence intervals for parameters.

Parameters:: alpha (float, default=0.05) – Significance level. Returns (1 - alpha) confidence intervals.
Returns:: {param_name: (lower, upper)} confidence intervals.
Return type:: dict
Raises:: RuntimeError – If the model has not been fitted yet.

predict_proba(states)[source]

Predict choice probabilities for given states.

Parameters:: states (numpy.ndarray) – Array of state indices.
Returns:: Choice probabilities of shape (len(states), n_actions). Each row sums to 1.
Return type:: numpy.ndarray