f-Divergence DRO

Training customization

Chi2NNDRO and CVaRNNDRO inherit the neural training controls. Their fit(...) methods accept paired train_loader and val_loader arguments, a PyTorch optimizer, and an optional scheduler with scheduler_step="epoch" or "batch". Build the optimizer after any update_user_mode(...) call so that it updates the current network. The existing fit(X, y, ...) path remains available with an internal split and Adam optimizer.

Radius certificates

Chi2NNDRO.certify_radius(losses, candidate_weights, eta, epsilon=None) returns epsilon_cert. CVaRNNDRO.certify_radius(losses, candidate_weights, eta, alpha=None) returns alpha_cert; a larger tail fraction means a smaller ambiguity set. Both methods also return the certified empirical loss bound, the upper bound at the input parameter, and their gap, or None when the reduced parameter cannot be certified at the candidate loss. Pass full-data per-example losses and feasible weights for the final frozen network. See the certificate derivations and runnable example.

class dro.neural_model.fdro_nn.Chi2NNDRO(input_dim, num_classes, task_type='classification', model_type='mlp', device=device(type='cpu'), size=0.1, reg=0.1, max_iter=100)

Bases: BaseNNDRO

Chi-square Divergence-based Neural DRO Model.

Initialize Chi-square DRO neural model.

Parameters:

  • input_dim (int) – Input feature dimension \(d \geq 1\)

  • num_classes (int) –

    Output dimension:

    • Classification: \(K \geq 2\) (number of classes)

    • Regression: Automatically set to 1

  • task_type (str) –

    Learning task type. Supported:

    • 'classification': Cross-entropy loss

    • 'regression': MSE loss

  • model_type (str) –

    Neural architecture type. Supported:

    • 'mlp': Multi-Layer Perceptron (default)

    • linear

    • resnet

    • alexnet

  • device (torch.device) – Target computation device, defaults to CPU

  • size (float) – Chi-square divergence radius \(\rho \geq 0\) controlling distributional robustness. Zero uses the mean empirical loss. Larger values increase conservativeness. Defaults to 0.1.

  • reg (float) – Regularization coefficient. Defaults to 0.1.

  • max_iter (int) – Maximum Sinkhorn iterations. Defaults to 100.

Raises:

ValueError –

  • If size < 0

  • If reg ≤ 0

  • If max_iter < 1

update(config)
certify_radius(losses, candidate_weights, eta, *, epsilon=None)

Certify a smaller empirical \(\chi^2\) divergence radius.

losses are the final network’s per-example losses on one complete reference dataset. candidate_weights must be feasible at the input radius, and eta is any finite scalar in the upper bound. epsilon defaults to size. The calculation uses unregularized losses even when training used reg. Returns None if the gap is too large; otherwise returns a dictionary with epsilon_cert, the certified loss bound, the input upper bound, and their gap.

class dro.neural_model.fdro_nn.CVaRNNDRO(input_dim, num_classes, task_type='classification', model_type='mlp', device=device(type='cpu'), size=0.1, reg=0.1, max_iter=100)

Bases: BaseNNDRO

Initialize Conditional Value-at-Risk DRO neural model.

Parameters:

  • input_dim (int) – Input feature dimension \(d \geq 1\)

  • num_classes (int) –

    Output dimension:

    • Classification: \(K \geq 2\) (number of classes)

    • Regression: Automatically set to 1

  • task_type (str) –

    Learning task type. Supported:

    • 'classification': Cross-entropy loss

    • 'regression': MSE loss

  • model_type (str) –

    Neural architecture type. Supported:

    • 'mlp': Multi-Layer Perceptron (default)

    • linear

    • resnet

    • alexnet

  • device (torch.device) – Target computation device, defaults to CPU

  • size (float) – CVaR ratio \(1 \geq \rho \geq 0\) controlling distributional robustness. Smaller values increase conservativeness. Defaults to 0.1.

  • reg (float) – Regularization coefficient. Defaults to 0.1.

  • max_iter (int) – Maximum Sinkhorn iterations. Defaults to 100.

Raises:

ValueError –

  • If size < 0

  • If reg ≤ 0

  • If max_iter < 1

update(config)
certify_radius(losses, candidate_weights, eta, *, alpha=None)

Certify a smaller empirical CVaR ambiguity set.

losses are the final network’s per-example losses on one complete reference dataset. candidate_weights must be feasible at the input tail fraction, and eta is any finite scalar in the upper bound. alpha defaults to size. The calculation uses unregularized losses even when training used reg. Returns None if the gap is too large; otherwise returns a dictionary with alpha_cert, the certified loss bound, the input upper bound, and their gap. Larger alpha_cert means a smaller ambiguity set.