About

dro 0.4.3 is a Python package for distributionally robust optimization in supervised learning. It supports linear and kernel models (including SVM, logistic regression, and linear regression), neural networks, and tree ensembles with several choices of ambiguity set. It also includes synthetic data generators, runnable examples, and tools for model evaluation.

The convex linear formulations use cvxpy with CLARABEL by default. Commercial solvers such as MOSEK are optional. See the installation guide for solver selection and the tutorials for formulations and usage.

Without specified, our DRO model is to solve the following optimization problem:

\[\min_{\theta} \max_{P: P \in U} E_{(X,Y) \sim P}[\ell(\theta;(X, Y))],\]

where \(U\) is the so-called ambiguity set and typically of the form \(U = \{P: d(P, \hat P_n) \leq \epsilon\}\) and \(\hat P_n := \frac{1}{n}\sum_{i = 1}^n \delta_{(X_i, Y_i)}\) is the empirical distribution of training samples \(\{(X_i, Y_i)\}_{i = 1}^n\). And \(\epsilon\) is the hyperparameter.

In release 0.4.3, dro supports:

Synthetic data generation

Python Module	Function Name	Description
dro.data.dataloader_classification	classification_basic	Basic classification task
	classification_DN21	Following Section 3.1.1 of [1]
	classification_SNVD20	Following Section 5.1 of [2]
	classification_LWLC	Following Section 4.1 (Classification) of [3]
dro.data.dataloader_regression	regression_basic	Basic regression task
	regression_DN20_1	Following Section 3.1.2 of [1]
	regression_DN20_2	Following Section 3.1.3 of [1]
	regression_DN20_3	Following Section 3.3 of [1]
	regression_LWLC	Following Section 4.1 (Regression) of [3]

Linear DRO models

Python Module	Class Name	Description
dro.linear_model.base	BaseLinearDRO	Base class for linear DRO methods
dro.linear_model.chi2_dro	Chi2DRO	Linear chi-square divergence-based DRO
dro.linear_model.kl_dro	KLDRO	Kullback-Leibler divergence-based DRO
dro.linear_model.cvar_dro	CVaRDRO	CVaR DRO
dro.linear_model.tv_dro	TVDRO	Total Variation DRO
dro.linear_model.marginal_dro	MarginalCVaRDRO	Marginal-X CVaR DRO
dro.linear_model.mmd_dro	MMD_DRO	Maximum Mean Discrepancy DRO
dro.linear_model.conditional_dro	ConditionalCVaRDRO	Y|X (ConditionalShiftBased) CVaR DRO
dro.linear_model.hr_dro	HR_DRO_LR	Holistic Robust DRO on linear models
dro.linear_model.wasserstein_dro	WassersteinDRO	Wasserstein DRO
	WassersteinDROsatisficing	Robust satisficing version of Wasserstein DRO
dro.linear_model.sinkhorn_dro	SinkhornLinearDRO	Sinkhorn DRO on linear models
dro.linear_model.mot_dro	MOTDRO	Optimal Transport DRO with Conditional Moment Constraints
dro.linear_model.or_wasserstein_dro	ORWDRO	Outlier-Robust Wasserstein DRO
dro.linear_model.group_dro	GroupDRO	Linear Group DRO minimizing the worst empirical group loss

NN DRO models

Python Module	Class Name	Description
dro.neural_model.base_nn	BaseNNDRO	Base model for neural-network-based DRO
dro.neural_model.fdro_nn	Chi2NNDRO	Chi-square Divergence-based Neural DRO Model
dro.neural_model.wdro_nn	WNNDRO	Wasserstein Neural DRO with Adversarial Robustness
dro.neural_model.hrdro_nn	HRNNDRO	Holistic Robust NN DRO
dro.neural_model.sinkhorn_nn	SinkhornNNDRO	Sinkhorn DRO for neural networks
dro.neural_model.groupdro_nn	GroupNNDRO	Neural Group DRO with adversarial group reweighting

Tree-Ensembles DRO models

Python Module	Class Name	Description
dro.tree_model.xgb	KLDRO_XGB	KL-DRO for XGBoost
dro.tree_model.xgb	CVaRDRO_XGB	CVaR-DRO for XGBoost
dro.tree_model.xgb	Chi2DRO_XGB	Chi2-DRO for XGBoost
dro.tree_model.lgbm	KLDRO_LGBM	KL-DRO for Light GBM
dro.tree_model.lgbm	CVaRDRO_LGBM	CVaR-DRO for Light GBM
dro.tree_model.lgbm	Chi2DRO_LGBM	Chi2-DRO for Light GBM

References

[1] John C. Duchi and Hongseok Namkoong. Learning Models with Uniform Performance via Distributionally Robust Optimization. The Annals of Statistics, 49(3):1378–1406, 2021. Institute of Mathematical Statistics.

[2] Aman Sinha, Hongseok Namkoong, and John C. Duchi. Certifying Some Distributional Robustness with Principled Adversarial Training. International Conference on Learning Representations (ICLR), 2018. OpenReview.net.

[3] Jiashuo Liu, Jiayun Wu, Bo Li, and Peng Cui. Distributionally Robust Optimization with Data Geometry. Advances in Neural Information Processing Systems, 35:33689–33701, 2022. Curran Associates, Inc.