MMD DRO

MMD_DRO fits a linear predictor while a kernel defines the MMD ambiguity set on joint feature–target samples. The default kernel is Gaussian RBF; Laplacian, polynomial, and linear kernels are also available.

fit(X, y, accelerate=True) (the default) approximates the kernel with a Nyström feature map fitted once on the sampled support, then transforms that support in parallel batches. It enforces the loss constraints on at most 5,000 sampled support points. Set n_components through update_kernel({"n_components": ...}) to choose the feature-map rank; the default is the smaller of 100 and the support size. With accelerate=False, fitting instead uses the full kernel Gram matrix and constraints for every sampled support point. In the accelerated mode, accelerated_regularizer="feature_coefficients" is the default and uses the feature-coefficient norm corresponding to the approximate RKHS penalty. "support_values" retains the legacy penalty on sampled function values, which differs from the RKHS penalty. Nyström approximation and constraint subsampling still distinguish accelerated fitting from the full formulation.

class dro.linear_model.mmd_dro.MMD_DRO(input_dim, model_type='svm', fit_intercept=True, solver='CLARABEL', sampling_method='bound', kernel='rbf', kernel_gamma=None, kernel_degree=2, kernel_coef0=1.0)

Bases: BaseLinearDRO

MMD-DRO (Maximum Mean Discrepancy - Distributionally Robust Optimization) Implementation with flexible sampling methods and model types.

Reference: <https://arxiv.org/abs/2006.06981>

Initialize MMD-DRO with kernel-based ambiguity set.

Parameters:

  • input_dim (int) – Dimension of input features. Must match training data.

  • model_type (str) –

    Base model type. Supported:

    • 'svm': Support Vector Machine (hinge loss)

    • 'logistic': Logistic Regression (log loss)

    • 'ols': Ordinary Least Squares (L2 loss)

    • 'lad': Least Absolute Deviation (L1 loss)

  • sampling_method (str) –

    Supported:

    • 'bound'

    • 'hull'

  • kernel (str) – Kernel defining the MMD ambiguity set. Supported values are 'rbf' (default), 'laplacian', 'polynomial' (or 'poly'), and 'linear'.

  • kernel_gamma (float or None) – Positive kernel scale for RBF, Laplacian, and polynomial kernels. If None, RBF and Laplacian use a robust median-distance heuristic, while polynomial uses 1 / (input_dim + 1).

  • kernel_degree (int) – Positive integer degree of the polynomial kernel. Defaults to 2.

  • kernel_coef0 (float) – Non-negative independent term in the polynomial kernel. Defaults to 1.0.

  • fit_intercept (bool)

  • solver (str)

Raises:

ValueError –

  • If model_type not in supported list

  • If input_dim ≤ 0

  • If sampling_method is invalid

  • If kernel or its hyperparameters are invalid

Example:
>>> model = MMD_DRO(input_dim=128, model_type='svm', kernel='rbf')
>>> model.sampling_method = 'hull' 
>>> model.eta = 0.5
update_kernel(config)

Update the MMD kernel using the package’s kernel-update convention.

metric is accepted as an alias for kernel; degree and coef0 are accepted as aliases for their kernel_ counterparts. n_components=None restores the default Nyström rank selection; fit(..., accelerate=False) selects the full kernel matrix.

Return type:

None

Parameters:

config (Dict[str, Any])

update(config)

Update MMD-DRO model configuration.

Parameters:

config (Dict[str, Any]) –

Configuration dictionary containing optional keys:

  • eta (float):

    MMD radius controlling distributional robustness. Must satisfy \(\eta > 0\). Defaults to current value.

  • sampling_method (str):

    Ambiguity set sampling strategy. Valid options:

    • 'bound': Sample uniformly in [-1, 1] for features and targets

    • 'hull': For regression, sample features in the pooled observed feature range and targets in their observed range. For classification, sample features in [-1, 1] and labels in {-1, +1}. Both fitting paths use these same sampling regions.

  • n_certify_ratio (float):

    Ratio of certification samples to training data size. Must satisfy \(0 < ext{ratio} \leq 1\). Defaults to current ratio.

  • kernel (str):

    MMD kernel. One of 'rbf', 'laplacian', 'polynomial'/'poly', or 'linear'.

  • kernel_gamma (float or None):

    Positive kernel scale. None selects the kernel-specific data-driven default.

  • kernel_degree (int):

    Positive polynomial degree. Defaults to 2.

  • kernel_coef0 (float):

    Non-negative polynomial offset. Defaults to 1.0.

Raises:

ValueError –

  • If eta is non-positive

  • If sampling_method not in {‘bound’, ‘hull’}

  • If n_certify_ratio ∉ (0, 1]

  • If config contains unrecognized keys

Return type:

None

Example:
>>> model = MMD_DRO(input_dim=10, model_type='svm')
>>> model.update({
...     'eta': 0.5,              
...     'sampling_method': 'hull',
...     'kernel': 'polynomial',
...     'kernel_degree': 2
... })
predict(X)

Predict with the linear model; kernel only defines the MMD set.

Return type:

ndarray

Parameters:

X (ndarray)

load(config)

Load linear MMD-DRO parameters independently of the MMD kernel.

Return type:

None

Parameters:

config (Dict[str, Any])

evaluate(X, y, fast=True)

Evaluate the linear predictor independently of the MMD kernel.

Return type:

float

Parameters:
fit(X, y, accelerate=True, accelerated_regularizer='feature_coefficients')

Fit MMD-DRO using a Nyström approximation or the full kernel matrix.

Parameters:

  • X (numpy.ndarray) – Training feature matrix of shape (n_samples, n_features). Must satisfy n_features == self.input_dim.

  • y (numpy.ndarray) –

    Target values of shape (n_samples,). Format requirements:

    • Classification: ±1 labels

    • Regression: Continuous values

  • accelerate (bool) – If True (default), fit a Nyström feature map on the sampled support and transform it in parallel batches; constrain at most 5,000 sampled support points. If False, use the full kernel Gram matrix and constrain every sampled support point.

  • accelerated_regularizer (str) – Regularizer used with the Nyström feature map. The default, "feature_coefficients", uses the feature-space norm corresponding to the unaccelerated RKHS penalty. "support_values" retains the legacy penalty on function values at the sampled support points. This setting has no effect when accelerate=False.

Returns:

Dictionary containing trained parameters:

  • theta: Weight vector of shape (n_features,)

Return type:

Dict[str, Any]