Pyomo integration

[radka-j]

This issue is to discuss design choices related to converting AutoEmulate emulators to Pyomo compatible objects.

[radka-j]

From the discussion so far, we agreed that we want to have a method or object that takes in a TransformedEmulator from AutoEmulate (expecting a PyTorch NN) and makes it compatible with Pyomo. Below are some questions that we still need to decide on + suggestions.

1: Where should to_pyomo() live

We agreed that it is preferable if this method is not attached to a specific Emulator. It also should not be part of compare.py. Instead it should live as a standalone function or object inside it's own file (e.g., we could have something like autoemulate/convert/pyomo.py). The method should take in an instance of TransformedEmulator. So the use would look something like:

ae = AutoEmulate(…)
ae.compare()
model = ae.best_result() 
pyomo_obj = to_pyomo(model:TranformedEmulator, pyomo_vars)

2: How to handle ReLU

Maybe the Pyomo converter object could have an input argument for how to handle ReLUs (set to some default choice of smooth function to approximate it with). The user can then choose to override this if they have a different approximation in mind. Having this in the input args helps to surface that this is being done. Adding a warning message as well is a great idea.

3: Transforms

I think it is acceptable to only support the current AutoEmulate transforms and to raise an exception if user provided ones are used. We can leave a TODO note in the code indicating it is something we could choose to revisit at some later point.

[GhostyWave]

Following up on the discussion, following changes have been implemented.

A standalone pyomofy function has been added to convert a fitted AutoEmulate model into Pyomo-compatible algebraic expressions, enabling surrogate-based optimization with solvers like Ipopt.

Location

The function lives in its own module, seperate from any specific emulator class:

autoemulate/
  convert/
    pyomo.py   ← pyomofy lives here

Usage

from autoemulate.convert.pyomo import pyomofy

ae = AutoEmulate(...)
model = ae.best_result()
expressions = pyomofy(model, [pyo_model.x1, pyo_model.x2])
model_expr = expressions[0]

API

pyomofy(model: TransformedEmulator | Result, pyomo_vars, relu_beta: int = 50)

• model — a TransformedEmulator or a Result object returned by ae.best_result().
• pyomo_vars — list of Pyomo decision variables, in the same order as the training inputs.
• relu_beta — sharpness parameter for the Softplus approximation used when the model was trained with nn.ReLU. Only relevant for MLP models with ReLU activations. Higher values produce a tighter approximation to ReLU; lower values are smoother and more Ipopt-friendly. Defaults to 50.

Supported Emulators

• PolynomialRegression
• MLP

ReLU Handling

nn.ReLU is not differentiable at zero, which can cause gradient-based solvers like Ipopt to stall. When pyomofy encounters a ReLU layer, it automatically substitutes a Softplus approximation with a tunable sharpness parameter:

$$\text{Softplus}_\beta(x) = \frac{1}{\beta} \log(1 + e^{\beta x})$$

As $\beta \to \infty$, this converges to ReLU. The default relu_beta=50 provides a close approximation while remaining smooth. Users can override this via the relu_beta argument if needed.

Supported Transforms

Only the built-in AutoEmulate transforms are currently supported:

• StandardizeTransform — linear, fully expressible in Pyomo.
• PCATransform — linear, fully expressible in Pyomo.

VAETransform is not supported because its encoder/decoder are non-linear neural networks, making the inverse non-differentiable and impractical to embed in a Pyomo model. A NotImplementedError is raised if an unsupported transform is detected.

Supported Activations (MLP)

Activation	Handling
nn.ReLU	Auto-approximated via Softplus(beta)
nn.SiLU	Natively supported
nn.Tanh	Natively supported
nn.Sigmoid	Natively supported
nn.Softplus	Natively supported

Any other activation raises a TypeError listing the supported options.

Any furthur comments are appreciated.

[radka-j]

This all looks great, thanks @GhostyWave! I have no comments so please go ahead and open a PR with the implementation :)