In [1]:
import numpy as np
import escape as esc

esc.require("0.9.8")

Loading material database from /home/dkor/Data/Development/workspace_escape/escape-core/python/src/escape/scattering/../data/mdb/materials.db

Multi-layer Perceptron regressor¶

In this notebook we demonstrate how to use Multilayer Perceptor Regressor for batch processing of experimental data. If you are not familiar with neural networks and particularly with multi-layer regression, please find a brief overview with some details here:

https://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html

There are several very popular frameworks for deep learning, like PyTorch, Tensorflow, Keras, etc. In this example, we'll use PyTorch through the skorch wrapper, which provides a scikit-learn compatible API. This wrapper is responsible for preparing training and testing data in terms of the ESCAPE framework and its parameterized objects. The prepared training and testing data is provided for further analysis, optimization and prediction to a wrapped regressor instance.

The skorch wrapper provides a scikit-learn compatible interface, requiring the following key methods:

  1. fit(X, y) for optimization
  2. score(X, y) for score calculation
  3. predict(X) for prediction

Additionally, there are methods which are responsible for noisifying and normalizing the training data. These can be provided when creating the regressor wrapper through the data_noisifiers and data_normalizers parameters.

Tip: Your generated training and experimental data which you are using for prediction of parameters are supposed to have the same coordinates. If this is not the case, you probably should use interpolation.

In the example below we use a model for a Ni/Si/Ni trilayer on a Si substrate. The regressor requires a model or modelstack object, so we need preliminary experimental data which is generated. The coordinates array of the preliminary experimental data will be used by the regressor wrapper to generate training and test data.

After optimization, the regressor wrapper can be applied to real experimental data for prediction of parameters and further optimization if required.

As usual, we start with the parameters and model description.

In [2]:
# Mass densities of Ni and Si

thknNi1 = esc.par("Ni1 Thkn", 20, units="nm", userlim=[15, 25], trained=True)
roughNi1 = esc.par("Ni1 Roughness", 1.5, units="nm", userlim=[1.0, 2.0], trained=True)
thknNi2 = esc.par("Ni2 Thkn", 20, units="nm", userlim=[15, 25], trained=True)
roughNi2 = esc.par("Ni2 Roughness", 1.5, units="nm", userlim=[1.0, 2.0], trained=True)
thknSi = esc.par("Si Thkn", 8, units="nm", userlim=[5, 10], trained=True)
roughSi = esc.par("Si Roughness", 0.2, units="nm", userlim=[0, 0.5], trained=True)

# we also add a background parameter
B = esc.par("Background", 0, units="", scale=1e-6, userlim=[0, 10])
In [ ]:
LayNi1 = esc.layer(
    "Layer: Ni1", material="Ni", thkn=thknNi1, rough=roughNi1, bydensity=True
)
LayNi2 = esc.layer(
    "Layer: Ni2", material="Ni", thkn=thknNi2, rough=roughNi2, bydensity=True
)
LaySi = esc.layer(
    "Layer: Si", material="Si", thkn=thknSi, rough=roughSi, bydensity=True
)
Sub = esc.substrate(
    "Substrate: Si", material="Si", rough=0.0, bydensity=True
)  # roughness is zero

sample = esc.multilayer("Sample", formula="LayNi1/LaySi/LayNi2//Sub", globals=globals())

Now we create calculation kernel for specular reflectivity

In [4]:
# we generate specrefl functor and add background to it
Qz = esc.var("qz")
src = esc.xrays(wavelength=0.154, units="nm")
Rf = esc.specrefl("Specrefl", Qz, sample, "matrix", source=src) + B
R = esc.kernel("Kernel", Rf, True)

Now we can generate experimental data. We also add some poisson noise.

In [5]:
qz = np.linspace(0.001, 2.5, 1000).copy()
# scaling coefficient for intensities. Smaller values will produce more noisy data
I0 = 1e8

y = np.empty(qz.shape)
R(qz, y)

y = np.random.poisson(y * I0) / I0
err = np.sqrt(y) / np.sqrt(I0)

dobj = esc.data("Ni/Si/Ni", qz, y, err, copy=True)
mobj = esc.model("Model: Ni/Si/Ni", R, dobj, residuals_scale="none", weight_type="data")

mobj.show().config(xlabel="Q[nm⁻¹]", ylabel="I/I0", ylog=True)
Out[5]:

Below we create a regressor wrapper. As an actual regressor instance we are going to use MLPRegressor from sklearn library. Our generated training data will be noisified and logarithmically normalized. The simulated specular reflectivity curve is usually normalized, thus, scale parameter is required to get a realsitic poisson noise.

We are ready to create regressor instance and its wrapper.

In [6]:
import torch
from escape.utils.regressor.torch import torch_regressor
from escape.utils.regressor import LogScaler
from skorch.callbacks import EarlyStopping

early_stopping = EarlyStopping(patience=50, threshold=1e-5)

rg = torch_regressor(
    "",
    [mobj],
    nsamples=300000,
    ntests=10000,
    data_normalizers=[LogScaler()],
    max_epochs=500,
    lr=1e-3,
    batch_size=128,
    optimizer=torch.optim.AdamW,
    verbose=0,
    callbacks = [early_stopping],
    device="cuda",
)
rg.show().config_model(ylog=True)
Out[7]:

We have trained our regressor. Let's check now how prediction of parameters works for some user provided data. You can run the next cell several times to see how good prediction works for the data out of training set.

In [8]:
# Since we do not have a real experimental data, we are going to generate
# the data object for the randomly changed model parameters
rg.shake()
# create data object with simulated array as intensities
y = np.asarray(mobj.simulation)
y = rg.noisify_data(y)
dobj = esc.data("Ni/Si/Ni Generated", qz, y, np.zeros_like(y), copy=True)
# run prediction
rg([dobj])
mobj.show().config(ylog=True)
Out[8]:
In [ ]:
In [ ]: