Utilizing a pre-training strategy for neural networks for Hysteresis modeling may be beneficial for generalization of the final network when fine-tuned to measured data
Simulations of magnetic hysteresis has in our case been done using the Jiles-Atherton model or the Preisach model with magnetic field responses fitted to the measured data.
While the simulations do not yield sufficient accuracy in field predictions for the SPS Magnets and subsequent Field compensation, they serve to generate a large and varied dataset to pre-train our neural networks.
Experiments
Prior experiments generated the current waveforms and simulated field response in the same notebook. New work has split the current waveform generation to dipole/notebooks/pretraining/generate_waveform.ipynb, which writes the current and time in a common parquet format for later simulation.
The simulation of the waveforms can be found in ~/cernbox/hysteresis/dipole/noteboos/pretraining/simulate_preisach.ipynb, which adds the columns B_sim_T and B_sim_eddy_T, where the latter includes an ODE simulation of Eddy current decay. In either case the calibration function is added to the simulation (since the simulation is only covering the field residual).
The new datasets are found in
~/cernbox/hysteresis/dipole/datasets/pretraining/pretrain_train_3h.parquet
~/cernbox/hysteresis/dipole/datasets/pretraining/pretrain_train_6h.parquet
~/cernbox/hysteresis/dipole/datasets/pretraining/pretrain_train_12h.parquet
~/cernbox/hysteresis/dipole/datasets/pretraining/pretrain_train_24h.parquet
~/cernbox/hysteresis/dipole/datasets/pretraining/pretrain_validation_1h.parquet
And defined for Lightning CLI in
~/cernbox/hysteresis/dipole/datasets/pretraining/pretrain_3h.yaml
~/cernbox/hysteresis/dipole/datasets/pretraining/pretrain_6h.yaml
~/cernbox/hysteresis/dipole/datasets/pretraining/pretrain_12h.yaml
~/cernbox/hysteresis/dipole/datasets/pretraining/pretrain_24h.yaml
The simulation for the 1h test set can be seen below.
, which can be compared against the MBA residuals from the measured magnets. Note that there are some “minor loops” seen at the plateaus, either at the beginning of the end. We can’t explain what they come from, but hypothetically from the injection plateau the current may ramp down slightly, causing a minor loop.
Experiments
Experiments started on Neptune: https://app.neptune.ai/o/lua/org/Pretrain-TFT-MBI/runs/table?viewId=standard-view with 12h dataset.
Training on ml4, and setting lightning==2.2.2 and jsonargparse==4.32.
We fit a TFT on the following config after hyperparameter tuning:
We reach a sub-gauss error in general, with small peaks caused by artifacts at the prediction horizons, capturing both hysteresis and static effects. This is equivalent to
| RMSE (e-4 T) | MAE (e-4 T) | SMAPE (%) | |
|---|---|---|---|
| TFT | 0.20 | 0.0325 | 0.46 |
| PF TFT | |||
| In normal dynamic range (units), where actual modeling range is only only on the residual, after subtracting the [[SPS Main Dipoles | MBI]] Calibration function. |
Here we see as well the estimation of the quantiles by the model, which are in the range for the 2nd and 98th percentiles, respectively.
Adaptive downsampling
We can apply Adaptive downsampling on the 5 datasets generated to train the model to learn the time step between each sample, and write the datasets to cernbox/hysteresis/dipole/datasets/pretraining/preisach_rdp for training, and keep the standard epsilon values , which keeps between 5-20% of the points, effective to a downsampling ratio between 5 and 20 (orange line below), and an RMSE below .
The dataset can be found in ~/cernbox/hysteresis/dipole/datasets/pretraining/preisach_rdp/pretrain_rdp.yaml
Datasets v2
Set idle current to 155 A, and injection plateaus to 161 and 301 A, reduce random perturbations at plateaus.
Datasets v3
Straight ramp down (no sampled subsections).
Datasets defined in
~/cernbox/hysteresis/dipole/datasets/pretraining_v3/pretrain_3h.yml
~/cernbox/hysteresis/dipole/datasets/pretraining_v3/pretrain_6h.yml
~/cernbox/hysteresis/dipole/datasets/pretraining_v3/pretrain_12h.yml
~/cernbox/hysteresis/dipole/datasets/pretraining_v3/pretrain_24h.yml
~/cernbox/hysteresis/dipole/datasets/pretraining_v3/pretrain_train_3h.parquet
~/cernbox/hysteresis/dipole/datasets/pretraining_v3/pretrain_train_6h.parquet
~/cernbox/hysteresis/dipole/datasets/pretraining_v3/pretrain_train_12h.parquet
~/cernbox/hysteresis/dipole/datasets/pretraining_v3/pretrain_train_24h.parquet
~/cernbox/hysteresis/dipole/datasets/pretraining_v3/pretrain_validation_1h.parquet