Curriculum Learning Stage 4

Fourth stage of Curriculum Learning where the model learns weak hysteresis behavior using Jiles-Atherton model for hysteresis.

Parameters

parameters = JilesAthertonParameters(  
    a = 1800.86,  # A/m  
    alpha = 0.0013740,  
    c = 0.0172,  
    k = 200,  # A/m  
    Ms = 1728184,  # A/m  
)

Training Configuration

For this stage, we reduce the weight decay to $10^{-3}$, unfreeze the attention layers, and decrease dropout to $0.1$. Additional steps to improve generalization could be to add more training data, and adding more LSTM layers (ensuring that the weight loading has strict=False).

The learning rate follows an OneCycleLR, with initial phase of 0.2, max or 1e-4, and total 40 epochs with 10618 steps per epoch.

Curriculum learning evaluation would suggest the simulation starts to diverge after half the data. The validation dataframe length is 1 500 000 points, so half is 750 000 points. For future steps we must cut off the rest of the points.

Related Concepts

• Curriculum Learning Stage 3 - Previous time-lag stage
• Curriculum Learning Stage 5 - Next stage with full hysteresis
• Jiles-Atherton model for hysteresis - Physics model used
• OneCycleLR - Learning rate scheduling strategy