Hyperparameters
The following hyperparameters are fixed for modeling the MBIs, and other parameters like architecture can be varied.
downsample: 20- 1 kHz → 50 Hz, matching the frequency of the 2024-10-14 - FGC Real-time channel correction
freq_cutoff: 25- Cutoff frequency set to half of the sampling frequency, since higher harmonics will not be visible anyways
ctxt_seq_len: 1020- Effective context length 20400 ms, equivalent to 1 LHC1
tgt_seq_len: 540- Effective prediction length 10800 ms, equivalent to 1 SFTPRO1
min_ctxt_seq_len: 180- Effective context length 3600 ms, equivalent to 1 MD1
min_tgt_seq_len: 180- Effective context length 3600 ms, equivalent to 1 MD1
Results
Training the model on Version 3 and evaluating on its validation and tests sets, we get the following metrics:
Validation set
| Model | MSE | MAE | RMSE | MAPE | SMAPE |
|---|---|---|---|---|---|
| TFTMBI-5 | 1.91e-09 | 2.23e-05 | 4.37e-05 | 9.32e-05 | 9.32e-05 |
| TFTMBI-7 | 3.95e-10 | 6.97e-06 | 1.99e-05 | 3.28e-05 | 3.28e-05 |
Test set 1
| Model | MSE | MAE | RMSE | MAPE | SMAPE |
|---|---|---|---|---|---|
| TFTMBI-5 | 2.31e-09 | 8.59e-06 | 4.80e-05 | 4.58e-05 | 4.58e-05 |
| TFTMBI-7 | 4.25e-09 | 1.47e-05 | 6.52e-05 | 7.38e-05 | 7.38e-05 |
Test set 2
| Model | MSE | MAE | RMSE | MAPE | SMAPE |
|---|---|---|---|---|---|
| TFTMBI-5 | 1.10e-08 | 6.70e-05 | 1.05e-04 | 2.52e-04 | 2.53e-04 |
| TFTMBI-7 | 2.12e-09 | 1.50e-05 | 4.60e-05 | 7.20e-05 | 7.20e-05 |
- Evaluate TFT autoregressively, and cycle-by-cycle [priority:: high] [due:: 2025-02-20] [completion:: 2025-02-26]
- Train TFT on MBI data on different quantiles [priority:: low] [completion:: 2025-02-27]
- Train TFT from pre-trained model [due:: 2025-02-21] [priority:: medium] [start:: 2025-02-21] [completion:: 2025-05-19]