Inference time of a model with 6m parameters on an unloaded TFT, 800 context length, 400 target length, ndim 256
Compiled model, autoregressive. Longer prediction times are when there are 2 sequences that are predicted (cycle divided into 2 chunks).
SPS.USER.LHCPILOT: mean 71.80ms | std 5.20ms | max 90.00ms | min 66.94ms | N 23
SPS.USER.MD1: mean 37.96ms | std 2.60ms | max 47.10ms | min 35.39ms | N 136
SPS.USER.SFTPRO3: mean 69.70ms | std 3.63ms | max 85.13ms | min 65.65ms | N 131
SPS.USER.ZERO: mean 36.44ms | std 1.82ms | max 39.46ms | min 34.27ms | N 9
SPS.USER.MD3: mean 38.24ms | std 1.50ms | max 41.18ms | min 36.21ms | N 9
SPS.USER.AWAKE1: mean 37.28ms | std 1.97ms | max 43.51ms | min 35.42ms | N 32
SPS.USER.HIRADMT1: mean 68.51ms | std 2.82ms | max 78.37ms | min 65.92ms | N 24