Been currently busy training another gemma 4 31B, did about 12 run's so far, playing with swa value, rope, turn masking, so far 2048 seems to be the sweet spot for training at 8k ctx. should be releasing a new model in about 2-3 weeks, training on local 3090s is not very fast =p.