Dan Petrovic Trains 1,000 Gemma 3 270M Weights to Lift Sorting to 97%

Training 1,000 of Gemma 3 270M's roughly 100 million weights lifted correct sorting of five two-digit numbers from about 1% to 97%, Dan Petrovic reports, repeating a NeurIPS 2025 study by Donoway and colleagues. Prompts alone topped out at 2 of 200. Multiplication stayed near 30% even with 100,000 weights, so the method cannot add absent skills.

• Petrovic reports 17 of the 20 prompt wordings score zero on sorting five two-digit numbers.
• Petrovic reports 30 trained weights make Gemma 3 270M return valid JSON 99% of the time.
• Seven of the ten most damaging weight changes sit in attention output matrices, Petrovic finds.