New 1B Parameter AI Model Slashes Pretraining Costs to Just $1000 With Hierarchical Recurrent Architecture
A new 1B parameter AI model called HRM-Text is making foundation model pretraining accessible to nearly anyone, requiring up to 900x less data and 600x less compute than traditional methods, bringing full pretraining costs down to roughly $1000 using a hierarchical recurrent architecture with strong benchmark performance across GSM8k, MATH, and MMLU.