Prefrontal-AI Releases Apache-2.0 Code Claiming 22.7x Faster Attention at 10M Tokens
Summary
Prefrontal-AI publishes Apache-2.0 code for Latent Sparse Attention, which clusters keys in a learned latent space so each query scores cluster centroids instead of every key. The authors report one attention layer runs 22.7x faster in prefill and 4.6x in decode on GLM-5.3-Flash at 10M tokens, with accuracy close to the original model.
Key Points
- Prefrontal-AI reports its native N-LSA trains 1.6x faster per layer at 128k tokens than full attention.
- The server requires the --disable-prefill-cuda-graph flag because LSA picks prefill keys on the host, which CUDA graphs cannot capture.
- Distilled maps for Qwen3.8-27B load from release assets tagged d-lsa-weights-v1 under models/maps.