Inferact Open-Sources TPU v7 Megakernels, Claims Kimi K3 Tops 700 Tokens/s
Inferact open-sources TPU v7 megakernels and claims its Kimi K3 implementation surpasses 700 tokens per second with speculative decoding on 16 TPU v7 chips, using one Pallas kernel to fuse all 92 MoE layers.