Apple's M5 Max MacBook Pro, with 128GB unified memory, is enabling developers to run massive 70-billion-parameter AI models locally for free as cloud AI inference costs spiral out of control, with some startups reporting costs tripling in just three months.
Apple's MLX framework now supports the M5 chip's Neural Accelerators, delivering a massive 4x speedup in AI response times and up to 27% faster token generation, powered by the M5's 153GB/s memory bandwidth, enabling MacBook Pro users to run powerful large language models locally with ease.
A deepening AI infrastructure crisis is forcing companies like Anthropic to actively degrade their own products to conserve compute, as surging demand from agentic tools outpaces supply — and with DRAM constraints expected to cap growth at 15GW until 2027, experts warn inference shortages will worsen through 2026 and 2027, …
MIT researchers unveil Attention Matching, a groundbreaking KV cache compression technique that slashes large language model memory usage by up to 50x in seconds, delivering near-identical accuracy without the slow, compute-heavy optimization methods that have long bottlenecked AI deployment.
Apple's 18-core M5 Max stuns in Geekbench 6 benchmarks, outscoring AMD's 96-core Threadripper Pro, though experts caution the test favors fewer, faster cores and doesn't scale beyond 32 threads — still, the chip's 29,644 multi-core score and 614 GB/s memory bandwidth signal a major leap forward for Apple silicon.
Red Hat is making major strides in open AI infrastructure, emerging as the third-highest global contributor to PyTorch while driving innovations in multi-cloud inference, distributed model serving, and hardware-agnostic AI deployment through projects like vLLM and llm-d.
SRAM-centric AI chips are challenging GPU dominance as NVIDIA's $20B Groq IP licensing deal and Cerebras's 750 MW OpenAI partnership signal a major shift toward faster, lower-latency accelerators purpose-built for AI inference workloads.
Arm's technology, found in 99% of smartphones worldwide, is revolutionizing on-device AI by bringing real-time generative features like creative style transfer at 30 frames per second to mid-range and entry-level devices, with a TECNO collaboration and KleidiAI optimizations delivering 30% faster AI compute across all price ranges.
Nvidia CEO Jensen Huang signals a major shift in private AI investment strategy, confirming the company's $30 billion stake in OpenAI — part of a massive $110 billion funding round — will likely be its last before OpenAI's anticipated IPO later this year, with a separate $10 billion Anthropic investment …
Alibaba debuts two AI smartglasses at MWC — the Qwen Glasses S1 and G1 — boasting a Snapdragon AR1 chip, 12MP camera, real-time translation, and a swappable dual-battery system, though the monochromatic green HUD display raises questions about whether the hardware can keep pace with full-color competitors.