Meta Says Its JiTTests Caught 74 Serious Privacy Bugs Before Launch
Summary
Meta has moved its LLM-generated Catching JiTTests from research prototype into production infrastructure, generating failing tests that expose bugs in code diffs before they land. Meta says 589 engineers engaged with the system at a 60% true-positive rate, and it caught 74 serious privacy bugs in the first half of 2026, cutting privacy SEVs by 20%.
Key Points
- Meta runs Catching JiTTests overnight on spare capacity, targeting risky diffs submitted the previous day.
- Meta pairs its LLM-as-Judge ensemble with a rule-based assessor called RubFake for deterministic verdicts.
- Meta presents its paper, Just-in-Time Catching Test Generation at Meta, at FSE 2026.