Yale Team Finds Seven LLMs Keep Behavior When Vectors Are Swapped for Symbolic Role-Filler Approximations

A Yale-led study finds the vector representations inside large language models implicitly encode symbolic structure. Linguist Tom McCoy's team swapped model representations for role-filler approximations across seven LLMs in language, arithmetic, logic and coding, and behavior stayed largely unchanged. Editing internal vectors also shifted outputs, such as moving "clever" from doctor to lawyer.

• Coauthors Paul Soulos and Paul Smolensky of Microsoft and Tal Linzen of New York University join Tom McCoy on the preprint.
• The researchers replace the vectors with tensor product representations, which encode information as combinations of "fillers" and "roles," such as numerator and denominator in a fraction.
• Linzen says the results "suggest a path towards controlling" LLMs more effectively, because not understanding how they structure information could make them behave in unsafe ways.