Anthropic has built a tool called the Jacobian lens that offers an unprecedented view into the internal reasoning of its Claude large language model. The technique reveals how the AI puzzles over concepts when processing queries or tasks. Findings range from mundane to unnerving, according to the company.
The breakthrough provides a clearer glimpse into what happens inside LLMs, a domain often described as a black box. This visibility could help researchers understand why models produce certain outputs or make errors. Anthropic's work focuses on the model's hidden processing spaces.
Using the Jacobian lens, the team observed Claude engaging with concepts in ways that were not previously detectable. The tool maps how the model's internal states shift during reasoning. This represents one of the most detailed looks at LLM cognition to date.
The findings may influence how AI systems are designed for transparency and safety. Understanding internal reasoning could help mitigate risks from unexpected model behavior. Anthropic has not yet released the tool publicly for external validation.
External researchers have called for independent verification of Anthropic's methods. Some experts caution that the technique may not fully capture the complexity of model reasoning.