# AI Incidents, Audits, and the Limits of Benchmarks  URL:: https://share.snipd.com/episode/036a7ea7-a435-4e59-8413-b249f4a8502c Author:: Practical AI  ## AI-Generated Summary None ## Highlights > **Verifying General Purpose Models Breaks Traditional Safety Frames** > - Frontier models are hard to assess because they're general-purpose, breaking safety processes that assume a specific operating context. > - You can't verify safety across all circumstances, so we must redefine how to encapsulate and verify assurance for customers. > Transcript: > Sean McGregor > A fundamental problem that we have, particularly at the frontier model level with things like OpenAI, Anthropic, Google's Gemini, and so forth, is it's very difficult to even know How safe your systems are because they're general purpose systems. This basically broke the safety frame. All the safety processes that we have are built around starting presumption of there's a specific context it's operating in, and you reason about its safety within that context. Well, if your context is just wildcard, star, everything, where does that leave you? Do you need to verify across all circumstances that it's going to be safe? Because the answer is going to be no. So how do you approach the task of verifying claims? ([Time 0:14:20](https://share.snipd.com/snip/a0fd342f-ade6-4492-81f8-6de3a054f537))