Our view on the emerging challenges facing frontier AI organisations, the technical practices most urgently needed to meet them, and how we will work with industry and others to develop them.
An update on the security changes we have recently made to our frontier AI evaluations, and the work that remains.
We introduce Transect, an open-source tool that helps evaluators follow an agent’s work, identify what to investigate closely, and check their interpretation against the transcript.
Frontier AI evaluations increasingly use open-ended, agentic, long-horizon tasks whose transcripts can span hundreds of pages of outputs and actions from complex multi-agent networks. The ...
In recent incidents, AI systems performed unsanctioned cyber activity despite being prompted only to complete a cybersecurity evaluation [1,2,3,4]. This includes AI systems engaging in supply-chain ...
We outline our approach to technical solutions for misuse and loss of control. To rapidly advance the science of AI safety and security, AISI aims to maximise the quality research tackling this ...
As LLM agents become more autonomous, they have more opportunities to cause harm. To mitigate risk, frontier AI developers, such as OpenAI and Anthropic, are already deploying agents under the watch ...
AI systems can generate vast amounts of outputs as they use tools or respond to queries across multiple turns. These outputs hold important information but making sense of them can be challenging.
Alignment research aims to ensure that AI systems reliably pursue intended goals and operate within specified constraints. As AI systems become more capable and are deployed in high-stakes ...