We are now the AI Security Institute
Please enable javascript for this website.
A
A
About

Our mission is to equip governments with a scientific understanding of the risks posed by advanced AI.

The AI Security Institute is a research organisation within the UK Government’s Department for Science, Innovation, and Technology.

Organisation

The AI Security Institute is a research organisation within the UK government's Department of Science, Innovation and Technology.

We are working to:

  • Test advanced AI systems and inform policymakers about their risks;
  • Advance research into solutions and mitigations;
  • Foster collaboration across companies, governments, and the wider research community to mitigate risks and advance publicly beneficial research; and
  • Strengthen AI development practices and policy globally.

To deliver on these ambitious goals, we designed the Institute as a startup in the government, combining the authority of government with the expertise and agility of the private sector.

Our team brings experience from across the public and private sectors  

  • Our Chief Technology Officer Jade Leung is also the Prime Minister's AI Advisor, and she previously led the Governance team at OpenAI.  
  • Our Chief Scientist Geoffrey Irving and Research Directors Yarin Gal and Chris Summerfield have led teams at OpenAI, Google DeepMind and the University of Oxford.  
  • Our Chair Ian Hogarth brings experience as a tech investor and entrepreneur.  
  • Our advisory board comprises national security and machine learning leaders, such as Yoshua Bengio.
  • Our 100+ technical staff bring experience from leading industry, academic and nonprofit labs.
  • Our policy, operations, and strategy teams bring experience from No10, the national security community, and many of the UK's best startups and companies.
View careers

We are backing our team with the resources they need to move fast

  • £66m in funding per financial year for the organisation‍‍
  • Privileged access to top AI models from leading companies and countless collaborations with external research groups
  • Priority access to over £1.5 billion of compute in the UK’s AI Research Resource and exascale supercomputing programme 

Research

Evaluations

A core driver of our work is the belief that governments need to understand advanced AI to inform policy decisions and keep the public safe and secure. This is why we rigorously evaluate leading AI systems before and after they are launched.

Our evaluations areas include:

  • Misuse: How much models could assist with dual-use cyber, chemical and biological attacks
  • Safeguards: How effective safety and security features of advanced AI systems are against attempts to circumvent them
  • Autonomy: How well models could conduct AI research & development, autonomously make copies of themselves, interact with and manipulate humans and evade human intervention  
  • Human Influence: How AI systems could influence humans and reduce individual autonomy
  • Societal Resilience: How can we make society more resilient to AI risks

By evaluating these risks now, we can help governments assess their significance and get ahead of them.

We have open-sourced our testing framework Inspect so the research community can use and build upon our work.   

‍

Solutions

We also believe governments have a key role to play in advancing the research field. This is why all our research teams work on not only evaluations, but also solutions. We both research solutions in-house, and we are mobilizing >£15 million in grants to accelerate the field.

Setting the global standard

The UK has driven the global conversation on AI governance. We have already shaped how companies and other governments are navigating the technology. To date, we have:

AI Safety Summit
We contributed to the first AI Safety Summit hosted by the UK at Bletchley Park and the follow-up summit hosted by the Republic of Korea in Seoul. The summits are bringing together world leaders, top AI companies and civil society to make unprecedented commitments to mitigate risks.
Partnered with the US AI Safety Institute
We partnered with the US AI Safety Institute to jointly test advanced AI models, share research insights, share model access, and enable expert talent transfers.
International AI Safety Report
We commissioned Yoshua Bengio to chair the International Scientific Report on the Safety of Advanced AI, an evidence-based report on the state of the science of advanced AI safety.