The UK will establish the world’s first AI safety institute, Prime Minister Rishi Sunak has announced. The institute will be tasked with evaluating new artificial intelligence models to identify the opportunities and risks posed by the technology.
Sunak revealed plans for the institute on Thursday morning following the publication of two new discussion papers highlighting the possible dangers of the next generation of AI. Produced ahead of next week’s AI Safety Summit, which Sunak hopes will establish the UK as a leading voice in the international debate around AI safety, the papers take an in-depth look at the risks and opportunities of so-called frontier AI, and highlight threats to cybersecurity and the labour market.
Rishi Sunak announces UK AI safety institute
At a speech in London on Thursday, Sunak gave more detail on the UK’s stance on AI safety. The new institute, he said, would “carefully evaluate and test new types of AI so we can understand what new models are capable of”.
Sunak said that the institute was needed because “right now the only people testing the safety of AI are the very organisations developing it”, referring to AI labs such as OpenAI, Google Deepmind and Anthropic. The prime minister said that these developers “don’t even know what their models are capable of, so we shouldn’t rely on them marking their own homework. Only governments can properly assess the risks to national security”.
The work of the institute will be “made available to the world”, Sunak explained. This is “the right thing to do, morally and economically, for families and businesses up and down the country,” the prime minister said in his speech, adding: “The British people should have peace of mind that we’re developing the most advanced protections from AI of any country in the world.”
Tech Monitor has contacted the government’s Department of Science, Innovation and Technology for more information on the plans for the institute.
But reiterating that his government does not plan to introduce legislation around AI, Sunak said: “We believe in innovation, which is a hallmark of the British economy, and we want to encourage that, not stifle it. How can we write laws for something we don’t understand?”
This approach puts the UK at odds with the European Union, which has developed an overarching AI act, and other countries around the world.
Content from our partners
Reynolds Catering spurs innovation by upgrading its ERP to level up capabilities at scale
Zeelandia leverages AI to optimise precision, efficiency and pricing
How financial services companies can prepare for the EU’s DORA regulations
Sunak added that next week’s safety summit will be a chance to try and find consensus on how to approach frontier AI. Confirming that China had been invited to attend, he said it was important to “engage all the world’s AI powers” on the risks posed by advanced systems.
View all newsletters Sign up to our newsletters Data, insights and analysis delivered to you By The Tech Monitor team
Frontier AI’s risk to cybersecurity and the labour market
The government’s discussion papers set out these risks in detail. On cybersecurity, it says frontier AI is likely to “increase the scale and speed of attacks”, and “may result in systems that can act on the internet to perform their own cyberattacks autonomously”.
The report says: “Behaviours such as autonomous replication and self-improving exploit generation are of particular concern, and some work has started to look at
how good today’s models are at these behaviours.”
Frontier AI’s effect “on the overall balance between cyber offence and defence is uncertain”, the report adds, “as these tools also have many applications in improving the cybersecurity of systems and defenders are mobilising significant resources to utilise frontier AI for defensive purposes.”
It also warns: “In the future, we may see AI systems both conducting and defending against cyberattacks with reduced human oversight at each step.”
Elsewhere, it says disruption and displacement in labour markets “are one of the risks through which rapid advances in AI may affect citizens and reduce social welfare.” It highlights IT workers, alongside staff in the legal and financial services, as the job roles most at risk from frontier AI, but says it has “already begun to reduce the administrative burden of some roles and has the potential to accelerate this considerably including in areas such as teaching and medicine.”
Read more: Rishi Sunak hails quality of UK AI safety research
