The Alan Turing Institute, the UK’s national institute for artificial intelligence, has issued a call for the government to establish clear boundaries on the use of generative AI, particularly in situations where the technology could make irreversible decisions without direct human oversight. In a report published on Friday night, the institute expressed concerns about the current unreliability and error-proneness of generative AI tools, cautioning against their use in high-stakes national security contexts.
The report emphasized the potential risks associated with excessive reliance on generative AI outputs, as users might unquestionably trust the results of large language models, leading to a reluctance to challenge AI-generated content. The institute urged a shift in mindset to account for unintended or incidental ways in which generative AI could pose national security risks.
Highlighting autonomous agents as a specific application of generative AI requiring close oversight, the report noted their potential to accelerate national security analysis by processing vast amounts of open-source data. However, critics pointed out the technology’s limitations in replicating human-level reasoning, raising concerns about its understanding of risk and potential failures.
To mitigate risks, the report suggested recording actions and decisions taken by autonomous agents, ensuring transparency and explainability. It recommended attaching warnings to every stage of generative AI output and documenting worst-case scenarios. The government, according to the report, may implement stringent restrictions in areas requiring “perfect trust,” such as nuclear command and control, and possibly extend them to policing and criminal justice.
The report also addressed the malicious use of generative AI, highlighting concerns about disinformation, fraud, and child sexual abuse material. Bad actors, it noted, are not constrained by the need for accuracy and transparency seen in governmental use, posing potential risks.
In dealing with AI-generated content, the report proposed government support for resistant watermarking features to prevent tampering, requiring collaboration with GPU manufacturers like NVIDIA and international coordination. The challenges involved in this approach were acknowledged as formidable.
On the regulatory front, the researchers recommended updating existing regulations as a more practical approach than introducing AI-specific rules, considering the slow pace of legislative efforts that could take several years to finalize. The report aligns with the UK government’s broader efforts to position the country at the forefront of responsible AI development while urging careful consideration of potential risks and regulatory frameworks.
