How do you secure an autonomous AI system?

How do you secure an autonomous AI system? How do you secure an autonomous AI system?
This entry is part 16 of 16 in the series Engineering explainers

As organisations give artificial intelligence greater freedom to act, they take on responsibility for decisions that people no longer direct step by step. Securing an autonomous AI system means accounting for how software pursues an objective once someone hands over a task. With that shift, protection will depend on connected layers of defence that keep independent action within boundaries.

The growing risks of autonomous AI systems

In electronics manufacturing and supply chains, granting software permission to modify production records or interact with connected equipment can have practical consequences. A mistaken action could disrupt work that depends on those resources, making the scope of delegated authority an important security consideration.

Even a clearly defined objective leaves room for interpretation when an agent chooses how to achieve it. For example, a request to resolve a scheduling conflict might prompt adjustments its operator never intended, particularly if the instructions leave acceptable methods unclear.

To help organisations manage such uncertainty, the National Cyber Security Centre offers interim practical advice while preparing formal guidance. Its recommendations treat built-in model protections as one part of a wider defence, with additional safeguards proportionate to the potential impact of failure.

Limiting what autonomous agents can access

Turning safeguards into working controls starts with separating an agent’s permissions from those of the employee who deploys it. According to Cequence, least-privilege controls should restrict a system to “only what its specific task requires, nothing more.”

The company describes researchers extracting credentials from agents, illustrating how confirming identity alone leaves sensitive information exposed. To close that gap, organisations should enforce the agreed boundaries whenever the software attempts to use a tool. Checking at that point allows teams to assess the actual request instead of relying on approval granted at startup.

Isolating agents within secure boundaries

While permission checks govern individual requests, Solo.io explains how sandboxing an agent prevents it from automatically inheriting the developer’s credentials or production access by placing it in a separate computing environment. The company says this “limits the blast radius when an agent acts autonomously,” meaning an unexpected action has fewer opportunities to disrupt other operations.

However, its account of isolation failures at major AI developers shows why the choice of boundary matters as much as creating one. For stronger separation, it favours hardware-supported virtual machines with their own operating systems, although teams using existing cloud infrastructure must still configure which connections they permit.

Detecting and containing abnormal agent behaviour

Even when software stays within its permissions, individually allowed actions can combine to produce an unintended outcome, which Darktrace addresses by examining behaviour in context. Its approach brings together specialised analytical methods to establish normal activity and assess connections among unusual events, providing security teams with more information than a single model’s judgement.

According to Nicole Carignan, Senior Vice President of Security and AI Strategy at Darktrace, securing an autonomous AI system requires “behavioural-based detection that can identify when AI begins to act outside its intended purpose,” paired with the ability to interrupt that specific risky action before it causes lasting harm. Precision matters wherever legitimate work depends on the same resources.

In practice, the response might block an unusual data transfer while leaving unrelated activity running, provided the monitoring system can distinguish between them. An explanation of why it intervened then gives reviewers a starting point for investigating what happened and deciding whether further restrictions are necessary.

Why human oversight still matters

While automated safeguards can interrupt concerning activity, Omdena places broader decisions about acceptable autonomy in people’s hands. Approval before every action offers one option, but routine work may warrant supervision within agreed boundaries instead. The consequences of an error should determine which arrangement fits.

Making that choice is only a starting point for effective oversight, since reviewers also need clear information and authority to intervene. The platform identifies “oversight design, not model quality” as a growing constraint as systems gain more autonomy, a concern reinforced by its cited surveys, which suggest governance maturity often lags adoption.

The road ahead for autonomous AI security

As autonomous AI takes on more demanding work, organisations will need evidence that their security arrangements remain fit for purpose. Lessons from testing and day-to-day use can inform those judgements as capabilities change. Progress will depend on turning that experience into better-informed deployment decisions, with confidence grounded in observed behaviour rather than assumptions about what a system should do.

Engineering explainers

How to choose the right rack architecture for non-hyperscale data centres

Engineering explainers

How to maximise battery life in wearable medical devices

As wearables become more common, more patients and providers expect incremental functionality, performance and longevity improvements.

How stream data processing enhances smart grid management

Global electronics has more information to process than it knows how to handle. Data scientists are engineering next-generation strategies to parse it all.

How to secure hardware against side-channel attacks

With the growing demand for faster, more connected devices, companies need stronger hardware security than ever. Encryption algorithms and secure protocols protect sensitive data, but lesser-known security threats – like side-channel attacks – can easily bypass these components.

Addressing 3 workforce challenges in electronics manufacturing

Electronics manufacturers in the UK face many workforce challenges also seen in other parts of the world. Tackling them is essential since the industry’s goods are in perpetually high demand. An awareness of the most pressing issues is also necessary before affected parties can begin mitigating the effects.

How to secure data in transit

How to secure data in transit

Today, data is constantly moving, with people worldwide continuously sending emails, uploading files to the Cloud and sharing information between devices. This is known as data in transit, which is often vulnerable to attacks.

Wireless communication challenges in Industrial IoT

Wireless communication challenges in Industrial IoT

4 cybersecurity challenges when deploying Edge AI

4 cybersecurity challenges when deploying Edge AI

How to reduce EMI in implantable medical devices

How to reduce EMI in implantable medical devices

5 common security flaws in embedded Linux systems

How are data centres addressing their water consumption problem?

How are data centres addressing their water consumption problem?

Overcoming 5 technical challenges of AI in medical imaging

Overcoming 5 technical challenges of AI in medical imaging

How to choose the right processor for AI development

How to choose the right processor for AI development

How to secure radio communications to resist electronic attacks

How to secure radio communications to resist electronic attacks

How to detect shadow AI use in enterprises

How to detect shadow AI use in enterprises

How to choose the right rack architecture for non-hyperscale data centres

How to choose the right rack architecture for non-hyperscale data centres

How do you secure an autonomous AI system?

How do you secure an autonomous AI system?

Keep Up to Date with the Most Important News

By pressing the Subscribe button, you confirm that you have read and are agreeing to our Privacy Policy and Terms of Use
Previous Post

Keysight introduces 14-bit low-profile oscilloscope for high-density, synchronised test systems