Insurers and Security Leaders Assess Risks From Autonomous AI Systems

Insurance providers and chief information security officers are examining how to manage losses linked to autonomous AI systems that act in unexpected or harmful ways. As organizations deploy AI agents to automate business, customer-service and security tasks, concerns are growing over who is responsible when those systems make damaging decisions.

The issue is especially complicated when an AI agent has authority to access internal data, interact with third-party services, approve transactions or alter cloud environments. Errors in configuration, flawed instructions, manipulated inputs and inadequate access controls can allow an automated system to create financial, operational or privacy consequences at scale.

Coverage and accountability questions

Traditional cyber insurance policies may address some resulting events, such as data breaches, business interruption or incident-response costs. However, insurers are likely to scrutinize whether an AI-related failure falls within existing policy language, whether human oversight was sufficient, and whether the event was caused by negligence, a software defect or malicious manipulation.

Organizations may also face uncertainty over liability shared among AI model providers, software vendors, systems integrators and the company operating the agent. Contractual terms, audit records and technical evidence could become important in determining responsibility after an incident.

Controls under review

Security teams are increasingly expected to apply governance measures before granting AI agents broad permissions. Common safeguards include:

• Limiting agent access through least-privilege permissions and segmented environments.
• Requiring human approval for high-impact actions, including payments, production changes and data exports.
• Logging agent activity and retaining evidence of prompts, tool calls and decisions.
• Testing systems for prompt injection, data leakage and unsafe tool use.
• Establishing clear incident-response procedures for disabling or containing automated workflows.

For insurers, stronger controls may become a factor in underwriting and pricing as the use of autonomous AI expands. For CISOs, the challenge is to enable productivity gains while ensuring that AI systems remain observable, constrained and accountable.