AI Governance Archives - Krishna Gupta

Agentic AI Systems: The Rise of Over-Autonomous Security Risks

1 July 2025 by Krishna

Artificial Intelligence (AI) is no longer just a tool—it’s becoming a decision-maker. With the emergence of Agentic AI Systems—AI with the ability to independently plan, act, and adapt across complex tasks—organisations are entering uncharted territory. While this autonomy promises operational efficiency, it also introduces over-autonomous risks that challenge traditional cybersecurity protocols.
For C-Suite executives and penetration testers alike, understanding the evolution of AI from a predictive model to a proactive actor is no longer optional—it’s imperative. The very qualities that make agentic systems powerful—initiative, goal-seeking behaviour, and environmental awareness—also make them vulnerable to sophisticated threats and capable of causing unintentional damage.

LLM10:2025 – Unbounded Consumption in LLM Applications: Business Risk, ROI, and Strategic Mitigation

26 June 2025 by Krishna

At its core, Unbounded Consumption refers to an LLM application’s failure to impose constraints on inference usage—resulting in an open door for resource abuse. Unlike traditional software vulnerabilities that might involve code injection or data leakage, Unbounded Consumption exploits the operational behaviour of the model itself—by coercing it into performing an excessive number of inferences.

LLM09:2025 Misinformation – The Silent Saboteur in LLM-Powered Enterprises

25 June 2025 by Krishna

In the digital-first world, Large Language Models (LLMs) such as OpenAI’s GPT series, Google’s Gemini, and Anthropic’s Claude are redefining how businesses operate. From automating customer service and accelerating legal research to generating strategic reports, LLMs are integrated into critical enterprise workflows.
LLM misinformation occurs when the model generates false or misleading information that appears credible. This is particularly dangerous because of the model’s inherent linguistic fluency—users often assume that well-phrased responses are factually correct.

LLM06:2025 Excessive Agency — A Critical Vulnerability in the Age of LLM Autonomy

18 June 2025 by Krishna

The surge of Large Language Model (LLM)-driven applications has revolutionised how businesses interact with data, automate processes, and deliver enhanced user experiences. From autonomous customer service bots to intelligent data summarisation tools and generative co-pilots, LLMs are transforming enterprise workflows at an astonishing pace.
However, this rise has not come without significant risk. Among the top concerns identified in the OWASP Top 10 for LLM Applications v2.0, LLM06:2025 – Excessive Agency stands out as a particularly insidious and business-critical vulnerability. It affects systems where LLMs are entrusted not only with information retrieval or generation but with the ability to act on behalf of users — often through invoking external tools, plugins, or APIs.

LLM04: Data and Model Poisoning – A C-Suite Imperative for AI Risk Mitigation

16 June 2025 by Krishna

At its core, data poisoning involves the deliberate manipulation of datasets used during the pre-training, fine-tuning, or embedding stages of an LLM’s lifecycle. The objective is often to introduce backdoors, degrade model performance, or inject bias—toxic, unethical, or otherwise damaging behaviour—into outputs.