Information Security Archives - Page 2 of 83

LLM06:2025 Excessive Agency — A Critical Vulnerability in the Age of LLM Autonomy

18 June 2025 by Krishna

The surge of Large Language Model (LLM)-driven applications has revolutionised how businesses interact with data, automate processes, and deliver enhanced user experiences. From autonomous customer service bots to intelligent data summarisation tools and generative co-pilots, LLMs are transforming enterprise workflows at an astonishing pace.
However, this rise has not come without significant risk. Among the top concerns identified in the OWASP Top 10 for LLM Applications v2.0, LLM06:2025 – Excessive Agency stands out as a particularly insidious and business-critical vulnerability. It affects systems where LLMs are entrusted not only with information retrieval or generation but with the ability to act on behalf of users — often through invoking external tools, plugins, or APIs.

LLM05:2025 – Improper Output Handling in LLM Applications: A Business Risk Executive Leaders Must Not Ignore

17 June 2025 by Krishna

At its core, Improper Output Handling refers to inadequate validation, sanitisation, and management of outputs generated by large language models before those outputs are passed downstream—whether to user interfaces, databases, APIs, third-party services, or even human recipients.

LLM04: Data and Model Poisoning – A C-Suite Imperative for AI Risk Mitigation

16 June 2025 by Krishna

At its core, data poisoning involves the deliberate manipulation of datasets used during the pre-training, fine-tuning, or embedding stages of an LLM’s lifecycle. The objective is often to introduce backdoors, degrade model performance, or inject bias—toxic, unethical, or otherwise damaging behaviour—into outputs.

LLM Models on Device: Unpacking Supply-Chain Vulnerabilities for Prompt Engineers and the C-Suite

15 June 2025 by Krishna

Supply-chain threats no longer pertain solely to enterprise software. In the world of on-device LLMs, hardware, firmware, model packaging, and even manufacturing processes are all potential vectors for compromise.

Exploiting Collaborative Development Processes: A Growing Threat to AI Integrity and Enterprise Risk

14 June 2025 by Krishna

The success of platforms like Hugging Face, GitHub, and Weights & Biases demonstrates a strong appetite for collaborative AI development. Organisations often benefit from open-sourcing internal models, using pre-trained datasets, or merging models to create new capabilities faster.