AI compliance Archives - Krishna Gupta

Explainable AI in Information Security

31 July 2025 by Krishna

In the escalating arms race between cyber defenders and attackers, artificial intelligence (AI) has emerged as a force multiplier—enabling real-time detection, adaptive response, and predictive threat intelligence. However, as these AI systems become increasingly complex, their decision-making processes often resemble a black box: powerful but opaque.
In sectors like healthcare or finance, the risks of opaque AI are already well-documented. But in cybersecurity—where decisions are made in seconds and the stakes are existential—lack of explainability is not just a technical inconvenience; it’s a business liability.
Security teams are already burdened by alert fatigue, tool sprawl, and talent shortages. Introducing opaque AI models into this environment, without explainable reasoning, exacerbates operational risks and undermines confidence in automated systems.
In a field that demands accountability, Explainable AI (XAI) isn’t a luxury—it’s a necessity.
From Security Operations Centre (SOC) analysts to CISOs and regulatory auditors, all stakeholders need clarity on what triggered a threat alert, why an incident was escalated, or how a threat actor was profiled. Without this transparency, false positives go unchallenged, real threats slip through, and strategic trust in AI-based defences begins to erode.
In this blog, we’ll explore how Explainable AI—XAI—helps transform cyber defence from a black-box model to a glass-box ecosystem, where decisions are not only accurate but also interpretable, auditable, and accountable.

LLM06:2025 Excessive Agency — A Critical Vulnerability in the Age of LLM Autonomy

18 June 2025 by Krishna

The surge of Large Language Model (LLM)-driven applications has revolutionised how businesses interact with data, automate processes, and deliver enhanced user experiences. From autonomous customer service bots to intelligent data summarisation tools and generative co-pilots, LLMs are transforming enterprise workflows at an astonishing pace.
However, this rise has not come without significant risk. Among the top concerns identified in the OWASP Top 10 for LLM Applications v2.0, LLM06:2025 – Excessive Agency stands out as a particularly insidious and business-critical vulnerability. It affects systems where LLMs are entrusted not only with information retrieval or generation but with the ability to act on behalf of users — often through invoking external tools, plugins, or APIs.

Weak Model Provenance: Trust Without Proof

12 June 2025 by Krishna

Weak Model Provenance: Trust Without Proof A critical weakness in today’s AI model landscape is the lack of strong provenance mechanisms. While tools like Model Cards and accompanying documentation attempt to offer insight into a model’s architecture, training data, and intended use cases, they fall short of providing cryptographic or verifiable proof of the model’s …

Continue

LLM03:2025 — Navigating Supply Chain Vulnerabilities in Large Language Model (LLM) Applications

10 June 2025 by Krishna

As the adoption of Large Language Models (LLMs) accelerates across industries—from customer service to legal advisory, healthcare, and finance—supply chain integrity has emerged as a cornerstone for trustworthy, secure, and scalable AI deployment. Unlike traditional software development, the LLM supply chain encompasses training datasets, pre-trained models, fine-tuning techniques, and deployment infrastructures—all of which are susceptible to unique attack vectors.

Secure System Configuration: Fortifying the Foundation of LLM Integrity

9 June 2025 by Krishna

When deploying LLMs in enterprise environments, overlooking secure configuration practices can unintentionally expose sensitive backend logic, security parameters, or operational infrastructure. These misconfigurations—often subtle—can offer attackers or misinformed users unintended access to the LLM’s internal behaviour, leading to serious data leakage and system compromise.