Large Language Models (LLM) have become powerful tools in the field of artificial intelligent, which is rapidly evolving. The models have revolutionized the way we consume and interact with information. They range from content generation to natural language processing. As with all new technologies, there are inherent challenges and risks.
Previously, we have reflected on “The Machines Are Learning… To Hack! Generative AI as the Attacker,” where we looked at the risk of GenAI being weaponized by attackers. In this article, we will deep dive on the security landscape surrounding LLMs and RAG systems, outlining potential threats and mitigation strategies.
The potential of adversarial attack is a major concern when it comes to LLMs. They can exploit text-processing weaknesses to bypass restrictions and system prompts. The integrity and security LLM systems are threatened by techniques such as “prompt injection”, where malicious inputs alter the output.
Security of data is another concern. Data that is contaminated or compromised can produce biased or malicious results. Using sensitive or copyrighted data to train employees raises both ethical and legal issues.
Retrieval-Augmented Generation (RAG) systems are one of the most popular implementations of LLMs. The systems use vector databases and LLMs to extract relevant information, contexts, and data during the creation process. This approach, while it can improve the accuracy and quality of outputs as well as their overall quality, also poses new security risks.
OWASP LLM Top 10
OWASP Top 10 Large Language Model Applications (LLM Apps) is a project of OWASP which aims at educating developers, security experts, and organisations about the 10 top security vulnerabilities and risks associated with deploying LLM apps.
OWASP Machine Learning Top 10
OWASP Machine Learning Top 10 (OWASP Machine Learning Top 10) is a project of the Open Web Application Security Project. It aims at identifying and raising awareness regarding the 10 top security vulnerabilities and risks associated with Machine Learning (ML) Systems.
The MITRE ATT&CK Framework
MITRE AT&CK is a framework that we use to combat threats like these. The framework allows us to categorize and understand the different techniques and tactics used by attackers. MITRE has extended this to AI with ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems), and guides how AI can be attacked and what can be done to defend against them.
NIST’s Role in AI Security
The National Institute of Standards and Technology (NIST) provides guidelines for securing AI. Their taxonomy for adversarial machine learning (ML) helps organizations classify and mitigate these risks. NIST’s work is essential because it gives a structured approach to understanding and defending against AI-specific threats.
Ragas Framework
To secure RAG implementations, it is important that the vector databases and data pipelines used in retrieval are protected. Ragas provides a tool to evaluate Retrieval Augmented Generation pipelines.
https://docs.ragas.io/en/stable
AI/ML Vulnerability Database
https://sightline.protectai.com/vulnerabilities
AI Vulnerability Database (AVID) Taxonomy
AVID aims at raising awareness of AI/ML technology and promoting responsible deployment. This includes details such as vulnerability types, models/systems affected, impact assessments and possible mitigations.
AI Bill of Material (AIBOM)
AIBOMs are a comprehensive documentation that includes the components of an AI system, its data sources, methods, ethical considerations, and other relevant information. They aim to promote transparency, reproducibility, and accountability in AI, as well as responsible AI practices.
https://www.manifestcyber.com/aibom
GenAI Reference Architecture & Maturity Model
Ali Arsanjani, PhD, provides readers with a reference architectures and maturity model that is comprehensive, yet easy to understand, allowing them to get beyond the hype cycle in order build effective solutions.
AI Attack Surface Map
Provides a visual map to understand attacks on AI and AI-based systems
To prevent attacks such as prompt injection or input-based attacks, always validate inputs before sending them to AI systems.
Secure Plugin Design — Ensure that any plugins used are built with security and tested thoroughly.
Protect Training Data: Data lineage (an end-to-end view of the data flow, from its source to its destination, including all the intermediate steps and processes it goes through) is a great way to ensure that the data used to train AI models are clean, free of malicious influences and have not been tampered with.
Security Audits: Perform regular audits to find and correct vulnerabilities in AI system before it can be exploited.
Model Encryption: Protect AI models from theft by encrypting them, particularly if they are commercially valuable.
Access Control — Implement access control measures to limit the interaction of AI systems with unauthorized individuals.
The Importance of Layered Security
Multi-layered security is the implementation of multiple defences in different points within the AI/LLM system. In addition to securing data and models one must also protect the APIs, and the user interfaces that the AI uses in its interactions with users and other systems. Every layer is an extra barrier for attackers.
Future-Proofing AI Security
To future-proof AI security, it is important that we stay ahead of the threats by implementing continuous adaptation and research. By investing in continuing education, organizations can keep their security teams up to date with AI trends and best practices.
Collaboration and Knowledge Sharing
Collaboration and sharing of knowledge are two effective methods to improve AI security. Participating in industry groups and conferences and exchanging insights with colleagues can help organizations stay up-to-date on new threats.
Concluding Thoughts
AI security landscape is constantly changing and complex, but we can prevent these powerful systems being abused with proactive and careful measures. Understanding the threats that AI faces and taking proactive measures to protect them will help to achieve the ultimate goal of ensuring AI benefits society, without it becoming a weapon for harm.
Get in touch to find out more about IT Minister AI and LLM Security assessment
How Can ITM Help You?
IT Minister covers all aspects of Cyber Security including but not limited to Home cyber Security Managed Solutions to automated, Manage Threat Intelligence, Digital Forensic Investigations, Penetration Testing, Mobile Device Management, Cloud Security Best Practice & Secure Architecture by Design and Cyber Security Training. Our objective is to support organisations and consumers at every step of their cyber maturity journey. Contact Us for more information.