Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams

🐢 Open-Source Evaluation & Testing for AI & LLM systems

[ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.

Deliver safe & effective language models

[ICML 2024] TrustLLM: Trustworthiness in Large Language Models

The open-sourced Python toolbox for backdoor attacks and defenses.

Moonshot - A simple and modular tool to evaluate and red-team any LLM application.

[NeurIPS-2023] Annual Conference on Neural Information Processing Systems

🚀 A fast safe reinforcement learning library in PyTorch

[NeurIPS'24] "Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration"

A comprehensive toolbox for model inversion attacks and defenses, which is easy to get started.

Code of the paper: A Recipe for Watermarking Diffusion Models

AI Verify

A toolbox for benchmarking trustworthiness of multimodal large language models (MultiTrust, NeurIPS 2024 Track Datasets and Benchmarks)

Neural Network Verification Software Tool

[USENIX Security 2025] PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models

Official code repo for the O'Reilly Book - Machine Learning for High-Risk Applications

A toolkit for tools and techniques related to the privacy and compliance of AI models.

The official implementation for ICLR23 paper "GNNSafe: Energy-based Out-of-Distribution Detection for Graph Neural Networks"

A project to add scalable state-of-the-art out-of-distribution detection (open set recognition) support by changing two lines of code! Perform efficient inferences (i.e., do not increase inference time) and detection without classification accuracy drop, hyperparameter tuning, or collecting additional data.