llm-safety repositories

Trending repositories tagged llm-safety, ranked by trending score. Updated daily.

  1. 1
    NVIDIA-NeMo/Guardrails

    NeMo Guardrails is an open-source library for applying programmable safety and security controls to LLM and agentic applications.

    GitHub · Python · 7,550 stars · 907 forks · +12 stars in 24 hours · Trending score 2.31

  2. 2
    chrisliu298/awesome-llm-unlearning

    A resource repository for machine unlearning in large language models

    GitHub · 640 stars · 36 forks · +13 stars in 24 hours · Trending score 1.91

  3. 3
    HeadyZhang/agent-audit

    Static security scanner for LLM agents — prompt injection, MCP config auditing, taint analysis. 51 rules mapped to OWASP Agentic Top 10 (2026). Works with LangChain, CrewAI, AutoGen.

    GitHub · Python · 249 stars · 30 forks · +7 stars in 24 hours · Trending score 1.35

  4. 4
    x-zheng16/Awesome-Embodied-AI-Safety

    Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses | 500+ Papers | Perception, Cognition, Planning, Interaction, Agentic System

    GitHub · Python · 165 stars · 5 forks · +5 stars in 24 hours · Trending score 1.33

  5. 5
    confident-ai/deepteam

    DeepTeam is a framework to red team LLMs and AI agents.

    GitHub · Python · 3,118 stars · 512 forks · +1 stars in 24 hours · Trending score 1.05

  6. 6
    PxA-Labs/safetypulse

    Early warning for LLM safety drift during fine-tuning: a Circuit Integrity Score read from the model's refusal circuit flags unsafe checkpoints before harm is measurable, with SafeTune repair and AuditKIT audit.

    GitHub · Jupyter Notebook · 6 stars · 0 forks · +1 stars in 24 hours · Trending score 0.56

  7. 7
    js-lee-AI/awesome-llm-agent-papers

    LLM agents: curated papers on planning, memory, tool use, multi-agent systems, evaluation, and safety.

    GitHub · Python · 115 stars · 40 forks · 0 stars in 24 hours · Trending score 0.55

  8. 8
    HowieHwong/TrustLLM

    [ICML 2024] TrustLLM: LLM trustworthiness evaluation across truthfulness, safety, fairness, robustness, privacy and ethics. Python/CLI toolkit for local models, compatible APIs and AI agent orchestration.

    GitHub · Python · 635 stars · 70 forks · 0 stars in 24 hours · Trending score 0.43

  9. 9
    Buyun-Liang/REALISTA

    [ICML 2026] REALISTA: Realistic Latent Adversarial Attacks that Elicit LLM Hallucinations

    GitHub · Jupyter Notebook · 32 stars · 0 forks · 0 stars in 24 hours · Trending score 0.28

  10. 10
    caiovicentino/jev-risk-check-provider

    x402 risk-check provider: signed pre-payment verdicts — OFAC, phishing feeds, transaction simulation, drainer-kit code, and our own EIP-7702 kit watch (poisoners, sweepers). $0.001 per check with prepaid credits; per call via x402 from $0.0035.

    GitHub · TypeScript · 12 stars · 2 forks · 0 stars in 24 hours · Trending score 0.19

  11. 11
    Jang-woo-AnnaSoft/execution-state-preflight

    Move both the checklist and the verdict outside the model. Slot-based preconditions for irreversible tool calls — execution reads a verdict record, never a conditional.

    GitHub · JavaScript · 5 stars · 0 forks · 0 stars in 24 hours · Trending score 0.09

  12. 12
    OMD-123/agent-loop-guard

    Zero-dependency guardrail framework for AI agents. Stops runaway loops, prompt injection, hallucination and unsafe tool calls before they execute. Provider-agnostic, dual ESM/CJS, TypeScript-first.

    GitHub · TypeScript · 13 stars · 0 forks · 0 stars in 24 hours · Trending score 0.07

  13. 13
    durgadasepalli/nsai-deterministic-gate

    Deterministic guardrails for LLM output in Spring Boot: check AI answers against YAML business rules before users see them. Fails closed, reports the exact rule broken, runs locally with docker compose.

    GitHub · Java · 10 stars · 11 forks · 0 stars in 24 hours · Trending score 0.05

  14. 14
    Tencent-Hunyuan/DiagSafe

    DiagSafe: diagnostic LLM safety benchmark with prompt-level rubrics (3,489 bilingual prompts, 84-label taxonomy)

    GitHub · Python · 39 stars · 4 forks · 0 stars in 24 hours · Trending score 0.04

  15. 15
    cvs-health/uqlm

    [JMLR 2026] UQLM: A Python package for LLM hallucination detection using uncertainty quantification

    GitHub · Python · 1,215 stars · 141 forks · 0 stars in 24 hours · Trending score 0.01

  16. 16
    Chimera-Protocol/csl-core

    Find, prove and control the AI agents on your machines: discovers every agent and what it can reach, proves policies with Z3 + TLA+, enforces them deterministically outside the model.

    GitHub · Python · 27 stars · 13 forks · 0 stars in 24 hours · Trending score 0.00

  17. 17
    h0j5bz0adh0-stack/jev-pilot

    Fast System-1 Decision, Arbitration & Safety Engine for Autonomous AI Agents (Powered by TypeSafe Jev)

    GitHub · Python · 6 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00

  18. 18
    SantanderAI/autoguardrails

    Alignment-research scaffold (autoresearch-style) for LLM guardrails: search over a single policy.md surface

    GitHub · Python · 128 stars · 36 forks · −1 stars in 24 hours · Trending score -0.00