10 GitHub Repositories Every AI Developer Should Explore

Ten GitHub projects that help AI developers build reliable applications: structured outputs, agent storage, RAG chunking and retrieval, prompt optimization, evaluation, agent testing, security, web agents and agent training.

10Repositories
RAG · Agents · EvalsTopics
Nov 2026Updated

The 10 Repositories

Each entry links to its GitHub repository. Check the README, license and recent activity before adopting one.

Define the exact shape an LLM function must return, instead of hand-parsing free-form text.

  • Typed AI functions, e.g. invoice extraction into a fixed schema
  • Built-in validation and testing
  • Works across several programming languages
Structured outputs · github.com/BoundaryML

A SQLite-backed filesystem built for AI agents, so their files, state and tool calls can be inspected.

  • Stores files, agent state and tool-call history
  • Debug a broken agent run by reviewing what it did
  • Snapshots to save and restore earlier states
Agent storage · github.com/tursodatabase

A lightweight Python library for splitting documents into useful chunks before indexing in a vector database.

  • Token, sentence, recursive and semantic chunking
  • Code and table chunking
  • Suits PDF question-answering chatbots
RAG chunking · github.com/feyninc

Tools for ColBERT-style retrieval, where queries and documents are compared token by token rather than as one vector each.

  • Training, indexing and retrieval in one library
  • More detailed document matching
  • A step beyond plain keyword or vector search
Late-interaction retrieval · github.com/lightonai

A framework for building LLM applications and tuning prompts and examples against evaluation data.

  • Automated prompt and example optimization
  • Measures each version's performance
  • PyTorch-inspired, component-based design
Prompt optimization · github.com/SylphAI-Inc

An open-source evaluation framework from the UK AI Security Institute for testing models and applications.

  • Create tasks, datasets and scoring methods
  • Test coding, instruction following and tool use
  • Measure how often an assistant solves a problem set
Model evaluation · github.com/UKGovernmentBEIS

Simulates conversations with AI agents to check how they behave across different user situations.

  • Simulate confused, frustrated or boundary-testing users
  • Evaluate whether replies are correct and safe
  • Fits customer-support agent testing
Agent testing · github.com/langwatch

An environment for studying and testing prompt-injection attacks and defenses against tool-using agents.

  • Hidden instructions in emails or documents
  • Compare defenses across scenarios
  • Shows how vulnerable an agent is to external content
Agent security · github.com/ethz-spylab

An environment for building and evaluating agents that operate websites.

  • Includes WebArena, WorkArena and VisualWebArena benchmarks
  • Tests navigation, form filling and multi-step tasks
  • Realistic website tasks
Web agents · github.com/ServiceNow

A Microsoft framework for improving agents with reinforcement learning and other optimization methods.

  • Works with different agent frameworks
  • Improve existing agents without rebuilding them
  • Learn from feedback on task performance
Agent training · github.com/microsoft

Frequently Asked Questions

What does this list cover?

Ten GitHub repositories for AI developers, covering structured LLM outputs, agent storage, RAG chunking, late-interaction retrieval, prompt optimization, model evaluation, agent testing, prompt-injection security, web agents and agent training.

Which repositories help with RAG?

Chonkie splits documents into chunks before indexing, and PyLate provides ColBERT-style late-interaction retrieval to find the most relevant documents for a query.

Which repositories help test and evaluate AI applications?

Inspect AI evaluates models and applications with tasks, datasets and scoring, Scenario simulates user conversations with agents, and BrowserGym benchmarks agents on website tasks.

Which repositories cover AI agent security?

AgentDojo is an environment for testing agents against prompt-injection attacks and evaluating defenses, which matters once agents read emails, documents and websites.

Are these repositories production-ready?

Maturity varies. Check each repository's README, license, recent commits and open issues before relying on it, and pilot it on a small project first.

Related Pages