BAML
Define the exact shape an LLM function must return, instead of hand-parsing free-form text.
- Typed AI functions, e.g. invoice extraction into a fixed schema
- Built-in validation and testing
- Works across several programming languages
Ten GitHub projects that help AI developers build reliable applications: structured outputs, agent storage, RAG chunking and retrieval, prompt optimization, evaluation, agent testing, security, web agents and agent training.
Each entry links to its GitHub repository. Check the README, license and recent activity before adopting one.
Define the exact shape an LLM function must return, instead of hand-parsing free-form text.
A SQLite-backed filesystem built for AI agents, so their files, state and tool calls can be inspected.
A lightweight Python library for splitting documents into useful chunks before indexing in a vector database.
Tools for ColBERT-style retrieval, where queries and documents are compared token by token rather than as one vector each.
A framework for building LLM applications and tuning prompts and examples against evaluation data.
An open-source evaluation framework from the UK AI Security Institute for testing models and applications.
Simulates conversations with AI agents to check how they behave across different user situations.
An environment for studying and testing prompt-injection attacks and defenses against tool-using agents.
An environment for building and evaluating agents that operate websites.
A Microsoft framework for improving agents with reinforcement learning and other optimization methods.
Ten GitHub repositories for AI developers, covering structured LLM outputs, agent storage, RAG chunking, late-interaction retrieval, prompt optimization, model evaluation, agent testing, prompt-injection security, web agents and agent training.
Chonkie splits documents into chunks before indexing, and PyLate provides ColBERT-style late-interaction retrieval to find the most relevant documents for a query.
Inspect AI evaluates models and applications with tasks, datasets and scoring, Scenario simulates user conversations with agents, and BrowserGym benchmarks agents on website tasks.
AgentDojo is an environment for testing agents against prompt-injection attacks and evaluating defenses, which matters once agents read emails, documents and websites.
Maturity varies. Check each repository's README, license, recent commits and open issues before relying on it, and pilot it on a small project first.