LLM Frameworks logopeterindia.net
LLM Frameworks

LLM Frameworks

Widely used frameworks for building LLM applications and agents, fine-tuning open models, serving them and evaluating results.

55 topics•8 categories•Updated November 2026

Application & Orchestration Frameworks

9 entries

Frameworks for chains, tools, memory and retrieval in LLM applications.

01LangChainWidely used framework for building LLM applications and agents, with integrations for most model providers and tools. 02LangGraphStateful, graph-based orchestration for long-running, controllable agents. 03LlamaIndexData framework for connecting LLMs to your documents, RAG pipelines and agents. 04HaystackOpen-source framework for production RAG, search and agent pipelines. 05Semantic KernelMicrosoft's SDK for integrating LLMs into C#, Python and Java applications. 06Spring AISpring framework support for building AI applications on the JVM. 07LangChain4jJava library for integrating LLMs, RAG and agents into applications. 08Vercel AI SDKTypeScript toolkit for building AI apps and agents with streaming and structured output. 09MastraTypeScript framework for AI agents, workflows and RAG.

Agent Frameworks

9 entries

Frameworks for tool-using and multi-agent systems.

10OpenAI Agents SDKOpenAI's lightweight framework for multi-agent workflows with tools, handoffs and guardrails. 11Claude Agent SDKAnthropic's SDK for building agents on the same harness that powers Claude Code. 12Google Agent Development Kit (ADK)Google's open-source framework for building and deploying agents. 13Microsoft Agent FrameworkMicrosoft's framework for building agents and multi-agent workflows in Python and .NET. 14CrewAIRole-based multi-agent orchestration framework. 15Pydantic AIType-safe agent framework from the Pydantic team. 16smolagentsLightweight Hugging Face library for agents that write and run code. 17Strands AgentsOpen-source agent SDK from AWS with a model-driven approach. 18AI Agent FrameworksThis site's directory of agent frameworks.

Prompt Programming & Structured Output

6 entries

Control what models return and how prompts are built and optimised.

19DSPyProgramming, not prompting, language models, with automatic prompt and weight optimisation. 20InstructorStructured outputs from LLMs using Pydantic models. 21OutlinesGuarantees structured generation from schemas, regexes and grammars. 22BAMLLanguage for defining typed LLM functions with schemas. 23GuidanceConstrains and interleaves control flow with model generation. 24LiteLLMOne interface and gateway to call many LLM providers.

Low-Code & Visual Builders

4 entries

Build and deploy LLM workflows and agents visually.

25DifyOpen-source platform for building LLM apps, agents and RAG workflows. 26LangflowVisual builder for agents and RAG workflows on top of LangChain. 27FlowiseOpen-source drag-and-drop builder for LLM flows and agents. 28RAGFlowOpen-source RAG engine with deep document understanding.

Fine-Tuning & Training Frameworks

7 entries

Adapt and train open models.

29Hugging Face TransformersThe standard library for pretrained text, vision, audio and multimodal models. 30PEFTParameter-efficient fine-tuning methods such as LoRA. 31TRLPost-training library for SFT, DPO and GRPO. 32UnslothFaster, lower-memory fine-tuning for open LLMs. 33AxolotlConfiguration-driven fine-tuning for many open models. 34LLaMA-FactoryUnified fine-tuning framework for 100+ models with a web UI. 35DeepSpeedDistributed training and inference optimisation library.

Inference & Serving Frameworks

6 entries

Run and serve models efficiently.

36vLLMHigh-throughput LLM inference and serving engine. 37SGLangFast serving framework with structured generation. 38llama.cppLightweight C/C++ inference for local and edge devices. 39OllamaRun open models locally with a simple CLI and API. 40TensorRT-LLMNVIDIA's library for optimised LLM inference on GPUs. 41LLM Inference EnginesThis site's directory of inference engines.

Evaluation & Observability

6 entries

Test, trace and monitor LLM applications.

42DeepEvalOpen-source evaluation framework for LLM applications. 43RagasEvaluation metrics for RAG and agent applications. 44promptfooTest and red-team prompts, models and agents. 45LangfuseOpen-source LLM observability, evaluation and prompt management. 46LangSmithTracing, evaluation and deployment for LLM apps and agents. 47Arize PhoenixOpen-source tracing and evaluation for LLM applications.

Related Pages

8 entries

Related pages on this site.

48Python Libraries for LLM ApplicationsPython libraries for LLM apps. 49Large Language ModelsDirectory of LLMs. 50AI Agent FrameworksAgent frameworks. 51LLM HubHub of LLM guides. 52LLMOpsLLM operations and tooling. 53Open-Source RAG StackOpen-source RAG layers. 54LLM Evaluation MethodsEvaluating language models. 55Fine-Tuning TechniquesAdapting language models.

Frequently Asked Questions

What is an LLM framework?

A library or platform that handles the repeated work of building with large language models: calling models, managing prompts and context, retrieval, tools and agents, structured output, evaluation and serving.

Which LLM framework should I start with?

For a general application or RAG, LangChain or LlamaIndex. For agents, a framework such as LangGraph, the OpenAI Agents SDK or Google ADK. For typed outputs, Pydantic AI or Instructor. Pick the smallest tool that does the job.

LangChain or LlamaIndex?

LangChain is a broad framework for chains, tools and agents. LlamaIndex centres on connecting LLMs to your data for RAG. They are often used together.

What is the difference between an agent framework and an orchestration framework?

Orchestration frameworks sequence model calls, retrieval and tools. Agent frameworks add loops in which the model decides which tool to call next, often with memory, handoffs and guardrails.

What is DSPy for?

DSPy lets you describe an LLM program as modules and metrics and then optimises prompts or weights automatically, instead of hand-writing prompts.

Which frameworks fine-tune open models?

Hugging Face Transformers with PEFT and TRL are the common base. Unsloth, Axolotl and LLaMA-Factory add faster or configuration-driven training.

How do I serve an open model in production?

Use an inference engine such as vLLM or SGLang for throughput on GPUs, or llama.cpp and Ollama for local and edge use.

How do I evaluate an LLM application?

Use an evaluation framework such as DeepEval, Ragas or promptfoo, and trace production behaviour with Langfuse, LangSmith or Arize Phoenix.

How were these frameworks chosen?

They are widely used, actively maintained and from official sources. The field changes quickly, so check each project's current release and licence.