LLM cost optimization logopeterindia.net
Knowledge Hub

LLM Cost Optimization Techniques

How to cut LLM spend without losing quality. Fifteen techniques — smaller models, routing, context trimming, summarization, output limits, RAG, three kinds of caching, structured outputs, batching, agent guardrails, tool-first design, query classification and cost dashboards — with provider docs, papers and open-source tools. Grouped by purpose, not ranked.

77 topics•10 categories•Updated November 2026

1. Foundations: Pricing, Tokens and Cost Drivers

8 entries

Understand what you pay for before optimising. Cost is roughly tokens in plus tokens out, multiplied by the model's price, multiplied by the number of calls. Every technique below reduces one of those three.

01OpenAI API pricingPer-token prices, cached-input and batch discounts for OpenAI models. 02Anthropic pricingClaude model prices, prompt caching and batch rates. 03Gemini API pricingGemini prices including context caching and batch mode. 04Anthropic token countingCount tokens before sending a request. 05tiktokenFast tokenizer library for OpenAI models. 06FrugalGPT: How to Use LLMs While Reducing Cost and Improving PerformancePaper on prompt adaptation, approximation and cascades to cut LLM spend. 07OWASP LLM10: Unbounded ConsumptionRisk of runaway cost and denial of wallet in LLM applications. 08FinOps FoundationFramework and community for cloud and AI cost management.

2. Smaller Models, Multi-Model Routing and Query Classification

9 entries

Use the cheapest model that can do the job and route by difficulty. Reserve large models for coding, planning and complex reasoning; send FAQs, classification and extraction to smaller ones.

09RouteLLM: Learning to Route LLMs with Preference DataPaper on routing between strong and weak models to cut cost. 10RouteLLM (GitHub)Open-source framework for serving and evaluating LLM routers. 11LiteLLM routingLoad balancing and fallbacks across models and providers. 12OpenRouter documentationUnified API and routing across many model providers. 13Semantic RouterFast intent routing using embeddings instead of an LLM call. 14Adaptive-RAGClassifies query complexity to pick no retrieval, single-step or multi-step retrieval. 15OpenAI model selection guideGuidance on balancing accuracy, latency and cost. 16Claude models overviewModel tiers and when to use each. 17SmolLM3 (Hugging Face)Example of a small model suited to cheap, high-volume tasks.

3. Context Trimming, Summarization and RAG

9 entries

Send only what the model needs. Cutting a 20k-token conversation to the 2k relevant tokens can reduce input cost dramatically.

18Effective context engineering for AI agentsAnthropic's guidance on curating what goes into the context window. 19LLMLinguaPrompt compression that removes low-value tokens. 20LLMLingua paperCompressing prompts for accelerated inference. 21LongLLMLinguaCompression for long-context scenarios. 22Lost in the MiddleWhy stuffing long context can hurt accuracy as well as cost. 23LangChain: build a chatbot with memoryTrimming and summarizing message history. 24Retrieval-Augmented Generation for Knowledge-Intensive NLP TasksThe original RAG paper. 25Open-Source RAG StackComponents for retrieval-augmented generation. 26Vector Indexing AlgorithmsChoosing an index that balances recall, latency and memory.

4. Output Limits and Structured Outputs

7 entries

Control response length and format. Fewer output tokens cost less, and structured output reduces retries and parsing failures.

27Anthropic Messages API (max_tokens)Limit response length with max_tokens. 28OpenAI Chat Completions APIOutput limits and response format options. 29OpenAI structured outputsSchema-constrained JSON responses. 30Anthropic tool useStructured calls that return predictable arguments. 31InstructorValidated structured outputs with retries. 32OutlinesConstrained generation for structured outputs. 33XGrammarFast grammar-constrained decoding engine.

5. Response, Prompt and Semantic Caching

9 entries

Pay once for repeated work. Exact caches return stored answers, prompt caches discount repeated prefixes, and semantic caches match by meaning.

34Anthropic prompt cachingCache repeated prompt prefixes such as system prompts and documents. 35OpenAI prompt cachingAutomatic discounts for repeated prompt prefixes. 36Gemini context cachingStore large context once and reuse it. 37LiteLLM cachingResponse caching with Redis and other backends. 38GPTCacheSemantic cache for LLM queries. 39What is semantic caching? (Redis)Explains matching queries by meaning. 40vLLM automatic prefix cachingReuse KV cache for shared prefixes in self-hosted serving. 41SGLang: Efficient Execution of Structured Language Model ProgramsPaper introducing RadixAttention for prefix reuse. 42LMCacheKV cache layer that reuses cache across requests.

6. Batch Processing and Tool-First Architecture

6 entries

Group work and use deterministic tools before calling an LLM. Batch APIs offer discounts for non-urgent jobs, and a clock, calculator or SQL query is cheaper and more accurate than a model.

43OpenAI Batch APIAsynchronous batch jobs at a discount. 44Anthropic Message BatchesProcess large volumes of requests asynchronously. 45Gemini batch modeBatch requests for non-urgent workloads. 46Amazon Bedrock batch inferenceRun batch inference jobs on Bedrock. 47Model Context ProtocolStandard way to expose tools and data to models. 48Building effective agents (Anthropic)Start with simple workflows and add agents only when needed.

7. Agent Guardrails: Iterations, Tool Calls, Tokens and Timeouts

5 entries

Agents are often the largest source of AI spend, so cap them. Set maximum iterations, tool calls, tokens and wall-clock time, and stop loops early.

49LangGraph recursion limitCap the number of steps in a graph run. 50OpenAI Agents SDK: running agentsMax turns and run configuration. 51OWASP Top 10 for LLM ApplicationsSecurity risks including excessive agency and unbounded consumption. 52Guardrails for Agentic AIControls that limit agent behaviour. 53AI Agent GatewaysCentral control points for agent traffic, budgets and policies.

8. Cost Monitoring, Observability and FinOps

8 entries

Track cost per user, request, token, cache hit and agent. Watch daily spend, cost spikes, expensive prompts, users and agents, and set budgets and alerts.

54Langfuse: model usage and costTrack token usage and cost per trace. 55Helicone documentationLLM observability and cost tracking proxy. 56Arize PhoenixOpen-source LLM tracing and evaluation. 57OpenTelemetry GenAI semantic conventionsStandard attributes for tokens, models and operations. 58OpenLLMetryOpenTelemetry-based instrumentation for LLM apps. 59LiteLLM spend trackingPer-key, per-user and per-team budgets. 60FinOps ToolsCloud and AI cost management tools. 61Observability ToolsMetrics, logs and traces.

9. Self-Hosted Inference Savings

9 entries

Lower unit cost when you run models yourself. Quantization, speculative decoding and efficient engines raise throughput per GPU.

62vLLM documentationHigh-throughput serving engine. 63Fast Inference from Transformers via Speculative DecodingDraft-and-verify decoding for faster generation. 64Distilling the Knowledge in a Neural NetworkTrain a smaller model from a larger one. 65LLM Inference EnginesvLLM, SGLang, llama.cpp and more. 66AI Model Quantization TechniquesCut memory and cost with lower precision. 67On-Device Inference EnginesRun models locally with no per-token fees. 68Platforms for Running LLMs LocallyOllama, LM Studio and similar. 69LLM Training and Inference OptimizationKV cache, batching and more. 70Small Language ModelsSmall models for cheap tasks.

10. Related Pages on This Site

6 entries

Directories for token optimization, FinOps and LLM operations. Where each part of an LLM cost programme is listed.

71Token Optimization ToolsTools that cut LLM and Claude Code tokens. 72Cloud Cost Optimization ToolsCloud cost management. 73LLMOpsOperating LLM applications. 74The Production AI Inference StackServing stack layers. 75LLM Inference Platforms and ToolsManaged and self-hosted inference. 76Open-Source LLMsOpen model families.

Related Pages

1 entries

Related research directories on this site.

77AI Models You Can Run for Free Without a GPUSmall open models, Ollama commands, embeddings, vision and speech models that run on a CPU.

Frequently Asked Questions

What are the main LLM cost optimization techniques?

Fifteen common ones are: use smaller models, route requests across models, trim context, summarize conversation history, cap output tokens, use retrieval-augmented generation, cache responses, cache prompt prefixes, use semantic caching, request structured outputs, batch requests, set agent guardrails, call deterministic tools first, classify queries before processing and monitor cost with dashboards.

How do smaller models and multi-model routing save money?

Most FAQ, classification and extraction tasks work well on smaller models, so reserve large models for coding, planning and complex reasoning. A router sends each request to a model that matches its difficulty, for example sentiment analysis to a small model, SQL generation to a medium one and coding to a large one. Research such as FrugalGPT and RouteLLM reports large savings, but results depend on your tasks, so evaluate on your own data.

How do context trimming and summarization reduce cost?

Input tokens are billed on every call, so sending only relevant messages, removing duplicates and replacing old history with a short summary of key facts, preferences and decisions can cut a 20,000-token prompt to a few thousand. Check answer quality after trimming, because dropped context can cause errors.

What is the difference between response, prompt and semantic caching?

Response caching returns a stored answer when the same question repeats. Prompt caching lets the provider reuse repeated prompt prefixes such as system prompts and documents at a discount. Semantic caching matches questions by meaning, so "What is OAuth?" and "Explain OAuth" can share one answer, at the risk of returning a near match that is wrong.

Why use structured outputs and output limits?

Short, structured responses use fewer output tokens, are easier to parse and cause fewer retries. Set max tokens, ask for concise formats such as three bullets, and use schema-constrained JSON where the provider supports it.

When should I use batch processing?

Batch APIs from providers such as OpenAI, Anthropic, Google and Amazon Bedrock process non-urgent jobs asynchronously, often at a discount, which suits classification, extraction, tagging and data enrichment. Check each provider current terms for the discount and turnaround.

Why are agents the biggest cost risk and how do I control them?

Agents loop, call tools and re-send growing context, so spend can climb quickly. Set limits on iterations, tool calls, tokens and timeouts, stop loops early and put budgets at a gateway. OWASP lists unbounded consumption as an LLM risk.

What does a tool-first architecture mean?

Use deterministic tools before calling a model: a clock API for time, a weather API for weather, a calculator or API for currency conversion, and SQL for database lookups. Use the LLM only when reasoning is required, which cuts cost and errors.

What should I monitor?

Track cost per user, cost per request, tokens per request, cache hit rate, model usage and agent cost, plus daily spend, spikes, and the most expensive prompts, users and agents. Tools such as Langfuse, Helicone and LiteLLM, and OpenTelemetry GenAI conventions, help collect this data.

How current is this list?

Links were checked in November 2026. Some sites block automated checks and may need a manual click. Provider prices and discounts change often, so confirm them on the provider pricing pages. This page is a resource list, not financial advice.