Chaos Engineering logopeterindia.netChaos Engineering
Home / Reliability Engineering / Chaos Engineering
Reliability · 2026

Break it on purpose, before it breaks on you

Chaos engineering is the discipline of experimenting on a system to build confidence that it can withstand turbulent conditions in production. This page covers the principles, the experiment loop, common faults to inject, and where to go next.

SYSTEM SERVICE POD

Start at the centre. Widen the blast radius only as confidence grows.

The experiment loop

Every chaos experiment follows the same five steps.

  1. 1

    Define steady state

    Pick measurable signals of normal behaviour, such as latency, error rate or throughput.

  2. 2

    Form a hypothesis

    State what you expect to stay true when something fails.

  3. 3

    Inject a fault

    Introduce a real-world failure on a small, controlled scope.

  4. 4

    Observe

    Compare the system against its steady state using your telemetry.

  5. 5

    Learn & fix

    Close the gap, then widen the scope or run it again automatically.

Five principles

From the Principles of Chaos Engineering.

  • Build a hypothesis around steady state

    Focus on measurable output, not internal details.

  • Vary real-world events

    Crashes, slow networks, bad deploys and dependency outages.

  • Run experiments in production

    Where real traffic and real configuration live, once you are ready.

  • Automate to run continuously

    Manual experiments do not keep up with change.

  • Minimise the blast radius

    Limit the impact so a failed experiment does not become an outage.

Faults worth injecting

Start with the failures your system is most likely to meet.

  • Pod and instance killsDoes the service recover when a replica disappears?
  • Latency and packet lossDo timeouts and retries behave under a slow network?
  • Network partitionsHow does the system split, and how does it heal?
  • CPU and memory stressWhat happens at resource limits?
  • Dependency outagesDo fallbacks work when a downstream service is gone?
  • Bad configuration or deploysCan you detect and roll back quickly?

Related on this site

Tools, reliability practice, testing and platform pages.

Tools & references

Official sites for the projects and services mentioned above. Opens in a new tab.

Questions

Short answers.

What is chaos engineering?

Chaos engineering is the discipline of experimenting on a system to build confidence that it can withstand turbulent conditions in production. Teams inject controlled failures and watch whether the system keeps meeting its goals.

Is chaos engineering the same as testing?

Not quite. Conventional tests check known conditions with expected answers. Chaos experiments explore how a running system behaves under unexpected failure, including interactions between services that tests rarely cover.

Do I have to run it in production?

The principles encourage production because that is where real traffic and configuration are. Most teams start in staging, keep the blast radius small, and move to production as confidence and observability grow.

How do SLOs relate to chaos experiments?

SLOs define what normal looks like. A chaos experiment checks whether the system stays within its SLOs while a fault is injected, which makes the result easy to judge.

What do I need before I start?

Good observability, a clear steady-state definition, an easy way to stop the experiment, and agreement from the teams affected. Start small, such as killing one replica of a non-critical service.