---
title: "Chaos engineering"
description: "Chaos engineering injects controlled failures into systems to verify resilience before real outages strike. Learn the method, tools and safety rules."
url: "https://nodea.io/en/glossary/chaos-engineering"
lang: "en"
updated: "2026-07-13"
---

# Chaos engineering

> Chaos engineering is the discipline of running controlled failure experiments against a system — killing instances, adding latency, cutting dependencies — to verify resilience hypotheses before a real outage does it for you.

**Chaos engineering** is the practice of deliberately injecting failures into a running system under controlled conditions to test whether it survives them the way its architects believe it will. Rather than waiting for a real outage to reveal weak points, engineers cause small, bounded failures on their own schedule and observe the result. The discipline was popularised by Netflix's Chaos Monkey, a tool that randomly terminated production instances and thereby forced every service to tolerate the loss of any machine.

## How chaos engineering works

Each experiment follows a scientific loop:

1. **Define steady state** — measurable indicators of normal behaviour, such as success rate or p99 latency;
2. **Form a hypothesis** — "if one database replica dies, checkout keeps serving 99.9% of requests";
3. **Inject the fault** — terminate instances, introduce network latency or packet loss, exhaust CPU or disk, block a dependency's endpoint;
4. **Observe and conclude** — compare metrics with steady state; any deviation is a discovered weakness to fix, and the experiment is re-run after the fix.

Safety discipline is what separates chaos engineering from recklessness: a limited blast radius (a fraction of traffic, one availability zone), an automatic abort switch when error budgets are threatened, and strong [observability](https://nodea.io/en/glossary/observability) so impact is visible in real time. Common tooling includes Chaos Mesh and LitmusChaos for [Kubernetes](https://nodea.io/en/glossary/kubernetes), Gremlin, and AWS Fault Injection Service.

## Practical use cases

Teams use chaos experiments to validate the machinery of [high availability](https://nodea.io/en/glossary/high-availability): database failover that has never been exercised, load balancers draining unhealthy nodes, retry policies and circuit breakers that should prevent cascading failure, autoscaling under sudden load, and multi-region traffic shifting. "Game days" extend the practice to people — a simulated outage tests runbooks, paging chains and disaster-recovery plans as much as the technology.

The honest prerequisite check: if a system lacks monitoring, redundancy or tested backups, fix those first. Chaos engineering verifies resilience that is supposed to exist — it cannot conjure it.

### Is chaos engineering only for large companies like Netflix?

No, but it assumes prerequisites: monitoring good enough to observe the impact, some redundancy to test, and rollback procedures. A small team can start with simple experiments — rebooting a server during business hours, failing over the database — and grow toward automated fault injection as the platform matures.

## Related

- [SRE (Site Reliability Engineering)](https://nodea.io/en/glossary/sre)
- [High availability (HA)](https://nodea.io/en/glossary/high-availability)
- [Observability](https://nodea.io/en/glossary/observability)
- [Kubernetes](https://nodea.io/en/glossary/kubernetes)
- [DRP (Digital Risk Protection)](https://nodea.io/en/glossary/digital-risk-protection)

---

Languages: [pl](https://nodea.pl/pl/slownik-pojec/chaos-engineering.md)  
Site index: [llms.txt](https://nodea.io/en/llms.txt)