Developer Guide

Complete guide to developing Chaos Mesh

This guide covers the essential information you need to develop and contribute to Chaos Mesh.

Project Overview

Chaos Mesh is a cloud-native Chaos Engineering platform for Kubernetes that provides various types of fault simulation. It consists of three main components:

1. Chaos Controller Manager: Core component responsible for scheduling and managing Chaos experiments through various CRD controllers (Workflow, Scheduler, and fault type controllers)
2. Chaos Daemon: Runs as a DaemonSet with privileged permissions, interferes with network devices, file systems, and kernels by accessing target Pod namespaces
3. Chaos Dashboard: Web UI for managing, designing, and monitoring Chaos experiments

Project Structure

The Chaos Mesh codebase is organized as follows:

• cmd/: Main entry points for binaries (controller-manager, daemon, dashboard, builder)
• controllers/: Controller implementations and reconciliation logic
• pkg/: Shared packages (chaosdaemon, dashboard, grpc, selector, metrics, etc.)
• api/: CRD API definitions (v1alpha1)
• config/: Kubernetes manifests (CRD, RBAC, webhook)
• helm/chaos-mesh/: Helm chart for deployment
• e2e-test/: End-to-end test suite
• images/: Dockerfiles for all components
• hack/: Build and development scripts
• ui/: Frontend dashboard (pnpm-based)

Development Workflow

Before Making Changes

Always run checks to ensure your environment is properly set up:

make check

This command runs all quality checks including formatting, linting, and vetting.

Making Changes

1. Fork and Clone the repository:

   git clone [e-mail]:your-username/chaos-mesh.git
   cd chaos-mesh
   

2. Set up upstream remote:

   git remote add upstream https://github.com/chaos-mesh/chaos-mesh
   git remote set-url --push upstream no_push
   

3. Create a working branch:

   git fetch upstream
   git checkout master
   git rebase upstream/master
   git checkout -b new-feature
   

4. Make your changes in the code

After Code Changes

Depending on what you modified, you may need to regenerate code:

If you modified CRDs or APIs:

make generate
make manifests/crd.yaml

Run quality checks:

make check

This will:

• Format code with goimports
• Lint with revive
• Run go vet
• Check go.mod is tidy
• Regenerate install scripts and Helm schemas

Testing Your Changes

Unit Tests

Run all unit tests:

make test

This command:

• Enables failpoint stubs for testing
• Runs tests with coverage
• Generates coverage reports in cover.out

E2E Tests

Build and run end-to-end tests:

make e2e-build
make e2e

E2E tests require a running Kubernetes cluster. Ensure you have access to a cluster before running these tests.

Building Components

Build all images:

make image

This builds container images for:

• chaos-controller-manager
• chaos-daemon
• chaos-dashboard

Build specific components locally:

make local/chaos-daemon
make local/chaos-controller-manager
make local/chaos-dashboard

Build the UI:

UI=1 make ui

Or manually:

cd ui
pnpm install --frozen-lockfile
pnpm build

Committing Changes

1. Keep your branch synced:

   git fetch upstream
   git rebase upstream/master
   

2. Commit with sign-off (required for DCO compliance):

   git add -A
   git commit --signoff
   

3. Push your changes:

   git push origin new-feature
   

All commits must be signed off using git commit --signoff to comply with the Developer Certificate of Origin (DCO).

Code Generation

Chaos Mesh uses several code generation tools. Here’s what each command does:

Generate everything:

make generate

This runs all generation steps including CRD manifests, deepcopy files, client code, and Swagger specs.

Individual generation commands:

# Generate CRD manifests
make config

# Generate chaos types code
make chaos-build

# Generate deepcopy methods
make generate-deepcopy

# Generate Kubernetes clients
make generate-client
make generate-clientset
make generate-lister
make generate-informer

# Generate protobuf code
make proto

# Generate Swagger/OpenAPI specs
make swagger_spec

# Generate combined CRD manifest
make manifests/crd.yaml

Development Environments

Chaos Mesh provides containerized development environments:

Build Environment

For compiling binaries with minimal build tools:

make image-build-env
make enter-buildenv

Dev Environment

For development tasks (code generation, linting, testing):

make image-dev-env
make enter-devenv

Code Style

Formatting

Use goimports with local import prefix:

make fmt

This runs:

goimports -w -l -local github.com/chaos-mesh/chaos-mesh

Linting

Run the revive linter:

make lint

Configuration is in revive.toml.

Go Modules

Keep go.mod tidy across all submodules:

make tidy

This runs go mod tidy in:

• Root directory
• api/
• e2e-test/
• e2e-test/cmd/e2e_helper/

Security Scanning

Run security scans with gosec:

make gosec-scan

Chaos Types

Chaos implementations are located in controllers/chaosimpl/:

• awschaos: AWS fault injection
• azurechaos: Azure fault injection
• blockchaos: Block device faults
• dnschaos: DNS fault injection
• gcpchaos: GCP fault injection
• httpchaos: HTTP fault injection
• iochaos: I/O fault injection
• jvmchaos: JVM fault injection
• kernelchaos: Kernel fault injection
• networkchaos: Network fault injection
• physicalmachinechaos: Physical machine faults
• podchaos: Pod lifecycle faults
• stresschaos: CPU/Memory stress
• timechaos: Time skew simulation

Each chaos type has:

• CRD definition in api/v1alpha1/
• Controller implementation in controllers/chaosimpl/
• Webhooks for validation and defaulting

Common Pitfalls

Avoid these common mistakes when developing Chaos Mesh:

1. Forgetting to run make check: Always run this before creating a PR
2. Not regenerating after CRD changes: When modifying CRD structs, always run:
   make generate && make manifests/crd.yaml
   
3. Failpoint misuse: Enable and disable failpoints properly in tests
4. Multi-module complexity: Remember to run go mod tidy in all directories:
   • Root
   • api/
   • e2e-test/
5. Controller design violations: Follow the “one controller per field” principle

Next Steps

• Read the Build System documentation
• Understand Controller Design Principles
• Review Contributing Guidelines
• Check out the Development Guide on chaos-mesh.org