Quantum Ink

Writing about the future as it unfolds.

Containerising an InnoDB cluster for local testing

Containerising an InnoDB Cluster for Local Testing

,

Running a full MySQL InnoDB Cluster on a laptop is a practical way to test HA behaviour, schema changes, and application failover logic without touching production. Containers make this fast and repeatable, but you still need to think like a DBA: networking, persistence, configuration, and failure modes all matter.

Goals and assumptions

This article focuses on a small local lab:

  • 3 MySQL Server instances (InnoDB Cluster members)
  • 1 MySQL Shell container to configure the cluster
  • Optional: 1 router/proxy container for app traffic

We will:

  • Use Docker-style containers (podman/docker) on RHEL/Rocky Linux
  • Use persistent volumes for data, so restarts do not wipe the cluster
  • Configure a basic InnoDB Cluster with MySQL Shell
  • Show how to break and rebuild nodes safely for testing

High-level architecture

The local environment will look like this:

┌──────────────────────────────┐
│   Docker / Podman network   │
│                              │
│  ┌────────┐  ┌────────┐  ┌────────┐
│  │mysql1 │  │mysql2 │  │mysql3 │  MySQL servers
│  └──┬─────┘  └──┬─────┘  └──┬─────┘
│     │           │           │
│     └─────┬─────┴─────┬─────┘
│           │           │
│        ┌──▼────────────▼──┐
│        │  MySQL Shell     │  Cluster admin
│        └──────────────────┘
│
│        ┌──────────────────┐
│        │  Router/Proxy    │  (optional)
│        └──────────────────┘
└──────────────────────────────┘

Each MySQL container runs with:

  • Its own data volume
  • Static container name (e.g. mysql1)
  • Shared custom network so nodes can resolve each other by name

Step 1: Prepare host and network

On RHEL or Rocky Linux, install podman or docker (use your preferred engine). Example with podman:

sudo dnf install -y podman

Create a dedicated network for the cluster:

podman network create innodb-net

Check it exists:

podman network ls

Step 2: Create persistent volumes

Use one volume per MySQL instance so that container recreation does not destroy data:

podman volume create mysql1-data
podman volume create mysql2-data
podman volume create mysql3-data

Also create a volume for configuration scripts (optional but convenient):

podman volume create innodb-scripts

Step 3: Start the MySQL containers

Use an official MySQL Server image that supports InnoDB Cluster (for example, a standard MySQL Server image). The exact tag is up to you; keep it consistent across nodes.

Example commands (adapt for docker if needed):

MYSQL_IMAGE=mysql:latest
MYSQL_ROOT_PASSWORD=rootpass

podman run -d --name mysql1 \
  --network innodb-net \
  -e MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD} \
  -e MYSQL_ROOT_HOST=% \
  -v mysql1-data:/var/lib/mysql \
  -p 33061:3306 \
  ${MYSQL_IMAGE}

podman run -d --name mysql2 \
  --network innodb-net \
  -e MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD} \
  -e MYSQL_ROOT_HOST=% \
  -v mysql2-data:/var/lib/mysql \
  -p 33062:3306 \
  ${MYSQL_IMAGE}

podman run -d --name mysql3 \
  --network innodb-net \
  -e MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD} \
  -e MYSQL_ROOT_HOST=% \
  -v mysql3-data:/var/lib/mysql \
  -p 33063:3306 \
  ${MYSQL_IMAGE}

Notes:

  • MYSQL_ROOT_HOST=% allows root connections from other containers on the network. In production, use more restrictive grants.
  • Host ports 33061–33063 expose each node separately for troubleshooting.

Step 4: Start a MySQL Shell container

MySQL Shell (mysqlsh) is the supported way to configure InnoDB Cluster.

PODMAN_IMAGE=mysql/mysql-server:latest  # or another image that includes mysqlsh

If your server image does not include MySQL Shell, use a dedicated MySQL Shell image and join the same network:

podman run -it --rm --name mysqlsh \
  --network innodb-net \
  -v innodb-scripts:/scripts \
  ${PODMAN_IMAGE} \
  mysqlsh

You now have an interactive mysqlsh prompt inside the container, with network access to mysql1, mysql2, and mysql3 by name.

Step 5: Configure the InnoDB Cluster

5.1 Connect to the first instance

From the mysqlsh prompt:

  # if needed, to clear any previous mode

oSession = shell.connect('root@mysql1:3306', 'rootpass')

Switch to AdminAPI (JavaScript) if not already:

shell.options.set('defaultMode', 'js')

5.2 Create the cluster

Initialise the instance for InnoDB Cluster (if needed) and create the cluster:

var dba = require('dba');

// Configure instance for InnoDB Cluster usage
// This may restart the server if configuration changes are needed.
dba.configureInstance('root@mysql1:3306', {mycnfPath: '/etc/my.cnf'})

// Create the cluster
var cluster = dba.createCluster('localCluster');

Follow any on-screen instructions. For a local lab, defaults are usually acceptable.

5.3 Add the remaining instances

dba.configureInstance('root@mysql2:3306', {mycnfPath: '/etc/my.cnf'})
dba.configureInstance('root@mysql3:3306', {mycnfPath: '/etc/my.cnf'})

cluster.addInstance('root@mysql2:3306');
cluster.addInstance('root@mysql3:3306');

Check status:

cluster.status()

You should see all three instances with ONLINE status and one primary.

Step 6: Optional – add a router/proxy

For application testing, you usually want a single endpoint that tracks the primary and/or read-only replicas. You can either:

  • Use MySQL Router (official tool)
  • Use a generic proxy (e.g. HAProxy) and point it at the cluster nodes

Example MySQL Router container (simplified):

podman run -d --name mysql-router \
  --network innodb-net \
  -p 6446:6446 \
  -e MYSQL_HOST=mysql1 \
  -e MYSQL_PORT=3306 \
  -e MYSQL_USER=root \
  -e MYSQL_PASSWORD=${MYSQL_ROOT_PASSWORD} \
  mysql/mysql-router:latest

In a real setup you would initialise the router using MySQL Shell and a dedicated router account. For local testing, the main goal is a stable endpoint (e.g. localhost:6446) that survives primary changes.

Step 7: Testing cluster behaviour

7.1 Basic read/write test

From your host, connect to the primary (or router) and create a test schema:

mysql -h 127.0.0.1 -P 33061 -u root -p

CREATE DATABASE clustertest;
USE clustertest;
CREATE TABLE t1 (
  id   INT PRIMARY KEY AUTO_INCREMENT,
  note VARCHAR(100)
) ENGINE=InnoDB;

INSERT INTO t1 (note) VALUES ('hello from primary');
SELECT * FROM t1;

Then connect to another node and verify replication:

mysql -h 127.0.0.1 -P 33062 -u root -p
USE clustertest;
SELECT * FROM t1;

7.2 Simulating node failure

Stop a node to observe failover behaviour:

podman stop mysql1

Check cluster status from MySQL Shell (connected to another node):

cluster = dba.getCluster('localCluster');
cluster.status();

For deeper testing, try:

  • Stopping the current primary and watching the new primary election
  • Restarting a node and observing how it rejoins
  • Testing application retries against the router endpoint

Best practices for containerised InnoDB Cluster labs

Use explicit configuration

  • Do not rely only on image defaults. Persist and version-control a minimal my.cnf with relevant replication and InnoDB settings.
  • Mount the same base config into each container, with per-node overrides (e.g. server-id) via environment or include files.

Example directory layout on the host:

/opt/innodb-lab/
  my.cnf
  mysql1.cnf
  mysql2.cnf
  mysql3.cnf

Then mount as:

-v /opt/innodb-lab/my.cnf:/etc/my.cnf:ro
-v /opt/innodb-lab/mysql1.cnf:/etc/mysql/conf.d/node.cnf:ro

Keep ports and names stable

  • Use fixed container names (mysql1, mysql2, mysql3) so MySQL Shell scripts remain valid.
  • Use predictable host ports (33061–33063) to simplify troubleshooting and monitoring.

Script your cluster setup

Instead of typing commands manually in MySQL Shell each time, place a JavaScript or Python script in /scripts and execute it:

podman run -it --rm --name mysqlsh \
  --network innodb-net \
  -v innodb-scripts:/scripts \
  ${PODMAN_IMAGE} \
  mysqlsh --js -f /scripts/init-cluster.js

This makes your lab reproducible and easy to share with teammates.

Understand the limits of local testing

  • Local containers share the same physical host, so they do not model network partitions or hardware failures accurately.
  • Performance characteristics (IO, fsync, latency) differ from production; use the lab for correctness and behaviour, not capacity planning.

Cleaning up safely

To stop the lab without losing data:

podman stop mysql1 mysql2 mysql3 mysql-router

To remove containers but keep data volumes:

podman rm mysql1 mysql2 mysql3 mysql-router

Warning: The next commands are destructive and will permanently delete your lab data volumes. Do not run them against anything you care about.

# DANGEROUS: permanently deletes all lab data
podman volume rm mysql1-data mysql2-data mysql3-data innodb-scripts

Conclusion

Containerising an InnoDB Cluster for local testing gives engineers a realistic HA environment on a single machine. With a small set of containers, a custom network, and scripted MySQL Shell configuration, you can repeatedly spin up and tear down clusters to test schema changes, failover logic, and application behaviour. Treat the lab like a real cluster: use persistent volumes, explicit configuration, and controlled failure tests to build confidence before touching production.

This article offers general technical guidance. Validate all configurations in a safe environment before applying them to production.

Smart reads for curious minds

We don’t spam! Read more in our privacy policy