Deployment Reference

This is the full reference for the supported way to run a SharpMUSH game: one small server running everything under Docker Compose. The Run with Docker guide is the short version; this page has every setting, port and command behind it.

The whole game fits on one small virtual machine (about €4–6 a month on a budget host such as Hetzner CX22/CAX11, or free on Oracle Cloud’s Always Free ARM tier). It uses:

• LMDB, an embedded database: the world is one directory on disk, with no database server to run.
• NATS, a message bus that carries commands and output between the parts of the game.
• restic, an optional backup container that sends encrypted nightly backups off the machine.

Kubernetes is not needed at this scale.

What is in the deploy/ folder

The deployment files live in the deploy/ folder of the SharpMUSH repository. There are two ways to run the stack. Pick one, based on how you want HTTPS handled:

File	What it is for
docker-compose.prod.yml	The stack (NATS, the connection server, the renderer, the game server, the optional backup container) plus Caddy, which gets and renews HTTPS certificates for you. Use this when the machine faces the internet directly.
docker-compose.cloudflare.yml	The same stack, reached through a Cloudflare Tunnel instead of Caddy: no open web ports. See Fronting with Cloudflare.
docker-compose.telnet-tls.yml	Optional: telnet over TLS on port 4203, added after either file above. See Telnet over TLS.
docker-compose.analysis.yml	Optional: the remote MCP and LSP analysis transports. See Remote MCP and LSP Analysis.
grafana/sharpmush-invocations.json	An optional Grafana dashboard for function and command timings. See Metrics.
Caddyfile	Caddy’s settings, used by docker-compose.prod.yml.
sharpmush.service	An optional systemd unit that starts the stack from the Compose file when the machine boots.
.env.example	A template for your secrets and settings. Copy it to .env and fill it in.
.gitignore	Keeps your real .env out of git.
../nats.conf	The NATS settings, shared with the development stack. It raises the largest message to 6 MB and gives NATS’s stored messages a 6 GB budget. Both Compose files use it.

First-time setup

cd deploy
cp .env.example .env
# Edit .env: set your domain.

# OPTIONAL: backups. Skip this whole block and the stack runs fine without them.
# Set the restic and storage-bucket credentials in .env, uncomment COMPOSE_PROFILES=backup,
# then create the encrypted backup repository once:
#   openssl rand -base64 32   # makes a RESTIC_PASSWORD. Keep it safe: without it the backups cannot be read.
docker compose -f docker-compose.prod.yml run --rm backup restic init

# Build and start everything:
docker compose -f docker-compose.prod.yml up -d --build

The web portal comes up at https://<your-domain> and telnet on port 4201.

Danger

Open https://<your-domain>/setup straight away. The first visitor to that page claims the administrator account linked to #1.

The socket and rendering containers

The connection server (which holds every player’s connection) and the renderer (which formats output for each kind of client) run as user and group 1654:1654. A new render-socket volume gets that ownership automatically. A socket volume owned by root must be given to 1654:1654 before the services start. Stop both services and change its owner; this disconnects players:

docker compose -f docker-compose.prod.yml stop connectionserver renderer
docker compose -f docker-compose.prod.yml run --rm --no-deps --user 0:0 --entrypoint chown renderer -R 1654:1654 /run/sharpmush
docker compose -f docker-compose.prod.yml up -d connectionserver renderer

Use your own Compose file for all three commands. The connection server waits for the renderer to answer before it starts. It accepts up to 10,000 browser (WebSocket) connections at once by default; set ConnectionServer__MaxConcurrentUpgradedConnections to change that. The limit does not apply to telnet connections.

The database

The world is stored at /app/data/lightning on the app-data volume: one data.mdb file plus a small lock.mdb file, both written by the game server itself. There is nothing else to run, tune or connect to. Two settings on sharpmush-server matter:

Variable	Set to	Why
SHARPMUSH_LIGHTNING_PATH	data/lightning	Relative to /app, so the world lands on the volume.
SHARPMUSH_LIGHTNING_SYNC	periodic	Write to disk once a second instead of after every change. A power cut loses at most that second, and the file stays consistent. full writes after every change, at about 5 ms each.

SHARPMUSH_LIGHTNING_MAPSIZE (in bytes, 64 GiB by default) is the largest the world file may grow. It is a ceiling, not memory or disk set aside: the file only grows as the world does. Raise it before the world reaches it. When the world is full the server refuses new changes with MDB_MAP_FULL rather than damage anything.

The keys the web server signs cookies and forms with are kept beside the world, at /app/data/lightning.dataprotection-keys, so they survive a recreated container. SHARPMUSH_DATAPROTECTION_PATH moves them. If you move them, add the new path to the backup container’s RESTIC_BACKUP_SOURCES too. The same keys decrypt the game’s stored secrets, such as the SQL password: a world restored without them cannot read its secrets until they are set again.

Caution

Nothing outside the game server should read data.mdb while the game runs. To get a copy that is safe to read, have the server make one: see Backups.

Plugins

Plugins the image ships (the Scene System) live inside the image, in /app/plugins/. Plugins an administrator installs from a plugin package, from a remote or by upload on the portal’s Plugins tab, live beside the world on the app-data volume, at /app/data/lightning.plugins/<id>/, so an image update keeps them. The same folder holds plugins.state.json (which plugins are turned off) and .uploads/ (uploads waiting to be applied).

Variable on sharpmush-server	Default	What it does
SHARPMUSH_PLUGINS_PATH	<world>.plugins	Where installed plugins live. Keep it on the volume.
SHARPMUSH_PLUGIN_INSTALL	on	false stops the server installing plugin packages at all. Each install still needs the administrator to confirm they trust the author.

A plugin runs compiled C# with full access to the server; there is no sandbox. The backup container sends this folder with the rest (/data/lightning.plugins is in RESTIC_BACKUP_SOURCES); if you move it, put the new path there too.

A newly installed plugin, and most plugins turned on or off, take effect at the next restart. The Plugins tab’s Restart button and @shutdown/reboot stop the server process and rely on the Compose file’s restart: unless-stopped to start it again; players stay connected, because the connection server holds their connections.

The browser terminal address

The portal’s terminal connects to /ws, which is served by the connection server on port 4202, not by the game server. The Caddyfile already sends https://<your-domain>/ws there, so in the portal set the terminal’s Server URI to:

wss://<your-domain>/ws

The default ws://localhost:4202/ws is for local development only. A browser on an HTTPS page will refuse a plain ws:// connection.

Passkeys

Players can add passkeys to their accounts and sign in with them. A passkey belongs to the address the portal was on when it was made, so nothing needs setting up as long as the portal has one stable HTTPS address and the proxy passes the browser’s Host header through (the Caddyfile does). Two server settings change that:

• Passkeys__RelyingPartyId=example.com makes passkeys work on every subdomain of example.com (play.example.com, www.example.com). Set it before players make passkeys: changing it later makes existing passkeys stop working, and their owners have to sign in with a password and add new ones.
• Passkeys__Origins__0=https://portal.example.com names a portal served from a different address than the server, or behind a proxy that rewrites Host. Without it, Cors__AllowedOrigins is used.

Ports

Port	Service	Open to the internet?
80 / 443 (TCP), 443 (UDP)	Caddy: the web portal, live updates, and /ws to the connection server. UDP 443 is HTTP/3.	Yes. Open UDP 443 on the firewall too, or browsers stay on HTTP/2.
4201	Telnet, unencrypted	Yes
4203	Telnet over TLS, with docker-compose.telnet-tls.yml	Yes, when enabled. Open TCP 4203 on the firewall.
8080	The game server’s web port	No. Reached through Caddy.
4202	The connection server’s web port and /ws	No. Reached through Caddy’s /ws route.
9092	The game server’s metrics for Prometheus (Metrics:Port; 0 turns it off)	No. Published on the machine’s own loopback address (127.0.0.1:9092) only, for a scraper on the machine or an ssh tunnel. Needs Docker Engine 28.3.3 or later; older versions could expose loopback-published ports to the local network.
9091	The connection server’s metrics (ConnectionServer:MetricsPort)	No. Scrape it from inside the Compose network.
4222 / 8222	NATS	No. Internal only.

If you don’t use Caddy (for example, a load balancer handles HTTPS), remember that /ws lives on the connection server (4202), not the game server. Publish 4202 or add an equivalent /ws route, as well as publishing 8080.

Port 4201 is unencrypted

Everything a player types on port 4201, including their password when they log in, crosses the network as plain text. The portal is HTTPS, but MU* clients connect straight to the connection server and bypass it. Offer them telnet over TLS on port 4203.

Telnet over TLS

The connection server can accept TLS from MU* clients on port 4203. Add docker-compose.telnet-tls.yml after your base Compose file, and set SHARPMUSH_TELNET_TLS_CERT_DIR in .env to an absolute directory on the machine holding fullchain.pem and privkey.pem, readable by user and group 1654:1654:

docker compose -f docker-compose.prod.yml -f docker-compose.telnet-tls.yml up -d

• TLS ends in the connection server itself, so the game still sees each player’s real address.
• Without the overlay nothing listens on 4203 (ConnectionServer__TelnetSslPort is 0). The game’s ssl_port setting only tells MSSP crawlers the port; it does not open one.
• Port 4201 stays open. Remove that mapping yourself if you don’t want plaintext access.
• The overlay does not get or renew certificates, and doesn’t use Caddy’s. A new certificate needs a connection server restart, which disconnects players.
• A missing or unreadable certificate stops the connection server from starting.

The certificate must be one players’ clients trust, for the host name they connect to. Players turn on their client’s TLS or SSL option and connect to that name on port 4203.

Fronting with Cloudflare

docker-compose.cloudflare.yml replaces Caddy with a Cloudflare Tunnel. A small cloudflared container connects out to Cloudflare, and Cloudflare sends visitors back down that connection. So:

• you open no web ports on the machine,
• you manage no web certificate (Cloudflare handles HTTPS),
• web traffic reaches the machine only through the tunnel. The telnet address still shows the machine’s public address (see Part B).

There are two separate front doors

Players reach SharpMUSH two ways, and Cloudflare only handles one of them:

Traffic	Protocol	Path	Through Cloudflare?
Web portal and browser terminal	HTTPS and WebSockets	Tunnel to sharpmush-server:8080	Yes
Telnet (MU* clients)	Raw TCP, optionally TLS	Client straight to the machine’s address on :4201 (plaintext) or :4203 (TLS)	No

Cloudflare’s normal proxy only carries web traffic. Telnet cannot go through it on the free or standard plans; that needs Cloudflare’s paid Spectrum product. So the tunnel steps below apply only to the web side, and telnet gets a plain DNS record pointing at the machine.

Part A: the web side

1. Create the tunnel. In the Cloudflare dashboard go to Zero Trust → Networks → Tunnels → Create a tunnel, choose Cloudflared, and name it (for example sharpmush).

2. Copy the token. Cloudflare shows an install command with a long token after --token. Copy just the token into .env:

   CLOUDFLARE_TUNNEL_TOKEN=eyJh...   # the token, nothing else

   You do not need to run the install command; the cloudflared container runs the tunnel.

3. Add your portal address. On the tunnel’s Public Hostname tab, add a route:

   • Subdomain/domain: mush.example.com
   • Service type: HTTP
   • URL: sharpmush-server:8080

   Cloudflare creates the DNS record for you. HTTP is correct here: that hop is inside the private Docker network, and visitors still use HTTPS.

4. Add the terminal route. The browser terminal’s /ws is served by the connection server, so it needs its own route, placed above the one from step 3 (Cloudflare checks them top to bottom):

   • Subdomain/domain: mush.example.com
   • Path: ws
   • Service type: HTTP
   • URL: connectionserver:4202

   Then set the portal terminal’s Server URI to wss://mush.example.com/ws.

   Instead of a path, you can give the terminal its own address: ws.mush.example.com → HTTP → connectionserver:4202, with the Server URI wss://ws.mush.example.com/ws. The path route keeps everything on one address.

Part B: the telnet side

Telnet does not use the tunnel. Give players an address that points straight at the machine:

1. In Cloudflare’s normal DNS page (not Zero Trust), add an A (or AAAA) record, for example telnet.mush.example.com → your machine’s public address.
2. Set the record to DNS only (grey cloud), not proxied. A proxied record would try to send telnet through the web proxy, which does not work.
3. For encrypted connections, turn on telnet over TLS, allow inbound TCP 4203 on the machine’s firewall, and give players telnet.mush.example.com 4203 with their client’s TLS option on. The certificate must cover that host name.
4. The stack also publishes TCP 4201, which is plaintext. Allow it through the firewall only if you offer plaintext access.

Because that record is DNS only, it reveals your machine’s real address, with or without TLS. Web traffic still goes through Cloudflare, but the address of the machine the portal runs on is no longer secret. Hiding it needs Cloudflare Spectrum or another TCP proxy.

Part C: start it

cd deploy
cp .env.example .env      # fill in CLOUDFLARE_TUNNEL_TOKEN and your domain
docker compose -f docker-compose.cloudflare.yml run --rm backup restic init   # optional, once, for backups
docker compose -f docker-compose.cloudflare.yml up -d --build

Check the tunnel with docker compose -f docker-compose.cloudflare.yml logs -f cloudflared (you want to see it register a connection), then load https://mush.example.com.

Ports with Cloudflare

Port	Service	Open on the firewall?
4201	Telnet, plaintext	Only if you offer plaintext access (Part B)
4203	Telnet over TLS	Yes, when enabled (Part B)
80 / 443	—	No. The tunnel only connects outward.
8080	The game server’s web port	No. Reached through the tunnel.
4222 / 8222	NATS	No. Internal only.

Metrics

The game server times every built-in function and command it runs and exports the times on /metrics (port 9092) as two histograms:

Metric	Labels
sharpmush_function_invocation_duration_milliseconds	function_name, success
sharpmush_command_invocation_duration_milliseconds	command_name, success

A label holds the function’s or command’s registered name in capitals, whatever the caller typed: @pe is counted as @PEMIT. A call to a function that does not exist is counted as (unknown), so players cannot add series by typing names. @function globals and @command/add commands appear under their own names. Softcode $-commands are not timed on their own; their time shows in the built-ins they run.

A call’s time is its own work only: time spent evaluating its arguments and in calls nested inside it is counted where it was spent, so null(iter(...)) charges the iteration to ITER, not to NULL. @profile still reports wall time, nested calls included. For a function, success is false when the call errored, was refused, or hit a limit. A command is timed only once it starts running, and its success is false only when it throws.

deploy/grafana/sharpmush-invocations.json is a Grafana dashboard with these and the call and failure rates for both. Import it in Grafana (Dashboards, New, Import) and pick the Prometheus data source that scrapes the server. The storage metrics are described in Storage and History Retention.

Starting on boot

The containers’ restart: unless-stopped setting brings them back after a reboot, which is enough for most machines. But it restores whatever was running when the machine went down, not what the Compose file says. A docker compose down, a container stopped by hand, or an edited Compose file carries over the reboot, and the machine quietly drifts from its configuration.

sharpmush.service fixes that by running docker compose up -d at boot:

# as root, on the host
ln -s /opt/SharpMUSH/deploy/sharpmush.service /etc/systemd/system/sharpmush.service
systemctl daemon-reload
systemctl enable --now sharpmush.service

It pulls images first, so a boot also picks up an image the machine missed while it was off, and it still starts the game if the image registry is down. Edit WorkingDirectory and the Compose file name in the unit if your copy is not at /opt/SharpMUSH or you use the Caddy stack.

This does not replace Watchtower, which handles updates while the machine is running (see Updating).

Backups

Backups are optional and off by default. The backup container only exists once you turn it on, so a machine without backup settings doesn’t get a container that keeps failing. To turn it on, fill in the restic settings in .env and uncomment:

COMPOSE_PROFILES=backup

Once on, the backup container sends /data/backup, /data/wiki-assets, /data/lightning.dataprotection-keys and /data/lightning.plugins to your storage bucket every night at 03:30, keeping 7 daily and 4 weekly snapshots. It reads the volume read-only, so a backup can never damage live data.

It backs up copies, never the live world. Reading data.mdb from start to end while the game keeps writing does not produce a file that can be trusted to open. So the server makes its own copies with LMDB’s built-in copy routine. Each one is a complete world, written into /app/data/backup/<timestamp> while the game keeps running, with latest pointing at the newest. SHARPMUSH_BACKUP_INTERVAL=6h on sharpmush-server makes one every six hours, so the 03:30 run always finds a recent one. The live world at /data/lightning is deliberately left out of RESTIC_BACKUP_SOURCES.

Setting on sharpmush-server	Default	What it does
SHARPMUSH_BACKUP_INTERVAL	unset (no scheduled copies)	How often a copy is made: 6h, 90m, 1h30m, or a number of seconds.
SHARPMUSH_BACKUP_KEEP	2	How many copies stay on disk. Each is a whole world, so this is a disk-space decision.
SHARPMUSH_BACKUP_PACKAGE_KEEP	2	How many of the copies made before a portal package install, rollback or uninstall stay on disk. They are kept in pre-package/, counted separately, so they never push out the copies above. 0 turns them off.
SHARPMUSH_BACKUP_PATH	<world>.backups	Where copies go. Both stacks set it to data/backup.
SHARPMUSH_LIGHTNING_BACKUP_COMPACT	on	Leave out unused space: smaller copies that take longer to make. false turns it off.

A wizard can make a copy at any time in the game with @backup, and list the copies on disk with @backup/list.

The portal makes a copy into pre-package/<timestamp> before every package install, rollback or uninstall, and refuses to go ahead if that copy fails. It is the way back if the operation is interrupted. It copies the world only: an installed plugin’s folder (lightning.plugins/<id>/, see Plugins) is not in it.

The commands below work whether or not backups are turned on. They leave out -f by setting COMPOSE_FILE once per shell to whichever stack you run:

export COMPOSE_FILE=docker-compose.prod.yml        # or docker-compose.cloudflare.yml

If you run an overlay such as docker-compose.telnet-tls.yml, list it after the base file, separated by :.

# List snapshots:
docker compose run --rm backup restic snapshots

# Back up right now:
docker compose run --rm backup backup

# Restore the latest snapshot into a scratch volume to look at it:
docker compose run --rm -v restore:/restore backup \
  restic restore latest --target /restore

Restoring for real

Stop the game, then put the snapshot’s contents back into the app-data volume: one of the backup/<timestamp> copies becomes lightning, and wiki-assets, lightning.dataprotection-keys and lightning.plugins go back beside it. Restore the whole plugin folder, including plugins.state.json (which plugins are turned off) and the hidden .uploads/ folder, and replace it rather than merging: plugins installed after the recovery point must not stay beside the restored world. The game reads whatever is in the volume when it starts.

Use a new, empty scratch volume for each restore; restoring over an earlier one can leave files the chosen snapshot did not contain. The commands below create a uniquely named volume and keep it until you have checked the result. Replace REPLACE_WITH_SELECTED_TIMESTAMP with a finished copy’s name from step 2, not latest or an .incoming-* folder. Don’t carry on after a failed stop or restic restore command.

RESTORE_VOLUME="sharpmush-restore-$(date -u +%Y%m%dT%H%M%SZ)-$$"
docker compose stop sharpmush-server connectionserver &&

# 1. Restore into a scratch volume. The -v keeps the files after the container exits;
#    without it restic writes into the container itself and they are lost.
docker volume create "$RESTORE_VOLUME" &&
docker compose run --rm -v "$RESTORE_VOLUME:/restore" backup \
  restic restore latest --target /restore &&

# 2. See which copies the snapshot holds, and pick one.
docker compose run --rm --no-deps -v "$RESTORE_VOLUME:/restore" --entrypoint sh sharpmush-server \
  -c 'ls /restore/data/backup' &&

# 3. Put it in place. The game server's image can write to app-data; the backup container cannot.
docker compose run --rm --no-deps -v "$RESTORE_VOLUME:/restore" --entrypoint sh sharpmush-server -c '
  set -eu
  source=/restore/data
  target=/app/data
  world_copy=REPLACE_WITH_SELECTED_TIMESTAMP
  case "$world_copy" in
    ""|.*|*/*|latest)
      echo "Choose a completed, dated world copy before restoring." >&2
      exit 1 ;;
  esac
  world="$source/backup/$world_copy"
  if [ ! -f "$world/data.mdb" ]; then
    echo "Selected world copy has no data.mdb; nothing was changed." >&2
    exit 1
  fi
  if [ ! -d "$source/wiki-assets" ]; then
    echo "Snapshot has no wiki-assets directory; nothing was changed." >&2
    exit 1
  fi
  if [ ! -d "$source/lightning.plugins" ]; then
    echo "Snapshot has no plugin directory; inspect the recovery point before continuing." >&2
    exit 1
  fi
  rm -rf "$target/lightning"
  cp -a "$world" "$target/lightning"
  mkdir -p "$target/wiki-assets"
  cp -a "$source/wiki-assets/." "$target/wiki-assets/"
  if [ -d "$source/lightning.dataprotection-keys" ]; then
    rm -rf "$target/lightning.dataprotection-keys"
    cp -a "$source/lightning.dataprotection-keys" "$target/lightning.dataprotection-keys"
  fi
  rm -rf "$target/lightning.plugins"
  cp -a "$source/lightning.plugins" "$target/lightning.plugins"' && \
  docker compose start connectionserver sharpmush-server

If the snapshot has no plugin folder, stop and look at the recovery point: the game may never have installed a plugin or saved which plugins are off. Recover the plugin files and state you need from a trusted source first. Only once you are sure the plugin folder should be empty, create one in the scratch volume and repeat step 3. This clears every installed plugin and the record of which are off:

docker compose run --rm --no-deps -v "$RESTORE_VOLUME:/restore" --entrypoint mkdir sharpmush-server \
  -p /restore/data/lightning.plugins

Check the world is the right one, that logins, wiki files and plugins work, and that a plugin that was off is still off, as in the restore drill. If any command failed, leave the game stopped and keep the scratch volume while you find out why. Once the restore is checked:

docker volume rm "$RESTORE_VOLUME"

A restored copy has no lock.mdb (LMDB makes a new one when it opens) and no lightning.previous, which is a world replaced by a promoted database import and never part of a copy. See Earlier worlds.

The game does not need to stop for the nightly backup: restic only reads finished copies. A copy still being written is named .incoming-* and is skipped until it is complete.

Disk space and history retention

How the world uses disk, what a backup run needs, cleaning up earlier worlds, compacting, container logs and history retention are on Storage and History Retention.

Updating

The Caddy stack does not update itself. docker-compose.prod.yml builds its images from your checkout and runs no Watchtower. To update it, from deploy/:

git pull
docker compose -f docker-compose.prod.yml up -d --build

Do this in an announced window: it recreates the connection server, which disconnects players.

The Cloudflare stack updates itself. Every change merged into SharpMUSH runs the full test suite and then publishes new images to Docker Hub: sharpmush/sharpmush-server:dev for the game server, sharpmush/sharpmush-connectionserver:dev for the renderer, and sharpmush/sharpmush-socketserver:dev for the connection server. The watchtower container checks every 5 minutes and replaces the game server and renderer when their images change. Neither replacement disconnects anyone: players see a restart notice and carry on.

The connection server is not updated automatically, because it holds every player’s connection and replacing it disconnects everyone. Update it on purpose, when you are willing to drop players:

docker compose -f docker-compose.cloudflare.yml pull connectionserver
docker compose -f docker-compose.cloudflare.yml up -d connectionserver

Nothing is built on the machine, and it needs no inbound access. The web portal is built into the game server image.

To update the other containers (NATS, cloudflared, backup, which Watchtower leaves alone), or to update the game now instead of waiting for Watchtower:

git pull
docker compose -f docker-compose.cloudflare.yml pull
docker compose -f docker-compose.cloudflare.yml up -d

The app-data volume is kept across updates, so the world is untouched. Database upgrades are recorded in the database and run only once, so unattended restarts are safe.

What players see during each kind of update, and how far a browser session can recover, is on Connections During Updates.