git-website-engine-control-plane/docs/OPERATIONS.md

Website Engine Operations

This document records the verified deployment and access design without containing passwords, tokens, private keys, or credential values.

Current topology

The current deployment is unstaged. The active release is the live site.

Purpose	Address	Exposure
Engine control plane	http://10.138.2.46/	Private network, Nginx port 80
Rendered live site	http://10.138.2.46:3000/	Private network, Nginx port 3000
Control-plane Node service	http://127.0.0.1:3001/	Engine container only
Gitea web/API	http://10.138.2.42:3000/	Private network
Gitea SSH	[email protected]:22	Private network
Public Gitea	https://git.labyricorn.com/	Public entry point

Port 80 proxies to the private Node listener. Port 3000 serves files through the active-release symlink; it does not proxy to Node.

In a future staged deployment, port 3000 may be assigned to the preview candidate instead. That behavior is not configured in the current unstaged deployment.

Server layout

Item	Location or name
Source checkout	/opt/website-engine-control-plane
Persistent builds	/var/lib/website-engine/builds
Active-release pointer	/var/lib/website-engine/builds/current
Service account	website-engine
systemd service	labyricorn-control-plane.service
Nginx site	/etc/nginx/sites-available/website-engine
Versioned service definition	deploy/labyricorn-control-plane.service
Versioned Nginx definition	deploy/nginx-website-engine.conf

The systemd service is enabled across reboots. Nginx and the control-plane service must both be active for the control plane to work. Nginx alone serves the activated static release on port 3000.

Private Gitea access

The engine checkout uses this read-only private SSH remote:

[email protected]:Labyricorn/git-website-engine-control-plane.git

The Gitea SSH username is gitea, not the common default git.

The deploy key is named website-engine-10.138.2.46 in Gitea. Its public fingerprint is:

SHA256:RtuazmPwcRLY6PbgtEaEELoQk8nN7OfsMyaB7Sa++J0

The pinned Gitea ED25519 host fingerprint is:

SHA256:XM3wWREeQesVBhLltXXnGCmpCr6M1bWH+545f792tGc

Credential files on the engine server:

File	Owner/mode	Purpose
/var/lib/website-engine/.ssh/gitea_ed25519	website-engine, 0600	Private deploy key; never display or copy into Git
/var/lib/website-engine/.ssh/gitea_ed25519.pub	website-engine, 0644	Public deploy key
/var/lib/website-engine/.ssh/known_hosts	website-engine, 0600	Pinned Gitea host key

The repository-local core.sshCommand selects this identity, requires the pinned host key, disables interactive prompting, and refuses unrecognized host keys. The deploy key has read access only.

Older protected HTTP credentials remain outside the repository under /root/.config/gitea/. They are not needed for deployment fetches now that private SSH works. Never print, commit, or copy their values into commands, logs, issues, or documentation.

Verified access matrix

Path	Result
Engine to http://10.138.2.42:3000/	HTTP 200
Engine to the private Gitea API	HTTP 200
Anonymous private HTTP Git read	HTTP 401, expected for the private repository
SSH as [email protected]	Rejected; incorrect Gitea SSH user
SSH as [email protected]	Authenticated with the deploy key
Private SSH git ls-remote	Successful
Engine checkout versus origin/main	Synchronized when last verified
GPG key for Git authentication	Not applicable; GPG signs commits but does not authenticate Git transport

Release and deployment flow

1. Verify changes locally with npm test, npm run lint, and npm run build.

2. Commit and push main to Gitea.

3. On the engine server, update only with a fast-forward pull:

   git -C /opt/website-engine-control-plane pull --ff-only
   

4. Install locked dependencies and rebuild when application code or dependencies changed:

   cd /opt/website-engine-control-plane
   npm ci --no-audit --no-fund
   npm test
   npm run lint
   npm run build
   

5. Install changed versioned service or Nginx configuration, validate it, and restart only the affected service.

6. Trigger and validate an engine build before activation.

7. Activate the release. Activation updates the current symlink atomically, and Nginx begins serving it on port 3000 without copying files into the Nginx configuration directory.

Do not deploy an unpushed working tree or bypass --ff-only. The server checkout should remain reproducible from Gitea.

Verification checks

Run these checks on the engine server after deployment:

systemctl is-active labyricorn-control-plane.service nginx
ss -lntp | grep -E '(:80|:3000|:3001)'
curl -fsS http://127.0.0.1:3001/api/health
curl -fsSI http://127.0.0.1:3000/
readlink -f /var/lib/website-engine/builds/current
git -C /opt/website-engine-control-plane rev-list --left-right --count HEAD...origin/main

Expected listeners:

• Nginx on 0.0.0.0:80 and 0.0.0.0:3000, with IPv6 equivalents;
• Node on 127.0.0.1:3001 only.

An unknown rendered-site route should return HTTP 404 and the generated 404.html page.

Security rules

• Add only public keys to Gitea. Never transfer or paste the private deploy key.
• Keep the deploy key read-only unless a separately reviewed workflow genuinely requires pushes from the engine.
• Prefer the private SSH remote for engine-to-Gitea Git traffic.
• Treat plain private HTTP as unencrypted. Do not send tokens through it when SSH can perform the operation.
• Do not disable strict host-key checking. If the Gitea host fingerprint changes, stop and verify the new fingerprint from the Gitea console before updating known_hosts.
• Do not place secrets in Git remotes, environment files committed to Git, service definitions, command histories, or documentation.
• Preserve the dedicated website-engine service account and its restrictive file permissions.

Key rotation

1. Generate a new key under a new filename on the engine server.
2. Add only the new public key to Gitea as a read-only deploy key.
3. Verify SSH authentication and git ls-remote with the new key.
4. Update the repository-local core.sshCommand to the new key.
5. Verify a normal fetch through the private remote.
6. Revoke the old deploy key in Gitea.
7. Remove the old private key only after the new path has been verified end to end.

Do not overwrite the active private key in place during rotation; retaining the old key until verification provides a safe rollback path.

Repository theme build contract

Production presentation is owned by .theme/ in the repository selected by packages/site-definition/site.yml. There is no built-in production renderer or preview fallback. Configuration is Git-owned and read-only in control-plane v1; PUT /api/site-config returns 405 E_CONFIG_READ_ONLY.

Each build resolves LABYRICORN_SITE_DEFINITION_REF (default HEAD) once to a full commit and materializes an archive of that commit. Set these variables when the site definition is not the application checkout:

LABYRICORN_SITE_DEFINITION_REPOSITORY=/srv/site-definition
LABYRICORN_SITE_DEFINITION_REF=refs/heads/main
LABYRICORN_SITE_DEFINITION_PATH=packages/site-definition
LABYRICORN_BUILD_ROOT=/var/lib/website-engine/builds

The artifact ID is derived from the canonical input descriptor. Operational run IDs and execution time do not enter artifact bytes. The release layout is:

<build-root>/
├── releases/sha256-<input-digest>/
├── staging -> releases/sha256-<input-digest>/
└── current -> releases/sha256-<input-digest>/

Staging and promotion verify every entry in checksums.json. A modified artifact is rejected. Promotion updates current; it does not render or copy a second release. On startup, valid staging and current pointers are rediscovered and verified.

Theme failure behavior

• Missing or invalid .theme/theme.yml: the build fails before a release directory is created.
• Unknown manifest keys, unsafe paths, missing templates/assets, or strict-Liquid errors: the build fails closed.
• No selected staging/live release: /api/live-site/html returns 503 E_RELEASE_UNAVAILABLE.
• Unknown route: the selected artifact's 404.html is returned with HTTP 404.
• Rollback activates a previously verified immutable release; it never recreates or mutates release bytes.

Repository themes may contain Liquid templates and manifest-declared browser assets. They may not execute build programs, declare custom filters/tags, use dynamic includes, emit inline scripts/event handlers/styles, or reference undeclared local dependencies. HTTPS fonts require a pinned SHA-256 checksum and are cached only after verification.