This repository is the canonical source for www.labyricorn.com. It contains the editable Lektor project—not generated production files—and is also the primary operations guide for building, deploying, verifying, rolling back, and troubleshooting the site.

Quick reference

Item	Value
Production URL	`https://www.labyricorn.com`
Canonical Git repository	`https://git.labyricorn.com/Labyricorn/labyricorn-site.git`
Production branch	`main`
Production host	`ubuntu2-47` (`10.138.2.47`)
Repository checkout	`/srv/labyricorn/repo`
Active document root	`/srv/labyricorn/current`
Release storage	`/srv/labyricorn/releases`
Deployment command	`sudo deploy-labyricorn`
Rollback listing	`sudo rollback-labyricorn`
Rollback command	`sudo rollback-labyricorn RELEASE_NAME`
Deployment log	`/srv/labyricorn/logs/deploy.log`
nginx site config	`/etc/nginx/sites-available/labyricorn`
Lektor executable	`/srv/labyricorn/.local/bin/lektor`
Deployment account	`labyricorn-deploy`

Architecture

Gitea: Labyricorn/labyricorn-site (main)
              |
              | authenticated Git fetch
              v
 /srv/labyricorn/repo
              |
              | Lektor static build
              v
 /srv/labyricorn/releases/<UTC timestamp>-<commit>
              |
              | validated, atomic symlink switch
              v
 /srv/labyricorn/current
              |
              | read-only static files
              v
            nginx :80
              |
              | Cloudflare proxy and public TLS
              v
 https://www.labyricorn.com

Gitea is the source of truth. The production checkout is disposable state and must not contain unpublished edits during deployment. Lektor's development server is not part of production serving; nginx serves generated files only.

Repository contents

Labyricorn.lektorproject  Lektor project definition
content/                  Editable site content (`contents.lr` files)
models/                   Lektor content models
templates/                Jinja templates
assets/static/            CSS, favicon, and other static source assets
.gitignore                Excludes generated and local files
README.md                 This runbook

Generated output such as build/, dist/, and .lektor/ is intentionally ignored and must not be committed.

Routine content workflow

The preferred workflow is to edit in a normal Git checkout, review the changes, push them to main, and then deploy on the production host.

git switch main
git pull --ff-only origin main

# Edit content, templates, models, or assets.

lektor build --output-path build
git status
git add <reviewed-files>
git commit -m "Describe the site change"
git push origin main

Then deploy on ubuntu2-47:

sudo deploy-labyricorn

Do not edit generated files under /srv/labyricorn/current or /srv/labyricorn/releases. They will be replaced by deployment and are not source-controlled.

Local development

Install Lektor in an isolated Python environment. For example:

python3 -m venv .venv
. .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install lektor

Run the local editor and preview server:

lektor server

Run a production-style build:

lektor build --output-path build
test -f build/index.html

The project currently has no third-party Lektor packages or plugins.

Production filesystem and permissions

/srv/labyricorn/
├── repo/                 Git working copy (not web-accessible)
├── releases/             Successful immutable builds
├── current -> releases/<release>
├── logs/
│   ├── deploy.log
│   └── deploy.lock
├── .local/bin/lektor     pipx-installed Lektor
├── .netrc                Git HTTPS credential (0600; secret)
├── .ssh/                 Reserved deployment SSH material
└── README-DEPLOYMENT.md  Supplemental host-local notes

The labyricorn-deploy account owns the checkout, releases, logs, Lektor environment, and Git credential. nginx's www-data account can traverse the base path and read active generated files, but cannot read the source checkout or credential. nginx cannot modify either source or releases.

Deployment behavior

/usr/local/bin/deploy-labyricorn performs the following sequence:

Changes from root to the unprivileged labyricorn-deploy account.
Acquires an exclusive flock lock to prevent concurrent deployments.
Refuses to proceed if the working tree contains any tracked or untracked changes.
Fetches and prunes origin.
Verifies that origin/main exists.
Switches to local main and resets it exactly to origin/main.
Locates exactly one .lektorproject file instead of assuming its name.
Creates a unique release named with UTC time and the 12-character commit ID.
Runs Lektor with an explicit output directory.
Requires the build to succeed and contain index.html.
Writes the full commit ID to .labyricorn-commit in the release.
Atomically replaces /srv/labyricorn/current with a relative symlink to the new release.
Keeps the active release plus recent prior releases, targeting five total.
Logs start, failure, success, release name, and commit.

Run it with:

sudo deploy-labyricorn

A failed build is removed before activation. The current symlink is not changed, so the previously working site remains online.

Deployment preflight

sudo -u labyricorn-deploy env HOME=/srv/labyricorn \
  git -C /srv/labyricorn/repo status --short --branch

sudo -u labyricorn-deploy env HOME=/srv/labyricorn \
  git -C /srv/labyricorn/repo fetch origin

The status must be clean. Do not bypass a dirty-tree refusal. Determine whether the files are legitimate unpublished admin edits, then commit and push them or remove them deliberately before deployment.

Verify a deployment

readlink -f /srv/labyricorn/current
cat /srv/labyricorn/current/.labyricorn-commit

sudo -u labyricorn-deploy env HOME=/srv/labyricorn \
  git -C /srv/labyricorn/repo rev-parse HEAD

sudo -u labyricorn-deploy env HOME=/srv/labyricorn \
  git -C /srv/labyricorn/repo ls-remote origin refs/heads/main

nginx -t
systemctl is-active nginx

curl -I -H 'Host: www.labyricorn.com' \
  -H 'X-Forwarded-Proto: https' http://127.0.0.1/

curl -I https://www.labyricorn.com/

The commit in the active release, local HEAD, and remote main should match. The local nginx request with X-Forwarded-Proto: https should return 200. Public HTTP should redirect to HTTPS, and public HTTPS should return 200.

Rollback

List releases and identify the current one:

sudo rollback-labyricorn

Switch atomically to a previous successful release:

sudo rollback-labyricorn 20260812T024742Z-92475d2ec759

Use a release name printed by the listing command. Rollback does not build, fetch, or alter Git. It only changes the current symlink and records the event in the deployment log.

Verify afterward:

readlink -f /srv/labyricorn/current
cat /srv/labyricorn/current/.labyricorn-commit
curl -I -H 'Host: www.labyricorn.com' \
  -H 'X-Forwarded-Proto: https' http://127.0.0.1/

A normal deployment after rollback rebuilds and reactivates the latest remote main commit.

nginx and URL handling

The site configuration is:

/etc/nginx/sites-available/labyricorn
/etc/nginx/sites-enabled/labyricorn -> ../sites-available/labyricorn

nginx serves /srv/labyricorn/current and supports:

index.html at directory-style URLs;
direct static assets;
optional .html fallback;
generated /404/index.html with an HTTP 404 response;
disabled directory browsing;
rejection of hidden paths and common executable-script extensions;
dedicated access and error logs.

Always validate before reloading nginx:

nginx -t
systemctl reload nginx

nginx logs:

/var/log/nginx/labyricorn-access.log
/var/log/nginx/labyricorn-error.log

HTTPS, Cloudflare, and DNS

Current production HTTPS terminates at Cloudflare. The origin nginx process currently listens on port 80, not 443. For www.labyricorn.com, nginx uses Cloudflare's X-Forwarded-Proto header to distinguish public HTTP from HTTPS:

public HTTP redirects to https://www.labyricorn.com;
Cloudflare-originated HTTPS requests are served without a redirect loop;
labyricorn.com is configured at nginx to redirect to the canonical HTTPS www hostname.

The apex labyricorn.com did not resolve during initial setup. Its redirect will not work publicly until DNS/Cloudflare has an apex record routed to this origin.

Certbot and the nginx integration are installed, and certbot.timer is enabled, but no origin certificate was issued during setup. Do not run Certbot blindly behind Cloudflare. First decide whether the desired origin mode is a Cloudflare Origin CA certificate, Let's Encrypt with a compatible challenge, or direct DNS without the proxy. After changing TLS, test both origin and public behavior for redirect loops.

Useful checks:

getent ahosts www.labyricorn.com
getent ahosts labyricorn.com
curl -I http://www.labyricorn.com/
curl -I https://www.labyricorn.com/
systemctl status certbot.timer
certbot certificates

Firewall and exposed services

UFW is active with default incoming denial. Allowed services are:

OpenSSH: TCP 22
nginx HTTP/HTTPS profile: TCP 80 and 443

Inspect current state:

ufw status verbose
ss -lntup

Lektor's development port must never be opened in UFW or bound to 0.0.0.0.

Optional server-side Lektor admin

An optional unit exists at:

/etc/systemd/system/lektor-admin.service

Current state: disabled and stopped. When started, it runs as labyricorn-deploy and binds only to 127.0.0.1:5000. It is not proxied by nginx and production remains independent of it.

Start it temporarily:

systemctl start lektor-admin
systemctl status lektor-admin
ss -lntp | grep ':5000'

Access it securely through an SSH tunnel from an administrator workstation:

ssh -L 5000:127.0.0.1:5000 [email protected]

Then browse to http://127.0.0.1:5000 on that workstation.

Stop it when finished:

systemctl stop lektor-admin

Committing admin edits made on the server

Lektor can edit files, but it does not automatically create Git commits or push them. The deployment account intentionally has authenticated Git write access so a controlled admin workflow can publish server-side edits.

Review everything before committing:

sudo -u labyricorn-deploy env HOME=/srv/labyricorn bash
cd /srv/labyricorn/repo
git status
git diff
git add <reviewed-files>
git commit -m "Describe the admin edit"
git push origin main
exit

Then run sudo deploy-labyricorn. The deployment script will refuse to run while uncommitted admin edits remain. This is deliberate protection against silently discarding edits when it resets to origin/main.

Before enabling persistent or public admin access, add authentication and make a separate security decision. Do not publish port 5000 directly and do not proxy an unauthenticated editor through nginx.

Git authentication and secrets

The private Gitea repository is accessed over HTTPS because the Gitea SSH port was not reachable through git.labyricorn.com during setup.

Credential purpose and location:

Purpose	Location	Permissions
Gitea HTTPS fetch/push	`/srv/labyricorn/.netrc`	`0600`, owned by `labyricorn-deploy`
Reserved SSH identity	`/srv/labyricorn/.ssh/id_ed25519`	`0600`, owned by `labyricorn-deploy`

Never place token contents, passwords, or private keys in this README, Git URLs, shell scripts, logs, commits, issue trackers, or command history.

Check credential-file metadata without printing its contents:

stat -c '%A %U:%G %n' /srv/labyricorn/.netrc

If the Gitea token is rotated, replace .netrc as labyricorn-deploy, preserve mode 0600, and verify both fetch and push access before revoking the old token. Avoid printing either credential during rotation.

Logs and diagnostics

# Deployment and rollback history
tail -n 100 /srv/labyricorn/logs/deploy.log

# nginx service and request errors
journalctl -u nginx --since today
tail -n 100 /var/log/nginx/labyricorn-error.log

# Optional admin service
journalctl -u lektor-admin --since today

# Repository state
sudo -u labyricorn-deploy env HOME=/srv/labyricorn \
  git -C /srv/labyricorn/repo status

# Active release
readlink -f /srv/labyricorn/current
cat /srv/labyricorn/current/.labyricorn-commit

# Services and sockets
systemctl status nginx
systemctl status certbot.timer
ss -lntup

Troubleshooting

Deployment says the repository is dirty

Inspect as the deployment account:

sudo -u labyricorn-deploy env HOME=/srv/labyricorn \
  git -C /srv/labyricorn/repo status --short

If these are intended admin edits, review, commit, and push them. Otherwise, identify their origin before removing anything. Never use a destructive reset until you are certain no work needs to be preserved.

Git fetch or push fails

Check DNS, HTTPS, the remote URL, and credential permissions without displaying the secret:

getent ahosts git.labyricorn.com
curl -I https://git.labyricorn.com/
sudo -u labyricorn-deploy env HOME=/srv/labyricorn \
  git -C /srv/labyricorn/repo remote -v
stat -c '%A %U:%G %n' /srv/labyricorn/.netrc

An authentication failure usually means the token was revoked, expired, or lacks repository access.

Lektor build fails

Run a disposable diagnostic build; do not build into current:

test_dir=$(mktemp -d /srv/labyricorn/releases/.diagnostic.XXXXXX)
chown labyricorn-deploy:labyricorn-deploy "$test_dir"
sudo -u labyricorn-deploy env HOME=/srv/labyricorn \
  /srv/labyricorn/.local/bin/lektor \
  --project /srv/labyricorn/repo/Labyricorn.lektorproject \
  build --output-path "$test_dir"

Inspect the error and source. Remove the diagnostic directory only after confirming its exact path and that it is not referenced by current.

nginx returns an error

nginx -t
systemctl status nginx
readlink -f /srv/labyricorn/current
test -f /srv/labyricorn/current/index.html
tail -n 100 /var/log/nginx/labyricorn-error.log

Do not point nginx at the Git checkout or start Lektor as a production server.

Public HTTPS loops or behaves differently from local HTTP

Check Cloudflare proxy mode and the forwarded protocol behavior:

curl -I -H 'Host: www.labyricorn.com' http://127.0.0.1/
curl -I -H 'Host: www.labyricorn.com' \
  -H 'X-Forwarded-Proto: https' http://127.0.0.1/
curl -I http://www.labyricorn.com/
curl -I https://www.labyricorn.com/

The first origin request should redirect; the simulated Cloudflare HTTPS request should return the site. A different result indicates a proxy/header or nginx configuration problem.

Future automatic deployment

Manual deployment is complete and remains the supported path. Automatic deployment is not configured.

The preferred future design is a Gitea Action triggered only by pushes to main, invoking deploy-labyricorn through a dedicated, narrowly restricted SSH identity. A webhook receiver is acceptable only if it authenticates a secret, verifies the repository and branch, never executes payload text, and runs with minimal privileges. Never expose a general shell endpoint or an unauthenticated HTTP deployment URL.

Change-control checklist

Before changing deployment, nginx, TLS, firewall, credentials, or admin access:

Inspect the live state and preserve unrelated configuration.
Back up the specific file being changed.
Validate syntax before reload or restart.
Keep SSH reachable before firewall changes.
Verify local origin behavior and public Cloudflare behavior separately.
Record operational changes in this README.
Commit and push source documentation to main.

This README describes the current verified installation on ubuntu2-47. Update it whenever the operational architecture changes.