Hive Team Server Deployment

Deploy cortexa-hive-team — the hosted multi-tenant Hive v2 server — as a Linux daemon with systemd on a VPS, behind nginx or Caddy for TLS termination. This is a separate binary and a separate deployment target from the legacy cortexa-hive relay (see docs/deployment/hive-server.md); the two are unrelated services and do not share data, ports, or config files. This doc's structure mirrors that legacy doc, but nothing else is shared — hive-team is a clean-break crate (Postgres, E2E design; see docs/superpowers/specs/2026-09-09-hive-v2-team-sync-design.md).

VPS Prerequisites

Postgres. cortexa-hive-team connects using the configured database URL and runs its own embedded migrations on startup (sqlx::migrate!, no manual SQL needed). The current migrations use PostgreSQL 13-compatible SQL; validate them and the integration suite on the installed version before deployment. Create a dedicated database + role — do not reuse the postgres superuser role in production:
bash
```
sudo apt install -y postgresql
sudo -u postgres createuser --pwprompt cortexa_hive_team
sudo -u postgres createdb --owner=cortexa_hive_team cortexa_hive_team
```
A reverse proxy (Caddy or nginx) doing TLS termination, forwarding :443 to the server's public API port (7575). The code default (bind in src/config.rs) is 0.0.0.0:7575 — reachable from anywhere, not just the proxy. The recommended production posture binds it to loopback instead (bind = "127.0.0.1:7575" in hive-team.toml), since the proxy normally runs on the same box and reaches it via loopback either way. See the Configuration example and Reverse Proxy below.
:7576 (the operator plane) must never be exposed to the internet. It binds to 127.0.0.1 by default (operator_bind in hive-team.toml) and is reachable only via SSH tunnel or from the box itself. There is no reverse proxy rule for it, no firewall allow rule for it, and no legitimate reason to change that binding to 0.0.0.0 on a production host. The operator plane is a second, unauthenticated-by-network-boundary attack surface if exposed — its only defense past the loopback boundary is the x-operator-token header, and unlike the legacy relay's admin token this one is a tenant control plane (suspend/unsuspend arbitrary teams), so treat the loopback restriction as load-bearing, not incidental.

Building the Web Console

cortexa-hive-team embeds the Hive console — a React SPA built from hive-console/ — into the binary at compile time (hive-team/build.rs reads hive-console/dist/). Build the console before compiling the server, on every build (native and cross-compiled):

bash

npm ci
npm run console:build      # writes hive-console/dist/
CORTEXA_REQUIRE_CONSOLE=1 cargo build -p cortexa-hive-team --release

CORTEXA_REQUIRE_CONSOLE=1 makes the build fail loudly if hive-console/dist is missing, instead of silently embedding a placeholder page. Always set it for a real deployment build; without it, a missing dist/ degrades to a placeholder that says the console isn't built, which is easy to miss. It does not detect a stale dist/ (one built from older sources): the binary embeds whatever is there, so run npm run console:build right before every cargo build.

Hosts with an old Node.js (e.g. the Pi's v16)

hive-console's Vite build needs a newer Node than some deployment hosts ship. If npm run console:build fails or refuses to run, build the console on another machine (your dev box, or CI) with a supported Node version, then copy the resulting hive-console/dist/ directory into this host's checkout at the same path before running cargo build here. The dist/ directory is a plain static bundle — nothing about copying it in is Node-version-specific.

Building the Binary

Native Build

bash

cargo build -p cortexa-hive-team --release

The binary lands at target/release/cortexa-hive-team.

Cross-Compilation

For a Linux ARM64 VPS built from a different architecture:

bash

rustup target add aarch64-unknown-linux-gnu
cargo build -p cortexa-hive-team --release --target aarch64-unknown-linux-gnu

The binary lands at target/aarch64-unknown-linux-gnu/release/cortexa-hive-team.

Install

bash

sudo cp target/release/cortexa-hive-team /usr/local/bin/
sudo chmod +x /usr/local/bin/cortexa-hive-team

Configuration: `hive-team.toml`

cortexa-hive-team reads its config from a TOML file (default hive-team.toml in the working directory; override with --config). Unlike the legacy relay, the config file is not auto-generated — create it before first run:

toml

# hive-team.toml — EXAMPLE ONLY. Replace every placeholder below.

# Postgres connection string. Use a dedicated, least-privilege role — never
# the Postgres superuser.
database_url = "postgres://cortexa_hive_team:REPLACE_WITH_A_STRONG_DB_PASSWORD@127.0.0.1:5432/cortexa_hive_team"

# Public API — the address the reverse proxy forwards to. Recommended:
# loopback, since the proxy runs on this same host and reaches it via
# 127.0.0.1 regardless. This gives the API port the same defense-in-depth
# as the operator plane — nothing to firewall, because nothing but the
# proxy can reach it. Use "0.0.0.0:7575" ONLY if the reverse proxy runs on
# a different machine, and in that case you MUST also firewall 7575 to
# accept only the proxy host. The code default if this key is omitted is
# "0.0.0.0:7575" — do not rely on the default in production; set it
# explicitly.
bind = "127.0.0.1:7575"

# Operator plane — tenant management (suspend/unsuspend teams, list teams).
# MUST stay bound to loopback in production. Never widen this to 0.0.0.0 and
# never add a reverse-proxy or firewall rule that reaches it from outside
# the box. Reach it via `ssh` + `cortexa-hive-team operator ...` (see below)
# or an SSH tunnel.
operator_bind = "127.0.0.1:7576"

# Operator token — gates every operator-plane route except /op/health.
# Generate with: openssl rand -hex 32
# MUST be a long random string. This is a tenant-suspension credential, not
# a convenience password — treat it like a root credential and store it in
# a secrets manager, not in shell history or a ticket.
operator_token = "REPLACE_WITH_32+_RANDOM_HEX_CHARS"

# MUST be false in production. dev_mode=true makes /api/auth/register return
# the email-verification token directly in the HTTP response body (so local
# dev doesn't need a mail server) — leaving it true in production lets
# anyone self-verify any email address they register, skipping verification
# entirely.
dev_mode = false

# Optional — pairs this Team server with a relay's /admin console. Set to the
# SAME UUID as the relay's `server_admin_team_id` in `hive.toml`. Owners of
# this team then get the full Hive console's server-management panels
# (relay overview, signal feed, peers, projects/grants, shares, publications,
# deployments, site) at /team/, and the relay's /admin redirects there
# instead of serving its own trimmed fallback page. Find the team's UUID with
# `cortexa-hive-team operator list-teams` once the team exists; leave unset
# if this Team server has no paired relay.
# server_admin_team_id = "11111111-1111-4111-8111-111111111111"

# Optional — the console's own public HTTPS URL, watched for certificate
# expiry (the `tls` service on the console's Health view). Defaults to
# `browser_origin` when unset, so most deployments never need to set this
# separately — it exists only for the rare case where the public URL differs
# from `browser_origin` (the origin browser sessions are cookie-scoped to).
# An `http://` URL (or leaving both this and `browser_origin` unset) means
# there's no certificate to watch, so no `tls` entry is reported at all —
# not even `Unknown`.
# public_url = "https://hive-team.example.com"

# Optional — pairs this Team server with a relay for the console's Health
# page (RelayCollector — peers online/total, outbox pending, file bytes,
# shares, and a req/min rate). This is independent of server_admin_team_id
# above: that grants the console UI's relay panels (peers/projects/grants) to
# a team's owners, while these two keys only feed Health's monitoring
# metrics — a deployment can have either, both, or neither. relay_url is the
# relay's own base URL (not the public hostname if the relay isn't reachable
# there — typically loopback, since the sampler runs on the same host).
# relay_admin_token is the SAME admin_token the relay's own hive.toml already
# has; it's sent as a Bearer token to the relay's /admin/stats. Never commit
# either to version control; relay_admin_token is redacted from every log
# line and Debug output. A plain http:// relay_url to anything other than
# loopback is allowed but logs a startup warning: the token then crosses the
# network unencrypted. A relay older than request counting reports no req/min
# rate at all (rather than a flat zero).
# relay_url = "http://127.0.0.1:7474"
# relay_admin_token = "REPLACE_WITH_THE_RELAY_hive.toml_admin_token"

# Optional — path to a JSON status file written by separate backup tooling,
# read every sample for the `backup` service (`backup.age_hours` metric).
# Unset (the default) reports `backup` as `Unknown`, not `Down` — nothing
# was configured, so nothing failed. See "Backup status file format" below
# for the exact schema.
# backup_status_path = "/var/lib/cortexa-hive-team/backup-status.json"

# Optional — the file holding the key that encrypts notification-channel
# secrets (SMTP password, webhook signing key, Telegram bot token) in the
# database. Created on first start if missing (32 random bytes, hex, mode
# 0600). Defaults to `secrets.key` next to this config file; set it
# explicitly when that directory isn't writable by the service (the systemd
# unit below makes /etc/cortexa-hive-team read-only). Back it up separately
# from the database — see "Alerts and notifications" below.
# secrets_key_path = "/var/lib/cortexa-hive-team/secrets.key"

# Test-only. Base URL for the Telegram Bot API; unset means
# https://api.telegram.org. The test suite points it at a local sink —
# never set it in production.
# telegram_api_base = "http://127.0.0.1:9"

# Optional. Abuse limits for the unauthenticated auth routes; the values
# shown are the defaults. A bucket allows `burst` requests, then one more
# every `refill_secs`.
[auth_limits]
login_per_ip = { burst = 10, refill_secs = 6 }             # login, verify, browser login
login_global_fallback = { burst = 120, refill_secs = 1 }    # used when the client IP is unknown
register_per_ip = { burst = 5, refill_secs = 720 }
register_global = { burst = 30, refill_secs = 120 }        # always applies, IP known or not
failures_per_client_free = 5                # failed logins per (account, IP) before backoff
failures_per_client_max_lock_secs = 900     # backoff doubles from 1s up to this cap
failures_per_account_free = 25              # failed logins per account, all IPs
failures_per_account_max_lock_secs = 60
invites_per_account = { burst = 30, refill_secs = 120 }    # team invites per account: 30 at once, then 30 an hour

Rate limits and login backoff are kept in memory and reset on restart. A limited sign-in request gets 429 with the wait in the error message (the invite limit says to wait a few minutes instead). Backoff never locks an account for longer than its cap after the last real failure, and an attempt refused during backoff doesn't extend it; failures from one address only back off that address, so an attacker can't lock the real user out from elsewhere. See Client addresses for the proxy header these limits need.

Warning — production requirements, not suggestions:
operator_token must be a long, random, unique secret (openssl rand -hex 32 or equivalent). It authorizes suspending/unsuspending any tenant on the service.
dev_mode must be false in production. Leaving it true defeats email verification service-wide.
Never commit a populated hive-team.toml to version control. Keep it readable only by the service user (see systemd unit below).

First Run

With Postgres reachable and hive-team.toml in place:

bash

./cortexa-hive-team --config /path/to/hive-team.toml

On startup the server:

Connects to Postgres using database_url.
Runs embedded schema migrations (hive-team/migrations/) — additive, idempotent, safe to run on every restart.
Binds the operator plane on operator_bind (default 127.0.0.1:7576).
Binds the public API on bind (code default 0.0.0.0:7575 if unset — the example config above overrides this to 127.0.0.1:7575, the recommended production setting).

Pre-mailer Operation

This section applies when no mail server is configured — no enabled email channel (see Alerts and notifications), no public_url/browser_origin, or the server's approval mode left at its default, "An operator approves new accounts". It is also what happens when approval mode is "email" but mail stops working: registration falls back to operator approval rather than leaving someone stuck.

Email verification tokens are generated and hashed into accounts.verify_token_hash, but with dev_mode = false (the production setting) they are never returned in an API response and never logged. Without a mail server there is no way for a registrant to receive their token, so the operator approves a new account manually — from the console (Settings → Account requests → Approve) or through the private operator CLI:

bash

cortexa-hive-team --config /etc/cortexa-hive-team/hive-team.toml operator pending-accounts
cortexa-hive-team --config /etc/cortexa-hive-team/hive-team.toml operator approve-account --email teammate@example.com
# Alternatively, select the exact account ID returned by pending-accounts:
cortexa-hive-team --config /etc/cortexa-hive-team/hive-team.toml operator approve-account --account-id ACCOUNT_UUID

Run this on the server using the protected service configuration. The CLI authenticates to the loopback operator listener with the token from that file; do not copy the operator token into browser code or expose the operator routes through the proxy. Approval accepts exactly one email or account ID and only updates a pending, active account. It clears the unused verification-token hash and enables login. The registration response reports approval: manual_operator; no email is sent. Never trust a value someone emails or messages you claiming to be their token — there is no token to check it against; you are vouching for the identity directly. Only do this for an identity you have confirmed out-of-band (a known teammate, a verified support ticket, etc.) — this bypasses email verification entirely, so treat it as a manual identity attestation, not a formality.

CLI Usage

cortexa-hive-team [--config <path>] [COMMAND]

Flag	Default	Purpose
`--config`	`hive-team.toml`	Config file path

Subcommands:

Command	Purpose
`serve` (default)	Run the API server and the operator plane
`operator pending-accounts`	List active accounts awaiting manual approval
`operator approve-account --email <email>`	Manually approve one pending account; alternatively use `--account-id <uuid>`
`operator list-teams`	List every tenant, status, member count
`operator suspend-team <id>`	Suspend a team (blocks its members' API access; suspended teams 404 to their own members, same as a nonexistent team)
`operator unsuspend-team <id>`	Restore a suspended team

The operator subcommand talks to the running instance's operator plane over HTTP using operator_bind and operator_token from the same config file — run it on the box itself, or over an SSH tunnel to operator_bind:

bash

# On the VPS, or via `ssh -L 7576:127.0.0.1:7576 user@vps`:
cortexa-hive-team operator list-teams
cortexa-hive-team operator suspend-team 3fa85f64-5717-4562-b3fc-2c963f66afa6
cortexa-hive-team operator unsuspend-team 3fa85f64-5717-4562-b3fc-2c963f66afa6

Verify

bash

curl http://127.0.0.1:7575/health
# ok
curl http://127.0.0.1:7576/op/health
# ok  (no token required — liveness only, see operator.rs)

systemd Service

These instructions work on Ubuntu, Debian, and any systemd-based distro.

1. Create a dedicated user

bash

sudo useradd --system --no-create-home --shell /usr/sbin/nologin cortexa-hive-team
sudo mkdir -p /etc/cortexa-hive-team
sudo chown cortexa-hive-team:cortexa-hive-team /etc/cortexa-hive-team

Place the populated hive-team.toml at /etc/cortexa-hive-team/hive-team.toml and lock it down:

bash

sudo chmod 600 /etc/cortexa-hive-team/hive-team.toml
sudo chown cortexa-hive-team:cortexa-hive-team /etc/cortexa-hive-team/hive-team.toml

2. Create the service unit

bash

sudo tee /etc/systemd/system/cortexa-hive-team.service > /dev/null << 'EOF'
[Unit]
Description=Cortexa Hive Team server (hosted multi-tenant Hive v2)
After=network.target postgresql.service
Wants=network-online.target

[Service]
Type=simple
User=cortexa-hive-team
Group=cortexa-hive-team
ExecStart=/usr/local/bin/cortexa-hive-team --config /etc/cortexa-hive-team/hive-team.toml
Restart=on-failure
RestartSec=5
StandardOutput=journal
StandardError=journal

# Hardening
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
ReadOnlyPaths=/etc/cortexa-hive-team
# /var/lib/cortexa-hive-team, owned by the service user, mode 0700: the one
# writable path, for the notification secrets key file.
StateDirectory=cortexa-hive-team
StateDirectoryMode=0700
PrivateTmp=true

[Install]
WantedBy=multi-user.target
EOF

This mirrors the legacy relay's cortexa-hive.service hardening pattern (docs/deployment/hive-server.md). All team data lives in Postgres; the config directory stays read-only. The only local file the server writes is the notification secrets key, created once on first start, so set secrets_key_path = "/var/lib/cortexa-hive-team/secrets.key" in hive-team.toml to put it in the state directory above. Without that line the key defaults to /etc/cortexa-hive-team/secrets.key, which the unit makes read-only: the server still starts, but channel secrets are unavailable until the key can be created. See The secrets key file.

3. Enable and start

bash

sudo systemctl daemon-reload
sudo systemctl enable cortexa-hive-team
sudo systemctl start cortexa-hive-team

4. Verify

bash

sudo systemctl status cortexa-hive-team
curl http://127.0.0.1:7575/health
journalctl -u cortexa-hive-team -f

Reverse Proxy

The reverse proxy forwards only the public API port (7575). Never add a rule for 7576 (operator plane) — see the prerequisites warning above. With bind = "127.0.0.1:7575" (recommended), both proxy examples below already work unchanged — reverse_proxy/proxy_pass connect via loopback, same as they would to 0.0.0.0:7575.

Phase B adds GET /ws — the live fan-out socket devices hold open for push delivery of new events. It's on the same public API port (7575) as every HTTP route, so it needs no separate proxy block, but the WebSocket upgrade (Connection: Upgrade) does need to survive the proxy hop, which is where Caddy and nginx differ below.

Caddy

Edit /etc/caddy/Caddyfile:

hive-team.example.com {
    reverse_proxy localhost:7575
}

Caddy obtains and renews Let's Encrypt certificates automatically and redirects HTTP to HTTPS with no further configuration. reverse_proxy detects and forwards WebSocket upgrade requests automatically — /ws needs no extra directive here; the block above already handles both plain HTTP and the /ws upgrade.

bash

sudo systemctl reload caddy

nginx

Unlike Caddy, nginx does not forward the Upgrade/Connection headers by default — an explicit map + header block is required, or /ws upgrades will fail (typically a 400 or a connection that never leaves HTTP). Create /etc/nginx/sites-available/cortexa-hive-team:

nginx

map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

server {
    listen 80;
    server_name hive-team.example.com;

    location / {
        proxy_pass http://127.0.0.1:7575;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Required for /ws (and harmless for plain HTTP routes): nginx
        # defaults to HTTP/1.0 to upstream and drops hop-by-hop headers,
        # both of which break the WebSocket upgrade unless overridden here.
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;

        # nginx's 60s default read timeout severs an idle /ws connection —
        # the server does not yet send keepalive pings on it, so a quiet
        # device would otherwise get silently disconnected well before it
        # has anything new to receive. Clients already reconnect and catch
        # up via cursors, so this isn't a correctness dependency, but don't
        # rely on the disconnect-and-reconnect churn in the meantime.
        proxy_read_timeout 3600s;

        # The console's live view (`GET /api/console/live`) is a
        # Server-Sent Events stream. nginx buffers proxied responses by
        # default, which holds events back until a buffer fills, so live
        # updates would stall silently. The server also sends
        # `X-Accel-Buffering: no` on that stream; this makes it explicit.
        # The 3600s read timeout above covers the stream's quiet stretches
        # (it sends a keepalive comment every 25s anyway).
        proxy_buffering off;
    }
}

bash

sudo ln -s /etc/nginx/sites-available/cortexa-hive-team /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx
sudo certbot --nginx -d hive-team.example.com

Unified setup: relay and Team server on one host

When the relay (cortexa-hive, port 7474) and the Team server share a hostname, the relay stays at the site root and the Team server lives under /team/. The console is served by the Team server at /team/; the relay's own /admin now answers with a single 308 to /team/ (keeping the query string, so a return_to survives).

nginx

# Bare /team (the desktop's Team server URL) lands on the console.
location = /team {
    return 308 /team/$is_args$args;
}

location /team/ {
    # The trailing slash strips the /team/ prefix, keeping the signed API
    # paths the desktop client uses unchanged.
    proxy_pass http://127.0.0.1:7575/;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;
    proxy_read_timeout 3600s;
    # Live console updates are Server-Sent Events; buffering would stall them.
    proxy_buffering off;
}

location / {
    proxy_pass http://127.0.0.1:7474;
    # ...the relay's existing proxy settings
}

Remove the old /team → /admin redirects

Earlier unified setups sent the console the other way, with blocks like location = /team { return 308 /admin...; } and location = /team/ { ... } that redirected to /admin. Delete both. With the relay now sending /admin to /team/, keeping them creates a redirect loop and the console is unreachable. Back up the site file first, then sudo nginx -t && sudo systemctl reload nginx.

Smoke checklist (after any nginx or binary change)

Replace hive.example.com with your host:

bash

# 1. /team/ serves the console directly: 200, no redirects.
curl -s -o /dev/null -w '%{http_code} redirects=%{num_redirects}\n' -L https://hive.example.com/team/
#    expect: 200 redirects=0
curl -s https://hive.example.com/team/ | grep -c '<div id="root">'
#    expect: 1

# 2. /admin is exactly one 308 to /team/ (query kept).
curl -sI https://hive.example.com/admin | grep -iE '^(HTTP|location)'
#    expect: HTTP/... 308 and location: /team/
curl -sI 'https://hive.example.com/admin?return_to=%2Fpub%2Fx.html' | grep -i '^location'
#    expect: location: /team/?return_to=%2Fpub%2Fx.html

# 3. No loops: following /admin ends on the console after one hop.
curl -s -o /dev/null -w '%{http_code} redirects=%{num_redirects}\n' -L --max-redirs 5 https://hive.example.com/admin
#    expect: 200 redirects=1  (a "Maximum redirects" error means a loop)

Client addresses and rate limits

Login and registration limits are keyed on the client's IP address. Behind a proxy, the server only sees the proxy's loopback address, so it reads the client address from the proxy's headers — and trusts them only when the connection comes from loopback. From any other peer those headers are ignored, since a client could set them to anything.

With nginx, every location that reaches 7575 needs:

nginx

proxy_set_header X-Real-IP $remote_addr;

The example above already has it. Without it, all clients share the login_global_fallback bucket, so one noisy client can slow everyone's login. Caddy's reverse_proxy sets X-Forwarded-For by itself, and the server uses the last address in it, the one Caddy added. If the proxy runs on another host, it isn't loopback and its headers are ignored: every request is keyed on the proxy's address, which amounts to one shared bucket.

Updating

bash

sudo systemctl stop cortexa-hive-team
sudo cp target/release/cortexa-hive-team /usr/local/bin/
sudo systemctl start cortexa-hive-team

systemctl stop (SIGTERM) drains instead of cutting connections: the server stops accepting, lets in-flight requests finish, closes each /ws socket with code 1001, and closes the database pool. The drain is capped at about 10 seconds, then the process exits anyway. Devices reconnect and resume from their cursors, so nothing is lost either way.

Schema migrations run automatically on every startup and are additive (sqlx::migrate! against hive-team/migrations/); no manual SQL is needed. Postgres data is untouched by a binary update.

Verify after update

bash

sudo systemctl status cortexa-hive-team
curl https://hive-team.example.com/health
journalctl -u cortexa-hive-team -n 20

Security Checklist

[ ] Postgres is a dedicated, least-privilege role/database — never the superuser — reachable only from the app host (pg_hba.conf / firewall).
[ ] bind = "127.0.0.1:7575" in hive-team.toml (the code default is 0.0.0.0:7575 — do not rely on it). The raw HTTP port is never exposed directly to the internet because it never listens on a non-loopback address, the same defense-in-depth as the operator plane. Reverse proxy (Caddy or nginx) terminates TLS in front of it via loopback. If the proxy must run on a different host than the API server, bind = "0.0.0.0:7575" is required instead — in that split-host case a firewall rule restricting 7575 to the proxy host's IP is the fallback control, not optional.
[ ] :7576 (operator plane) is bound to 127.0.0.1 and has no reverse-proxy rule and no firewall allow rule. Reach it only via SSH (locally, or via ssh -L).
[ ] operator_token is long, random, and stored as a secret (chmod 600 hive-team.toml, dedicated service user) — never committed, never logged, never shared as a team credential (it is not one; it is the service operator's credential).
[ ] dev_mode = false in the production config.
[ ] hive-team.toml is readable only by the cortexa-hive-team service user (chmod 600, correct chown).
[ ] The notification secrets key (secrets_key_path) is 0600, owned by the service user, and backed up somewhere other than the Postgres dumps. "Allow LAN webhooks" stays off unless you need a local receiver.
[ ] Firewall allows only 80, 443, and SSH (or your custom SSH port) from the public internet.
[ ] Per the spec's threat model (docs/superpowers/specs/2026-09-09-hive-v2-team-sync-design.md#8-threat-model--explicit-v1-limitations), server compromise yields account emails, team/space membership graphs, metadata, and audit logs — but no plaintext content and no space keys. Postgres backups therefore still warrant standard access controls, but are not a content-confidentiality boundary the way the legacy relay's SQLite file is.

Monitoring

Logs

bash

journalctl -u cortexa-hive-team -f
journalctl -u cortexa-hive-team -n 100
journalctl -u cortexa-hive-team -b

Health Checks

bash

curl -sf http://127.0.0.1:7575/health
curl -sf http://127.0.0.1:7576/op/health   # loopback only — run on the box

Both return the plain string ok when healthy.

Tenant Status

Use the operator CLI (on the box, or over an SSH tunnel) rather than direct Postgres queries where possible — it goes through the same audited path as suspend/unsuspend:

bash

cortexa-hive-team operator list-teams

TLS certificate expiry

When public_url (or browser_origin) is an https:// URL, the sampler connects to that host:port every minute and reads the leaf certificate's not_after. This is a read of whatever certificate the reverse proxy (Caddy or nginx, per Reverse Proxy above) is presenting — it does not manage or renew certificates itself.

Unlike relay/team/database/backup, TLS is not a service on the console's service list — the Health view shows it as a plain metric line ("Certificate: N days left"), so this collector never reports a ServiceStatus. On a successful handshake it emits two points: tls.days_left (whole days until not_after, floored — negative once the certificate has expired, which the console shows as "Certificate expired N days ago") and tls.verified (1 if the leaf verified against the standard web trust store, 0 if it didn't — expired or self-signed). Both are still emitted when verification fails — that's precisely the situation where knowing "how expired" or "how long has this been self-signed" is most useful. A handshake or connection failure (host unreachable, connection refused, or the attempt exceeding its 5s timeout) emits no points at all — there's no certificate to read — logged at debug level only (never surfaced as a service Down, and never containing anything secret). An http://-only deployment (no public_url and no browser_origin, or either set to http://) runs no TLS collector at all — nothing to watch means nothing to report, not even as points.

Backup status file format

When backup_status_path is set, the sampler reads it every minute and reports it as the backup service with a backup.age_hours metric point. hive-team never runs backups itself; this only reads whatever a separate, operator-supplied backup tool last wrote. That tool should atomically replace (not append to) the file after every backup attempt — write a temporary file in the same directory and rename it over the target, so the sampler never reads a half-written file. The sampler reads at most 64 KiB of it; a real status file is a few hundred bytes, so keep it to just this shape:

json

{
  "last_success": "2026-09-27T03:00:00Z",
  "last_attempt": "2026-09-27T03:00:00Z",
  "size_bytes": 104857600,
  "error": null
}

Field	Type	Meaning
`last_success`	RFC 3339 timestamp or `null`	When a backup last completed successfully. `null` if none ever has.
`last_attempt`	RFC 3339 timestamp or `null`	When a backup was last attempted, successful or not. Informational only — not yet surfaced as its own metric or detail.
`size_bytes`	number or `null`	Size of the last successful backup. Informational only — not yet surfaced as its own metric or detail.
`error`	string or `null`	The last attempt's error, if any. Non-null always reports `Down`, with this string as the detail, regardless of how recent `last_success` is.

Reported state:

Up — last_success is within the last 26 hours (a day plus an hour of slack for a slightly slow or delayed job) and error is null.
Down — error is set (detail: the error string), or last_success is older than 26 hours (detail: how long it's been), or last_success has never happened at all (detail: "No backup has completed successfully yet").
Unknown — backup_status_path isn't set (detail: "Backups aren't configured"); or it is set but the file doesn't exist yet (detail: "No backup status yet" — the backup tool hasn't written its first status); or the file exists but can't be read as this exact shape (detail: "Backup status file is unreadable") — this covers invalid JSON, the path pointing at a directory instead of a file, a permission error, and a file over the 64 KiB cap, all alike: the file exists but isn't a usable status file, which is a different situation than "not configured" even though both currently report Unknown.

Pi example: monitoring config keys

A representative slice of /etc/cortexa-hive-team/config.toml on the shared relay+Team host (see docs/deployment/pi-team-hive.md):

toml

public_url = "https://cortexa.2dq.us"
relay_url = "http://127.0.0.1:7474"
relay_admin_token = "<the relay hive.toml admin_token>"
# backup_status_path = "/var/lib/cortexa-backup/status.json"   # set by the backup tooling (deployment sub-project)

Metric retention and storage

Every collector runs once a minute (aligned to the minute boundary), each independently time-boxed to 5s — a slow or failing one reports Unknown for that tick rather than delaying or dropping the others' points. Storage (hive-team/src/monitor/store.rs):

metric_samples (raw, one row per point per minute) — pruned past 7 days.
metric_hourly (one row per (series, labels, hour) holding min/avg/max/n) — rolled up from the raw samples once an hour has fully elapsed, and pruned past 365 days.

Storage cost is an approximation — it scales with how many spaces and route groups a deployment actually has traffic on, both of which sit as one row per value (sync.events_per_min/sync.lag_seconds per space; req.* per route group: auth, console, sync, teams, assets, other). Counting every collector's series (hive-team/src/monitor/{collectors,relay,tls,backup}.rs) for a small deployment — one disk mount, a handful of spaces, all six route groups seeing some traffic — comes to roughly 60 series per minute sample. At a rough 120–150 bytes per metric_samples row (timestamp + series text + labels jsonb + value float8, plus its two indexes), that's on the order of 10–13 MB/day, or under 100 MB for the full 7-day raw retention window. The hourly rollup is far smaller per row-count (60 series × 24/day instead of × 1440/day) — even at 365 days that stays in the tens of MB. None of this scales with account/content volume — only with the number of spaces, disks and active route groups the deployment actually has.

Alerts and notifications

The sampler that feeds Health also evaluates a fixed set of alert rules every minute. Operators manage them in the console under Alerts (Open, Resolved, Rules, Channels) and Settings → Server. Nothing is sent until an operator adds a channel: a fresh install has no channels, quiet hours off, LAN webhooks off and operator approval for new accounts.

Who sees what: server alerts, rules, channels and server settings are for operators only (owners of server_admin_team_id). Team alerts are visible to that team's owners and admins plus operators. Plain members see no alerts in this version, and the Alerts entry only appears in the sidebar for operators and team admins/owners. The same filter applies to the alert event on the console's live stream.

Rules

Rules are built in; each can be turned on or off and has editable thresholds. There's no rule language. Defaults:

Rule	Fires when (default)	Severity	Scope
`service_down`	relay, team or database reported Down for 2 consecutive samples (backup has its own rule)	Critical	server
`disk_full`	a mount is over 85 % full (Warning) / over 95 % (Critical)	Warning / Critical	server, per mount
`backup_stale`	no successful backup in 26 h — only when a backup status file is configured	Warning	server
`tls_expiring`	the certificate expires in under 14 days (Warning) / 3 days (Critical)	Warning / Critical	server
`error_spike`	5xx responses over 5 % of requests in the last 5 min	Warning	server
`auth_abuse`	over 50 rate-limited sign-ins in 10 min	Warning	server
`devices_waiting`	the team has had devices waiting for a space key for 60 min straight	Warning	per team
`account_requests`	an account request has been pending over 24 h	Warning	server
`relay_backlog`	over 500 relay deliveries pending for 15 min	Warning	server

error_spike also has a minimum requests threshold (default 20 in the window): below it the rule stays clear, so one failed request out of two on a quiet server isn't a "50 % error rate".

Missing data is never read as "all clear". If a collector times out or a database check fails, an open alert stays open and no resolve is sent. Two exceptions keep an alert from staying open forever on data that is gone for good:

disk_full: an open mount that stops reporting while other mounts still report has been unmounted or removed, so its alert resolves. If no mount reports at all, that is a collector problem and the alert stays open.
Any rule: an open alert whose data has been missing for 24 hours straight resolves with the reason "no data", and the resolve is sent like any other. The clock is stored on the alert, so a restart doesn't reset it; any real reading in between starts it over.

An open disk_full alert only clears once usage drops 2 points below the Warning threshold (below 83 % with the default 85 %), so a disk hovering at the line doesn't fire and resolve every minute. Sustain counts are kept in memory, so after a restart a sustained rule such as devices_waiting starts counting again (it can fire up to its window later).

Lifecycle

Firing → Resolved automatically when the condition clears, or → Acknowledged when someone acknowledges it (still firing, notifications paused).
One open alert per rule and scope. Fire, clear, fire again gives two alert rows, never two open ones.
Notifications go out when an alert fires, when it escalates above the highest severity it has reached so far (which also clears an acknowledgement), and when it resolves. A de-escalation is recorded but not sent, and so is a return to a level the alert already reached: a disk flapping between 94.9 % and 95.1 % pages Critical once, and an acknowledgement stays in place.
Acknowledging sends the severity the person saw; if the alert has changed since, the console says "This alert changed — refresh to see the latest." instead of acknowledging it. A resolve is only sent if the fire was, and it is routed by the alert's peak severity, so it reaches everyone who heard about the worst of it.
An unacknowledged Critical is re-sent every 4 hours while it keeps firing. Warnings aren't repeated.
Quiet hours (Settings → Server, off by default) hold every Warning and Info notification, resolves included, and release them once quiet hours end. Critical always sends. Times are saved with the browser's UTC offset; a window like 22:00–07:00 wraps past midnight.
Owed notifications live on the alert row, so a restart or a slow mail server doesn't lose them. Criticals are delivered first.
Resolved alerts and their event history are kept for 90 days.

Channels

Channels are added under Alerts → Channels (operator only). Each has a minimum severity (default Warning) and an on/off switch; "Send test" works even on a channel that's off and records the result as the channel's last result. Secrets — the SMTP password, webhook signing key and Telegram bot token — are write-only: the console only ever shows "Saved — hidden", and no API response, log line or live event contains them. Messages carry alert titles, rule names, scopes and counts only, never thought content, keys or passwords.

Sends run on a bounded background worker (queue 256, 4 at a time, 10 s per send), so a slow or broken endpoint never delays the sampler or other notifications. A failure shows as the channel's last error.

Email (SMTP). One email channel per server. It emails people, not a fixed address: server alerts go to the operators, team alerts to that team's owners and admins plus the operators — only active, email-verified accounts. Each person picks their own threshold under Settings → Notifications ("Email me alerts", default Warning and above); the channel's minimum severity is a floor on top of that.

Security	Usual port	Meaning
STARTTLS	587	Plain connect, then STARTTLS is required — never falls back to plain text
TLS	465	TLS from the first byte
None	25	No encryption — only for a relay on a trusted local network

Username and password are optional (leave them empty for an unauthenticated local relay). "From" takes a plain address or Name <address>.

Unlike webhooks, the SMTP host is not checked against private addresses: a mail relay on the local network (or on the same host) is a normal setup. Only operators can configure the channel, and the host is whatever they enter.

Webhook. POST with Content-Type: application/json:

json

{
  "title": "Disk / is 97% full",
  "severity": "critical",
  "scope": "Server",
  "url": "https://hive-team.example.com/team/#/alerts?focus=<alert id>",
  "text": "[Critical] Disk / is 97% full — Server. https://hive-team.example.com/team/#/alerts?focus=<alert id>",
  "kind": "fired"
}

severity is info, warn or critical; kind is fired, escalated, renotify, resolved or test; scope is Server or Team <name>; url is null when no public URL is configured. Any 2xx counts as delivered. Redirects are never followed. Team alerts reach a webhook only when its "Also send team alerts" box is ticked.

A webhook's URL is shown to operators in the console (they configured it), unlike the signing key. For Slack, Discord and similar services the URL itself is the credential — anyone who has it can post to that channel — so treat operator access accordingly, and rotate the URL at the service if an operator account is compromised. The URL never appears in logs, audit rows, live events or error messages.

With Sign requests on, each request carries X-Cortexa-Signature: sha256=<hex> — an HMAC-SHA256 of the exact request body, keyed with the channel's signing key. Verify it over the raw bytes, before parsing the JSON, with a constant-time compare:

python

import hashlib, hmac

def signature_ok(raw_body: bytes, header: str, signing_key: str) -> bool:
    expected = "sha256=" + hmac.new(signing_key.encode(), raw_body, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, header or "")

Telegram. Create a bot with @BotFather (/newbot) and copy its token into the channel's "Bot token". Add the bot to the chat or channel that should receive alerts (as an admin for a channel) and send it a message. For the chat id, use @channelname for a public channel, or read the numeric id (-100… for groups and channels) from https://api.telegram.org/bot<token>/getUpdates after that message — don't paste that URL anywhere it could be logged, since it contains the token. Messages are sent with sendMessage as plain text, link previews off.

telegram_api_base in the config is test-only (it points sends at a local sink in the test suite). Leave it unset in production.

LAN webhooks. By default a webhook whose host resolves to a private, loopback, link-local (including 169.254.169.254 cloud metadata), CGNAT, unique-local, unspecified or multicast address is refused. The console is a web-facing admin tool, and a webhook it can point anywhere on the server's own network would let a compromised or careless operator account probe internal services. The address that passed the check is the one connected to, so DNS can't be switched to a private address in between. If you do want a local receiver (Home Assistant, say), an operator can turn on Allow LAN webhooks under Settings → Server.

The secrets key file (`secrets_key_path`)

Channel secrets are encrypted in Postgres with AES-256-GCM, each bound to its channel id. The 256-bit key lives in the file at secrets_key_path (default: secrets.key next to the config file), never in the database or config:

On first start, if the file is missing and no channel has a saved secret, the server creates it: 32 random bytes as hex, mode 0600, owned by the service user. The directory must be writable by the service for this one write (see the systemd unit below).
Back it up separately from the database dump. The dump holds the ciphertext and the file holds the key: stored together, one leak exposes every channel secret. Keep the key with your other server secrets (operator_token), not next to the pg_dump.
If the file is lost, existing secrets can't be decrypted, and the server refuses to create a new key while any channel still has a saved secret (a new key would silently orphan them). It starts without channel secrets: the journal names the path and how many channels are affected, Alerts → Channels shows "Channel secrets are unavailable", saving a new secret returns an error, and channels that need their secret can't send. Either restore the file from backup and restart, or delete those channels (for a signed webhook, turning signing off and removing its key is enough), restart so a fresh key is created, and re-enter the secrets.
A present but corrupt file is treated the same way; it is never replaced.

Account approval and emails

Settings → Server → New accounts chooses how registrations are approved:

An operator approves new accounts (default) — see Pre-mailer Operation.
New accounts confirm their email address — only selectable once an enabled email channel exists and public_url (or browser_origin) is set. Registration then emails a "Confirm your Cortexa Hive account" link to <public URL>/team/#/verify?token=…; opening it confirms the account, signed out, and the token is removed from the address bar. If the channel is later turned off or deleted, or the send fails, registration quietly falls back to operator approval — the Server card warns when that's the case.

Whenever an enabled email channel and a public URL exist, creating a team invite also emails the invitee a link to <public URL>/team/#/teams?invite=<code> (the console pre-fills the code) and the code itself. The invite is created either way; the email is best-effort and a failure is only logged. Each account may create 30 invites in a burst and then one more every 2 minutes (invites_per_account under [auth_limits]; beyond that, 429), and at most 4 invite emails are sent at once: an invite created while 4 are still in flight is not emailed (emailed: false; share its code by hand).

dev_mode = true still returns the verification token in the register response and sends no email — never use it in production.

Cortexa desktop alerts

Alerts, and requests for space access, can also reach an admin directly on their desktop — alongside, or instead of, email and webhook channels. Opting in is two switches, one on each side:

In the console, under Settings → Notifications, turn on Show alerts in the Cortexa app and pick a minimum severity (default Warning). This is the same per-person notify_prefs row that already guards the email toggle (migration 0005; extended with the desktop columns in migration 0007) — not a new channel.
In Cortexa itself, under Settings → Notifications, turn on Hive alerts and access requests. It's off by default, and it gates only the OS pop-up: with the console-side switch on but this one off, alerts still arrive in the Hive tab's "Needs you" card (and the Hive tab's badge) while Cortexa is running and connected — they just don't pop up a notification.

Only online devices receive anything — a device has to be connected to the server (Cortexa open, not just installed) at the moment an alert fires, escalates or resolves; nothing is queued for a device that was offline. Whether a "Resolved" is sent at all is decided per alert, not per device: if nobody was told about the alert — no email or webhook went out and no desktop was online for it — the resolve is dropped. Once anyone was told, the resolve goes to every desktop recipient online at that moment, including one that was offline for the fire and so shows "Resolved: …" for an alert it never saw fire.

Request access. A member whose device is waiting for a space key can ask an admin from the web console: Request access on the console's Overview page notifies every admin's connected device, naming the space and the requester's own waiting device(s) — a request from someone who can manage the space names every device still waiting in it. Only admin devices online at that moment get the request; it isn't queued. The response says how many were reached, and when none were, the Overview page says so and points at Cortexa's Team Hive settings. To keep a reopened page from re-asking, one requester can ask about one space at most once every 5 minutes; a repeat inside that window is refused (429). The waiting devices don't depend on the request being seen: an admin can find them, and grant access, in Cortexa under Settings → Team Hive → Spaces for any space they manage, whether or not a request reached them.

Whichever admin sees it grants access from their own Cortexa app, the same way any other waiting-device grant works. Grant runs entirely on that admin's device — because keys never leave devices, it never touches the server or the browser. That's also why the console has no Grant button of its own for this case: it can only ask an admin's desktop to do it.

Client onboarding (Cortexa app)

Once a cortexa-hive-team server is reachable (this doc, above) and at least one account has been through email verification (pre-mailer: the Pre-mailer Operation section above, or dev_mode = true for a local/test server), a user connects the Cortexa desktop app to it entirely from Settings → Team Hive — no config files, no CLI. The panel walks through the same steps as team_cmds.rs, in order:

Add account — enter the server URL, email, and display name. This only creates a local, unregistered row (team_add_account); nothing is sent to the server yet.
Register — sets a password and calls the server's /api/auth/register (team_register_account). The account exists server-side now but is unverified.
Verify — enter the verification token. Where does the token come from? Two paths, both landing on the same team_verify_email call:
- Dev/test server (dev_mode = true): register returns the token directly in its response, and the panel pre-fills it — just click Verify.
- Production server (dev_mode = false, the required setting per the Security Checklist): there is no mailer yet (see Pre-mailer Operation above) — the server operator approves the account manually through the private operator CLI. There is no user-facing token to type in this path; the operator's operator approve-account command grants access after their identity review. The user retries login once the operator confirms approval.
Login — authenticates with the password from step 2 (team_login), storing a session token locally (sealed at rest, like the device seeds below).
Enroll device — generates this device's Ed25519 (envelope signing) and X25519 (sealed-box key wrapping) keypairs locally and registers the public halves with the server (team_enroll_device). The private seeds never leave the device; they're sealed at rest in the local vault, same as the session token. One device per enrollment — re-enrolling requires removing and re-adding the account.
Join — paste an invite code (issued by a team owner/admin via POST /api/teams/:id/invites, out of band — email, chat, however) to accept membership and pull in the team's spaces (team_join). Accepting an invite makes the account a team member only — every device-signed space endpoint (space_access) is gated on an explicit space_members row, so the new member can't pull keys or receive anything in a given space until an existing space admin adds them (team_add_space_member, the "Add member" control on each space card in the panel). The team's General space already has the team's owner as an admin, so this step is exactly one admin action, not a bootstrapping problem.
Map space — pick which local project (if any) a joined space's incoming thoughts land in (team_map_space). Defaults to unmapped — a newly joined space does nothing until explicitly mapped. The panel also shows an "Auto-share new thoughts into this space" checkbox (checked={sp.auto_share}), live: checking it turns on outgoing auto-share for every new thought ingested into the mapped project — the share happens automatically, the same way it does for the manual "Share to team" action below. There's no everyone-scope broadcast for Team like Hive's: a space match requires an explicit project mapping, so an item with no project has no auto-share candidates at all, and team:local-only on an item blocks auto-share for it regardless of mapping.
Connect — opens the live WebSocket push connection for every enrolled, enabled account (team_connect) and kicks an initial catch-up pass. From here the cursor sync engine (Task 7) keeps the space's feed current automatically; Sync now (team_sync_now) forces an immediate catch-up pass without waiting for the next push.
Per-item Share / Unshare — beyond auto-share, any thought's detail view has a "Team" action menu (next to the Hive "Share" menu) listing every space this account is a non-viewer member of (team_list_spaces_local) as <team> / <space> entries; picking one shares that single thought immediately (team_share_item). Once shared, the same menu offers "Unshare from <team> / <space>" for each space it's currently shared to (team_unshare_item), which tombstones the share for that space's peers. The menu is hidden on thoughts imported from a team space (tag team:shared) — sharing something back out isn't supported yet. An unshare from a different device than the one that originally shared the item, or before this device holds a current key for the space, is accepted but deferred rather than completed inline (the row is kept for a later retry) — the UI's confirmation is worded "Unshare requested" rather than claiming it's done.

Admin key-management duties (client-side)

A space admin's job isn't done once members have joined — two actions in the panel keep everyone able to read the space, and this is entirely a client responsibility; the server only stores and serves opaque sealed blobs, it never mints or distributes keys itself:

Seal pending (team_seal_pending) — for every epoch key this device already holds, seal a copy to every device that's joined the space but hasn't received it yet (pending-keys), and upload the batch. Run this after new members join and enroll a device — until an admin does, a new member can see envelope metadata but cannot decrypt any content, past or future.
Rotate (team_rotate_space) — mint a fresh epoch, generate a new space key, and seal + distribute it to every current member device (not just the ones missing a key — a rotation always redistributes the new key to everyone still in the space). Run this after removing a member or revoking a device: the old epoch's key remains recoverable by whoever already had it (space history is never rewritten), but nothing encrypted under the new epoch is readable by a device that didn't get the new key — which is exactly the point of rotating.

Both actions are on-demand only in Phase B2 (a button in the panel, no background scheduler) — an admin who forgets to run them after a membership change doesn't break anything, but new content silently stays unreadable by whoever missed the seal until the next manual pass.

Hive Team Server Deployment ​

VPS Prerequisites ​

Building the Web Console ​

Building the Binary ​

Native Build ​

Cross-Compilation ​

Install ​

Configuration: hive-team.toml ​

First Run ​

Pre-mailer Operation ​

CLI Usage ​

Verify ​

systemd Service ​

1. Create a dedicated user ​

2. Create the service unit ​

3. Enable and start ​

4. Verify ​

Reverse Proxy ​

Caddy ​

nginx ​

Unified setup: relay and Team server on one host ​

Smoke checklist (after any nginx or binary change) ​

Client addresses and rate limits ​

Updating ​

Verify after update ​

Security Checklist ​

Monitoring ​

Logs ​

Health Checks ​

Tenant Status ​

TLS certificate expiry ​

Backup status file format ​

Pi example: monitoring config keys ​

Metric retention and storage ​

Alerts and notifications ​

Rules ​

Lifecycle ​

Channels ​

The secrets key file (secrets_key_path) ​

Account approval and emails ​

Cortexa desktop alerts ​

Client onboarding (Cortexa app) ​

Admin key-management duties (client-side) ​

Hive Team Server Deployment

VPS Prerequisites

Building the Web Console

Building the Binary

Native Build

Cross-Compilation

Install

Configuration: `hive-team.toml`

First Run

Pre-mailer Operation

CLI Usage

Verify

systemd Service

1. Create a dedicated user

2. Create the service unit

3. Enable and start

4. Verify

Reverse Proxy

Caddy

nginx

Unified setup: relay and Team server on one host

Smoke checklist (after any nginx or binary change)

Client addresses and rate limits

Updating

Verify after update

Security Checklist

Monitoring

Logs

Health Checks

Tenant Status

TLS certificate expiry

Backup status file format

Pi example: monitoring config keys

Metric retention and storage

Alerts and notifications

Rules

Lifecycle

Channels

The secrets key file (`secrets_key_path`)

Account approval and emails

Cortexa desktop alerts

Client onboarding (Cortexa app)

Admin key-management duties (client-side)