Hosted Preview Runbook

The protected hosted preview is the first controlled tubebrain-hosted deployment for design-partner testing. It keeps tubebrain.ai as the static marketing surface and exposes the backend only through the Tinyland tailnet.

Deployment Authority

The normal deployment authority is the Blahaj OpenTofu stack at tinyland-inc/blahaj/tofu/stacks/tubebrain-preview. That stack owns the live Kubernetes state for the protected preview: namespace, tubebrain-hosted, tubebrain-stt, ClusterIP Services, Tailscale HTTPS Ingress, Kubernetes Secrets, NetworkPolicies, GHCR pull Secret, and the Whisper model-cache PVC.

This tubebrain repo owns the binaries, images, API contract, public docs, and smoke scripts. The checked-in deploy/hosted-preview Kubernetes manifests are legacy/manual recovery references. Do not use them as the default state owner after the Blahaj OpenTofu adoption unless an operator has explicitly chosen a break-glass recovery path.

Current target:

  • Kubernetes context: honey
  • Namespace: tubebrain-preview
  • Workload: deployment/tubebrain-hosted
  • Internal STT workload target: deployment/tubebrain-stt
  • Tailnet URL: https://tubebrain-preview.taila4c78d.ts.net
  • Image: ghcr.io/jesssullivan/tubebrain-hosted:<git-sha> or :main
  • Internal STT image: ghcr.io/jesssullivan/tubebrain-stt:<git-sha> or :main
  • State owner: tinyland-inc/blahaj/tofu/stacks/tubebrain-preview
  • Reference/fallback manifests: deploy/hosted-preview

The application still enforces bearer-token auth on every endpoint except GET /v1/health.

The hosted API remains the only preview surface exposed to callers. The tubebrain-stt workload is an internal Whisper service for live/radio stream chunks and should stay reachable only by cluster-local traffic from tubebrain-hosted.

Pilot metering, retention, and redaction boundaries are documented in Hosted Pilot Policy.

For a pre-call paid design-partner checklist that ties deployment, credentials, quota, STT, redaction, demo artifacts, and customer-facing acceptance criteria together, use the Paid Pilot Operator Runbook.

Build And Publish

The Hosted Image GitHub Actions workflow builds the hosted API and internal STT Nix OCI archives and pushes both to GHCR. On main, it publishes immutable git-SHA tags and the moving main preview tags.

Manual local build:

just hosted-image-build
just stt-image-build

Manual push from a Linux-capable Nix shell, if needed:

nix develop --command skopeo login ghcr.io
nix build .#packages.x86_64-linux.hostedImage --out-link hosted-image
nix develop --command skopeo copy docker-archive:$(realpath hosted-image) docker://ghcr.io/jesssullivan/tubebrain-hosted:<tag>
nix build .#packages.x86_64-linux.sttImage --out-link stt-image
nix develop --command skopeo copy docker-archive:$(realpath stt-image) docker://ghcr.io/jesssullivan/tubebrain-stt:<tag>

Secret Bootstrap And Rotation

No API keys or GHCR credentials are committed. Normal secret bootstrap and rotation happens through the Blahaj OpenTofu stack and its encrypted tubebrain-preview secret source. Keep secret material in SOPS or operator environment variables; never write raw tokens into docs, shell history, logs, or pilot evidence.

The commands below are retained only for manual recovery of the legacy deploy/hosted-preview path:

kubectl apply -f deploy/hosted-preview/namespace.yaml

secret_key_name=TUBEBRAIN_API_KEY
kubectl -n tubebrain-preview create secret generic tubebrain-hosted-api \
  --from-literal="${secret_key_name}=${!secret_key_name}" \
  --dry-run=client -o yaml | kubectl apply -f -

stt_key_name=TUBEBRAIN_STT_API_KEY
kubectl -n tubebrain-preview create secret generic tubebrain-stt-api \
  --from-literal="${stt_key_name}=${!stt_key_name}" \
  --dry-run=client -o yaml | kubectl apply -f -

If the GHCR image is private, also create ghcr-jesssullivan-pull from an operator token with package read access.

Deploy And Roll Forward

Normal roll-forward uses Blahaj OpenTofu. Update the hosted and STT image tags together in tinyland-inc/blahaj/tofu/config/tubebrain-preview-honey.tfvars.json or via reviewed -var inputs, then plan and apply from the Blahaj operator path:

cd ../blahaj
just tofu-plan tubebrain-preview honey
just tofu-apply tubebrain-preview honey --auto-approve

The Blahaj stack manages:

  • a dedicated tubebrain-preview namespace
  • the tubebrain-hosted Deployment
  • the internal tubebrain-stt Deployment
  • a ClusterIP Service on port 8787
  • a ClusterIP Service on port 8788
  • a Tailscale Ingress for tubebrain-preview.taila4c78d.ts.net
  • bearer-token Kubernetes Secrets for preview and internal STT auth
  • a GHCR pull Secret when configured
  • a NetworkPolicy that only accepts hosted ingress from the Tailscale namespace
  • TUBEBRAIN_STT_BACKEND=remote on tubebrain-hosted
  • TUBEBRAIN_REMOTE_STT_URL=http://tubebrain-stt:8788
  • TUBEBRAIN_REMOTE_STT_TOKEN sourced from the same secret material as TUBEBRAIN_STT_API_KEY
  • a NetworkPolicy that permits STT ingress only from tubebrain-hosted
  • the tubebrain-whisper-model-cache PVC for re-downloadable model cache state

Manual Recovery Manifests

Use this path only when deliberately bypassing Blahaj OpenTofu for recovery or local operator rehearsal:

kubectl apply -f deploy/hosted-preview/namespace.yaml
just hosted-preview-apply \
  ghcr.io/jesssullivan/tubebrain-hosted:<tag> \
  ghcr.io/jesssullivan/tubebrain-stt:<tag>

The manual manifests are intentionally narrower than the Blahaj stack. In particular, the Blahaj stack owns the PVC-backed Whisper model cache and OpenTofu labels used for state authority evidence.

Rollback:

kubectl -n tubebrain-preview rollout undo deployment/tubebrain-hosted
kubectl -n tubebrain-preview rollout undo deployment/tubebrain-stt
kubectl -n tubebrain-preview rollout status deployment/tubebrain-hosted
kubectl -n tubebrain-preview rollout status deployment/tubebrain-stt

Smoke Test

With the preview bearer token in the environment:

export TUBEBRAIN_API_KEY
just hosted-preview-smoke

The smoke test proves:

  • GET /v1/health returns 200
  • unauthenticated transcript calls return 401 without source or token leaks
  • authenticated POST /v1/transcript/section returns the agent section contract
  • authenticated stream start, bounded poll, list, and stop are reachable

The smoke path uses the public GStack demo URL and a public SomaFM radio stream by default, uses per-request curl timeouts, and best-effort stops an opened stream session on failure. Override TUBEBRAIN_DEMO_URL or TUBEBRAIN_PROOF_RADIO_URL when a fixture changes.

Environment

Required:

  • TUBEBRAIN_API_KEY: preview bearer token
  • TUBEBRAIN_STT_API_KEY: internal STT service bearer token when remote STT is deployed

Optional:

  • TUBEBRAIN_HOSTED_BIND: defaults to 0.0.0.0:8787 in the image
  • TUBEBRAIN_STT_BIND: defaults to 0.0.0.0:8788 in the STT image
  • TUBEBRAIN_STT_BACKEND: set to remote on hosted API pods when using the internal STT service
  • TUBEBRAIN_REMOTE_STT_URL: internal STT base URL
  • TUBEBRAIN_REMOTE_STT_TOKEN: hosted API token for calling the internal STT service
  • TUBEBRAIN_REMOTE_STT_TIMEOUT_MS: optional hosted-to-STT timeout
  • TUBEBRAIN_REMOTE_STT_MAX_BYTES: optional hosted-to-STT chunk byte cap
  • TUBEBRAIN_STT_MAX_BYTES: optional internal STT chunk byte cap
  • RUST_LOG: defaults to info,tubebrain=debug in the Deployment

The preview does not persist raw audio. Responses and logs must not expose cookies, signed media URLs, PoTokens, BotGuard internals, or raw audio.