Omnistrate Deployment

This document outlines the process for deploying Experio to Kubernetes using the Omnistrate platform.

What is Omnistrate

Omnistrate is a platform that simplifies deploying applications to Kubernetes. It uses your docker-compose.yaml file to build Docker images, push them to GitHub packages, and deploy them as a managed service on Kubernetes clusters. This eliminates the need to manually manage Kubernetes configurations while providing enterprise-grade deployment capabilities.

Installation

Install Omnistrate CLI

Install the Omnistrate CLI tool using the installation script:

curl -fsSL https://raw.githubusercontent.com/omnistrate/cli/master/install-ctl.sh | sh

Authentication

Once installed, login to your Omnistrate account:

omctl login

You’ll be prompted to enter your Omnistrate credentials.

How It Works

We do not use omctl build-from-repo. That command builds every image serially on one machine (45+ min) and ignores the compose image: tag — it only ever pushes :latest and :sha-<digest>, which makes it impossible to pin sidecars to a real version. Instead, CI builds the images itself and hands Omnistrate a compose that references already-pushed, version-pinned images:

Build images in parallel - GitHub Actions fans out a matrix job, one runner per Experio image, with per-image BuildKit layer caching (type=gha). Wall-clock ≈ the slowest single image (the server), not the sum of all of them.
Push tags to GHCR - Each image is pushed as both :${VERSION} (the semver from VERSION, a human-facing label) and :sha-${git-sha} (unique per commit).
Generate the Omnistrate compose - scripts/generate_omni_compose.py converts docker-compose.yaml → docker-compose.omni.yaml: it strips every build: block, replaces it with the pinned image: ref, and re-pins the server sidecars (scale-to-zero, flow-runner). It pins to the :sha-${git-sha} tag, not :${VERSION} — see Image Tagging for why the version tag is unsafe to pin.
Release the plan - omctl build -f docker-compose.omni.yaml registers a new service plan version that references those immutable image tags.
Deploy - the dev instance auto-upgrades; prod requires a manual omctl upgrade.

The local-dev docker-compose.yaml (with its build: blocks) is unchanged — docker compose up still builds locally. Only CI produces the derived docker-compose.omni.yaml (gitignored, never committed).

Image naming

Images are named after the Dockerfile suffix, not the compose service name (preserving the names build-from-repo historically used, so registry paths stay stable). For example service downloader builds Dockerfile.download → ghcr.io/experio-ai/experio-dockerfile.download.

Core Deployment Commands

Standard Deployment (with Docker build)

CI normally does this. To reproduce a full release locally you would build & push each image to GHCR tagged with the commit SHA (see .github/workflows/omnistrate-build.yml for the exact matrix), then generate the compose and release:

TAG="sha-$(git rev-parse HEAD)"
# ... docker build + push each Dockerfile.* to ghcr.io/experio-ai/experio-dockerfile.* :$TAG ...
python scripts/generate_omni_compose.py "$TAG"
omctl build -f docker-compose.omni.yaml --product-name Experio-1.1 --release-as-preferred

Fast Deployment (skip Docker build)

When the images for this commit already exist in GHCR (e.g. only spec/config changed), skip the build entirely and just re-release the plan:

TAG="sha-$(git rev-parse HEAD)"
python scripts/generate_omni_compose.py "$TAG"
omctl build -f docker-compose.omni.yaml --product-name Experio-1.1 --release-as-preferred

In CI this is the “skip docker build” path (dispatch input or a build(skip-docker) commit): the matrix build job is skipped and only the release job runs.

File Dependencies

The Omnistrate deployment depends on several key files:

Core Files

docker-compose.yaml - Main deployment specification defining all services, dependencies, and configurations
.omnistrate.env - Environment variables for the deployment (contains database credentials, API keys, etc.)

Dockerfiles

CI builds and pushes one image per Dockerfile below (named experio-dockerfile.<suffix>):

Dockerfile.server - Django backend server with integrated client assets
Dockerfile.migrations - DB migration / init job (compose service initdb)
Dockerfile.reader - File reading job service
Dockerfile.structured_data - Structured-data job service
Dockerfile.enrichment - Post-processing enrichment job service
Dockerfile.coordinator - Ingestion coordinator service
Dockerfile.download - File download job service (compose service downloader)
Dockerfile.parser - Document parser job service
Dockerfile.classifier - Document classifier job service
Dockerfile.ingestion_v2 - V2 ingestion job service
Dockerfile.cleanup - Cleanup job service

Supporting Files

certs/ - SSL certificates directory (currently used for ManTech deployment compatibility)

CI/CD Integration

Automated deployments are configured using GitHub Actions in .github/workflows/omnistrate-build.yml. The workflow has three jobs:

compute-meta - Decides whether to run (workflow_dispatch, a chore(release) commit on main, or a build(full) / build(skip-docker) keyword in the commit message), reads the semver from VERSION, and computes the release description from CHANGELOG.md.
build-and-push - A matrix job (one runner per image) that builds each Dockerfile.* with docker/build-push-action, caches layers via type=gha, and pushes :${VERSION} + :sha-${git-sha} to GHCR. Skipped on the “skip docker build” path.
deploy - Runs scripts/generate_omni_compose.py "sha-${git-sha}", then omctl build -f docker-compose.omni.yaml --product-name Experio-1.1 --release-as-preferred, and finally auto-upgrades the dev instance. Prod requires a manual omctl upgrade.

Triggers: pushes to staging, main (release commits), and feature/** / fix/** / bugfix/** branches, plus manual workflow_dispatch. GHCR auth uses secrets.GH_PAT (needs write:packages); Omnistrate auth uses secrets.OMNISTRATE_API_KEY.

Image Tagging

Floating :latest tags let instances silently drift when a node pulls a fresh image (and build-from-repo only ever produced :latest / :sha). We avoid that by building the images ourselves and pinning every Experio image — main containers and server sidecars — to the immutable per-commit tag :sha-${git-sha}.

Why pin to the commit SHA, not `VERSION`

VERSION is mutable across rebuilds: rebuilding without bumping it re-pushes the same :<version> tag in GHCR, overwriting the previous image. Kubernetes uses an IfNotPresent pull policy, so a node that already cached :<version> will not re-pull — meaning a brand new plan version can keep silently running the old image (observed as “new plan, unchanged behavior / unchanged timestamp”). The :sha-${git-sha} tag is unique to every commit, so each build is a distinct reference that always pulls. VERSION remains the human-facing release label (release description, app runtime version), just not the thing the plan pins to.

How it works

CI builds each Dockerfile.* and pushes ghcr.io/experio-ai/experio-dockerfile.<suffix> tagged with both :${VERSION} (label) and :sha-${git-sha} (the pinned, immutable tag).
scripts/generate_omni_compose.py "sha-${git-sha}" produces docker-compose.omni.yaml: it removes every build: block, sets image: to the pinned :sha-… ref, and re-pins the scale-to-zero / flow-runner sidecars from :latest to the same tag. It uses a real YAML parser, so it cannot reintroduce the duplicate-image:-key error that broke an earlier envsubst-based attempt (reverted PR #985).
omctl build records those immutable tags in a new plan version. Prod stays on its plan until an explicit upgrade.

Local / manual deploys

Build & push the images for the current commit to GHCR (see the workflow matrix), then pin to that commit’s SHA tag:

TAG="sha-$(git rev-parse HEAD)"
python scripts/generate_omni_compose.py "$TAG"
omctl build -f docker-compose.omni.yaml --product-name Experio-1.1 --release-as-preferred

Do not use :latest for Experio-built images. Prod promotion is omctl upgrade <instance-id> --version=<plan-version> only.

Helper Scripts (Potentially Deprecated)

omnistrate-deploy.sh

Located at scripts/omnistrate-deploy.sh, this script provides an interactive deployment process that:

Validates the docker-compose.yaml file
Offers options to skip Docker builds
Provides guided deployment with user prompts

Status: This script can probably be deleted if the simple omctl commands are sufficient for your workflow.

test-omni-deploy.sh

Located at scripts/test-omni-deploy.sh, this script:

Tests the deployment configuration locally before sending to Omnistrate
Provides options for fresh deployments (with volume deletion)
Offers core-only testing to speed up validation

Status: This script can probably be deleted if local testing is not needed before Omnistrate deployments.

Future Enhancements

Several improvements are planned to simplify the deployment process:

1. Remove .omnistrate.env Dependency

Current: Uses .omnistrate.env file for environment variables
Future: Move all environment variables to GitHub Secrets for better security and management
Benefit: Eliminates the need to manage sensitive credentials in repository files

2. Remove Certificate Dependencies

Current: Includes certs/ folder with SSL certificates for ManTech deployment compatibility
Future: Remove certificate dependencies that were added specifically for ManTech
Benefit: Simplifies deployment and removes ManTech-specific configurations

3. Clean Up Local Development Files

Current: Includes various local Docker Compose files for testing/debugging
Future: Remove docker-compose.local.yaml and docker-compose.local-web.yaml if they’re no longer needed
Benefit: Reduces confusion and maintenance overhead

Key Components

The Experio deployment consists of several containerized services:

Server: Django backend service with integrated client assets (Dockerfile.server)
Reader: Job service for reading and processing files (Dockerfile.reader)
Ingestion: Job service for downloading and processing files from external sources (Dockerfile.ingestion)
PostgreSQL: Database
Redis: Cache and session storage
RabbitMQ: Message broker for job queues
Neo4j: Graph database (default active provider; manual scale-to-zero)
FalkorDB: Alternate graph database (manual scale-to-zero; off by default in cluster)
JupyterLab: Data analysis and exploration environment

Docker Container Architecture

Container Structure

Server Container (Dockerfile.server)

The server container is built using a multi-stage approach:

Client Builder Stage: Builds frontend assets using Node.js
Python Server Stage: Creates the Django backend and copies built client assets

FROM node:22.14-alpine AS client-builder
# Build client assets
COPY ./client /app/client
RUN npm run heroku-postbuild

FROM python:3.12-bullseye
# Install Python dependencies and copy built assets
COPY --from=client-builder /app/client/dist /app/client/dist

Worker Containers (Dockerfile.reader & Dockerfile.ingestion)

Both worker services use similar lightweight Python containers:

FROM python:3.11-slim
# Install specific job dependencies
COPY jobs/base/requirements.txt /app/jobs/base/
COPY jobs/reader/requirements.txt /app/jobs/reader/  # or ingestion
RUN pip install -r /app/jobs/reader/requirements.txt

Volume Strategy

The deployment uses Docker volumes for:

Persistent Data Storage:
- experio_file_storage: Uploaded files and documents
- local_postgres_data: Database data
- local_redis_data: Cache data
- local_rabbitmq_data: Message queue data
- local_neo4j_data: Neo4j graph database data
- local_falkordb_data: FalkorDB graph database data
Certificate Management:
- ./certs:/certs:ro: SSL certificates (read-only)

Graph databases (Neo4j & FalkorDB)

Both graph backends are deployed as internal Omnistrate services with manual scale-to-zero (same pattern as JupyterLab). They are not required for platform boot — only for chat, ingestion graph writes, and admin graph tools.

Service	Default scaling mode	Default replicas	Purpose
`falkordb`	`keep_alive`	1	Active graph provider (`GRAPH_PROVIDER=falkordb` on a new instance)
`neo4j`	`keep_down`	0	Standby / migration target

Control via Django admin ServiceConfiguration (scale-to-zero sidecar), or Omnistrate capacity API. After changing modes, allow a few minutes for the sidecar to reconcile replica counts. See Graph backend (Neo4j & FalkorDB) for provider switching and migration runbooks.

Service Startup Order

The deployment enforces startup dependencies for platform services only. Graph databases are intentionally not in the critical path:

Core infrastructure (PostgreSQL, Redis, RabbitMQ) start with health checks
Database initialization (initdb) runs after PostgreSQL is healthy — needs Postgres only (seeds NEO4J_* / FALKOR_* config from secrets; does not connect to graph DBs)
Application server starts after initdb completes and core infra is healthy
Worker services (reader, ingestion pipeline, etc.) start after RabbitMQ and initdb; they tolerate a cold graph and reconnect when the active provider is up
Graph DBs (neo4j, falkordb) scale independently via manual scale-to-zero

Chat and graph ingestion features require the active provider (GRAPH_PROVIDER) to be running. The rest of the platform (auth, admin UI, Postgres-backed settings) starts without either graph DB online.

Authentication Setup

The deployment includes automatic creation of default accounts during the initdb job:

Default Accounts

Admin User
- Email: admin@experiolabs.ai
- Password: Value from DJANGO_SUPERUSER_PASSWORD environment variable
- Has full superuser privileges
Test User
- Email: playwright@thinknimble.com
- Password: Value from PLAYWRIGHT_TEST_USER_PASS environment variable
- Used by automated tests

Omnistrate secrets

Graph credentials are injected via Omnistrate service secrets referenced in docker-compose.yaml. Add these in the Omnistrate dashboard (Secrets) before releasing a new service API version:

Secret key	Used by	Purpose
`neo4jPassword`	`neo4j`, `initdb`	Neo4j auth and seeded `NEO4J_PASSWORD`
`falkordbPassword`	`falkordb`, `initdb`	FalkorDB auth and seeded `FALKOR_PASSWORD`

If falkordbPassword is missing, Omnistrate release validation fails with an error referencing the initdb component. Other existing secrets (dbPassword, rabbitmqPassword, etc.) are unchanged.

Environment Configuration

The deployment uses environment variables defined in .omnistrate.env. Key configurations include:

# Database configuration
DB_NAME='experio_db'
DB_USER='experio'
DB_HOST='postgres'

# Neo4j Graph Database (when GRAPH_PROVIDER=neo4j)
NEO4J_URI='neo4j://neo4j:7687'
NEO4J_USER='neo4j'
NEO4J_DATABASE='neo4j'

# FalkorDB (when GRAPH_PROVIDER=falkordb) — cluster internal DNS
# FALKOR_URI='redis://falkordb:6379'

# AI/ML Services
USE_AZURE_OPENAI='True'
USE_GOOGLE_GENAI='False'

# Authentication
VITE_USE_AUTH0='False'

Deployment Checklist

Before deploying to Omnistrate:

Install and authenticate: Ensure omctl is installed and you’re logged in
Add secrets: Ensure falkordbPassword (and other required secrets) exist in Omnistrate
Update configurations: Modify docker-compose.yaml and .omnistrate.env as needed
Choose deployment type: Decide whether to rebuild Docker images or skip the build
Run deployment: Execute the appropriate omctl build-from-repo command
Monitor deployment: Check the Omnistrate UI for deployment status and any issues
Post-deploy (existing instances): Run once after upgrade:
python manage.py migrate python manage.py seed_autoscaling_config python manage.py seed_config
seed_config without --force leaves an existing GRAPH_PROVIDER default in place, including a row whose live value is still the old implicit neo4j. seed_autoscaling_config uses get_or_create, so an instance that already has neo4j / falkordb rows keeps those scaling modes. A brand-new database seeds FalkorDB as the provider and keeps that service at 1 replica. Pass seed_config --force only when you intend a null GRAPH_PROVIDER value to adopt the new falkordb default.

Health endpoints

Endpoint	Purpose
`GET /health/`	Comprehensive check. Critical: Postgres, Redis, RabbitMQ, LLM. Non-critical: active graph DB (`checks.neo4j`), MCP. Returns `degraded` (HTTP 200) when platform is up but graph is down (scale-to-zero, warm-up, migration). Returns `unhealthy` (HTTP 503) only when a critical dependency fails.
`GET /health/simple/`	Process liveness only — always 200 when Django is running

Do not treat checks.neo4j.status: unhealthy alone as a platform outage when graph is intentionally scaled down.

Container health checks on Omnistrate

Omnistrate ignores the Compose healthcheck: key. The compose spec is explicit: “These tags are ignored. Healthchecks are configured via x-omnistrate-action-hooks.” A healthcheck: block therefore only affects local docker compose up, where it is purely informational — Compose reports a container unhealthy but never restarts it. To get a real probe in production, define an action hook:

services:
  coordinator:
    x-omnistrate-actionhooks:
      - scope: NODE
        type: HEALTH_CHECK
        commandTemplate: |
          /app/scripts/worker_quorum_check.sh coordinator_worker COORDINATOR_WORKERS

Hook type	Kubernetes probe	Effect on failure
`HEALTH_CHECK`	livenessProbe	Container is restarted
`READINESS_CHECK`	readinessProbe	Removed from service endpoints; defaults to `HEALTH_CHECK`
`STARTUP_CHECK`	startupProbe	Gates the other probes; defaults to `HEALTH_CHECK`

All three are scope: NODE. With no hook defined, the default is a TCP ping on the service port — so a service that exposes no port gets no liveness probe at all. That is the case for every job worker, which is why they need an explicit hook.

How the hook reaches the container

A HEALTH_CHECK hook is not an exec probe on the app container. Observed on the dev instance instance-i03ms58l6, the delivery path is:

Omnistrate writes the command into a per-version ConfigMap — e.g. coordinator-health-check-command-874-0-0 — containing exactly the commandTemplate body: /app/scripts/worker_quorum_check.sh coordinator_worker COORDINATOR_WORKERS.
That ConfigMap is mounted into the pod’s monitor-sidecar container at /etc/monitoring/scripts (MONITORING_SCRIPTS_MOUNT_PATH=/etc/monitoring/scripts).
The app container’s probes are httpGet requests to the sidecar — e.g. /healthz/process/liveness on port 49750. kubelet probes the sidecar; the command runs there, not as an exec into the app container.
The pod sets shareProcessNamespace: true. That shares PIDs between containers, not mount namespaces — the sidecar does not get the app container’s filesystem view.

The command references /app/scripts/worker_quorum_check.sh, which lives in the app image. How the sidecar resolves that path across the mount-namespace boundary is not established here; if a probe ever reports a missing file, that is the first thing to check. Practical consequence: when a probe misbehaves, look at the ConfigMap and the monitor-sidecar, not at kubectl describe expecting an exec handler on the app container — there isn’t one.

Rendered probe parameters

The probe timings are set by Omnistrate, not by the hook. As rendered on the app container:

Probe	Handler	`initialDelaySeconds`	`periodSeconds`	`failureThreshold`	Time before it acts
`livenessProbe`	`httpGet` (sidecar)	120	20	9	~180 s of failures, after the 120 s delay, before restart
`readinessProbe`	`httpGet` (sidecar)	5	5	18	~90 s before the pod is marked not-ready
`startupProbe`	`httpGet` (sidecar)	5	5	4320	~6 h of startup grace

The startupProbe gates the other two, so a container gets roughly six hours to come up before the liveness probe can restart it. scripts/worker_quorum_check.sh also counts supervisor’s STARTING and BACKOFF states towards quorum, but that tolerance is defensive — the six-hour startup grace, not the state tolerance, is what keeps a slow boot from looping. The flip side of that grace: a service that never reaches quorum sits not-ready for hours rather than restarting quickly.

Command ConfigMaps are empty until a pod is scheduled

The <service>-health-check-command-<version> ConfigMaps are created empty when a plan version is deployed and are only populated once a pod is actually scheduled. Grepping for the command before a pod exists finds nothing — that is not evidence the hook is missing. The worker services are minReplicas: 0 (scale-to-zero; an active ScanOrder warm-keeps them), so this is the normal state for an idle instance: wake the service, then re-read the ConfigMap. The pre-change version (873) also has an empty command, but there the cause is upstream: those services carried no x-omnistrate-actionhooks entry at all, so they had no health check before this change. Confirm that from the compose at the pre-change commit rather than from the ConfigMap, which is ambiguous for the reason above.

Worker quorum checks

The supervisord-managed services (coordinator, downloader, parser, classifier, ingestion_v2) run N workers each. Their HEALTH_CHECK hook fails when fewer than half the configured workers are alive, which restarts the container. Validated end to end on dev: stopping the workers produced Liveness probe failed: 503, then Container service failed liveness probe, will be restarted, restartCount 0 -> 1 on the same pod, then self-recovery. The hook does not run in your container. Omnistrate writes the commandTemplate into a per-version ConfigMap, mounts it into the monitor-sidecar, and executes it there. The app container’s probes are httpGet against that sidecar. Measured on dev, the sidecar:

Can it…
read the app filesystem (`/app`)	❌ separate mount namespace
`nsenter` into the app container	❌ `runAsNonRoot`, uid 1000, no `CAP_SYS_ADMIN`
see the app container’s processes	✅ `shareProcessNamespace: true`
read the service’s env vars	✅

A hook that references an app-container path fails like this:

probe health-check-command: exit status 127
stderr: /app/scripts/worker_quorum_check.sh: No such file or directory
-> 503 -> pod never Ready -> that resource's "Deployment" step fails and the
   upgrade stalls (and the stalled workflow then blocks every later upgrade)

So the hook must be self-contained and use the one thing both contexts share — the PID namespace.

Writing a hook command: use no `$` at all

omctl build mangles every $ form. Observed in the deployed ConfigMap:

written	what actually lands
`$VAR`	interpolated away
`$$VAR`	left literally as `$$VAR` — it does not un-escape
`${VAR:-50}`	collapses to `$50`, destroying the default
`$(( ))`	`invalid action hook: (( can only be used to open an arithmetic cmd`

Write hooks with zero $: printenv NAME reads a variable, backticks substitute, and expr does arithmetic. The quorum test becomes:

expr `grep -al 'jobs.coordinator.runne[r]' /proc/[0-9]*/cmdline | wc -l` \* 2 \>= `printenv COORDINATOR_WORKERS`

alive * 2 >= expected is exactly alive >= ceil(expected / 2), so the 50% rule needs no division. The pattern’s final character is bracketed so grep cannot match its own /proc entry — its argv contains the literal [r], which the regex does not match. Without that a fully dead service reports one live worker. scripts/test_compose_hooks.py (in CI) enforces all of this: no $, no ((, no app-container path, a bracketed pattern, and that each inlined hook returns the same exit codes as scripts/worker_quorum_check.sh on identical /proc fixtures. The duplication is deliberate — the hook cannot call the script — and that equivalence test is what stops the two drifting.

Recovering a stalled upgrade

A plan version whose hook fails leaves its upgrade workflow RUNNING indefinitely, and that blocks every later upgrade, so the instance keeps recreating the old pod. Cancel it before retrying:

omnistrate-ctl workflow terminate <workflow-id> -s <service-id> -e <env-id> -y

The -y is required; without it the command tries to open a TTY and fails.

Verifying a deploy actually contains your hook

A pod can be Ready for opposite reasons: the check passing, or no check existing. A main release will produce a plan version with zero action hooks, and --version=latest may pick it up ahead of your branch’s build. Confirm both:

kubectl get pod coordinator-0 -o jsonpath='{.spec.containers[0].image}'   # your SHA?
kubectl get cm coordinator-health-check-command-<ver>-0-0 \
  -o jsonpath='{.binaryData.health-check-command}' | base64 -d           # non-empty?

An empty ConfigMap means the version has no health check at all.

Troubleshooting

Common Deployment Issues

Authentication Failed
- Ensure you’re logged in: omctl login
- Check your Omnistrate account credentials
Invalid Docker Compose Configuration
- Validate locally: docker compose config
- Check for YAML syntax errors
Build Failures
- Review build logs in the Omnistrate UI
- Test Docker builds locally: docker build -f Dockerfile.server .
- Missing secret references: e.g. falkordbPassword not defined in Omnistrate Secrets
Service Health Check Failures
- Check the x-omnistrate-actionhooks HEALTH_CHECK hook, not the healthcheck: block — Omnistrate ignores the latter (see Container health checks on Omnistrate)
- Read the command back from the per-version <service>-health-check-command-<version> ConfigMap, and remember it is empty until a pod is scheduled
- The probe itself is an httpGet to the monitor-sidecar, so failures show up in the sidecar’s logs, not as an exec on the app container
- Verify service dependencies are correctly defined

Useful Commands

# Validate docker-compose configuration
docker compose config

# Test local deployment
./scripts/test-omni-deploy.sh  # If script is still available

# View Omnistrate deployment status
omctl get deployments

# Check deployment logs (in Omnistrate UI)
# Navigate to your service in the Omnistrate dashboard

Container Startup Failures

If containers fail to start:

Check logs in Omnistrate UI: Navigate to your deployment and examine container logs
Verify health checks: Ensure the HEALTH_CHECK action hook command is appropriate for the service. The startupProbe grants ~6 h of grace, so a hook that fails during boot will not restart-loop the pod — it will leave it not-ready instead
Check dependencies: Verify depends_on configurations match your service requirements
Volume permissions: Ensure mounted volumes have appropriate permissions

Image Build Problems

If Docker image builds fail:

Review build logs: Check the build process output in Omnistrate UI
Test locally: Run docker build commands locally to identify issues
Check base images: Ensure base images in Dockerfiles are accessible
Verify file paths: Ensure all COPY commands reference existing files

Getting Help

Omnistrate Documentation: Check the official Omnistrate documentation
GitHub Issues: Review the repository’s issue tracker for known problems
Omnistrate Support: Contact Omnistrate support for platform-specific issues

​Omnistrate Deployment

​What is Omnistrate

​Installation

​Install Omnistrate CLI

​Authentication

​How It Works

​Image naming

​Core Deployment Commands

​Standard Deployment (with Docker build)

​Fast Deployment (skip Docker build)

​File Dependencies

​Core Files

​Dockerfiles

​Supporting Files

​CI/CD Integration

​Image Tagging

​Why pin to the commit SHA, not VERSION

​How it works

​Local / manual deploys

​Helper Scripts (Potentially Deprecated)

​omnistrate-deploy.sh

​test-omni-deploy.sh

​Future Enhancements

​1. Remove .omnistrate.env Dependency

​2. Remove Certificate Dependencies

​3. Clean Up Local Development Files

​Key Components

​Docker Container Architecture

​Container Structure

​Server Container (Dockerfile.server)

​Worker Containers (Dockerfile.reader & Dockerfile.ingestion)

​Volume Strategy

​Graph databases (Neo4j & FalkorDB)

​Service Startup Order

​Authentication Setup

​Default Accounts

​Omnistrate secrets

​Environment Configuration

​Deployment Checklist

​Health endpoints

​Container health checks on Omnistrate

​How the hook reaches the container

​Rendered probe parameters

​Command ConfigMaps are empty until a pod is scheduled

​Worker quorum checks

​Writing a hook command: use no $ at all

​Recovering a stalled upgrade

​Verifying a deploy actually contains your hook

​Troubleshooting

​Common Deployment Issues

​Useful Commands

​Docker-Related Issues

​Container Startup Failures

​Image Build Problems

​Getting Help