Skip to main content

Overview

Navigate to Admin > Monitoring > Scaling to monitor and manually scale the platform’s document processing services. The dashboard shows each service’s health, current and maximum replicas, and how they fit into the ingestion pipeline. Reading the dashboard requires Monitoring read access. Changing settings or force-scaling a service requires Monitoring write access.

Dashboard

The Dashboard tab groups services into three sections:

Ingestion Pipeline

The core document processing chain — Reader → Downloader → Parser → Classifier → Ingestion — rendered as a left-to-right flow. An arrow between two stages turns green when both stages are healthy; a queued-message count appears under the arrow when the downstream stage has backlog waiting to be picked up. Pipeline satellites (services that support the pipeline but aren’t sequential stages — coordinator, MinIO, the Kreuzberg text extractor, enrichment, structured data, cleanup) appear as a row of cards underneath the flow.

Core Infrastructure (Always On)

Services that cannot be scaled manually: always-on infrastructure with no scaling configuration (server, PostgreSQL, Redis, RabbitMQ) and platform-safety-locked services (Neo4j, FalkorDB). These render with a lock icon and no selection checkbox.

Other Services

Any remaining scalable service that isn’t part of the pipeline or its satellites.

Health indicators

Each service card shows a colored dot summarizing its status:

Service Detail Page

Click any scalable service’s card to open its detail page, which shows:
  • Current State — status, current replicas, scaling mode, and configured min/max bounds.
  • Settings tab — edit the service’s autoscaling configuration (requires write access).
  • Events tab — the scaling event history for just this service.

Settings

Basic settings: Advanced settings (collapsed by default, with a warning that misconfiguration can break the pipeline dependency chain):
  • Capacity step size and messages per replica — control how aggressively the queue-based strategy scales.
  • Queue names — which RabbitMQ queues this service’s strategy watches.
  • Schedule times / window duration — for schedule-based strategies.
  • Warm-keeping enabled / warm-keeping policies — keep a service warm during specific in-progress operations (scan orders, structured data scan orders, enrichment jobs, dependency chains) instead of scaling it to zero.
  • Dependency order — this service’s position in the pipeline dependency chain.
The Strategy shown at the top of the panel (queue, reader, schedule, manual, phoenix, propelauth) is display-only and cannot be changed from the UI.

Force Scaling

Manually scaling a service switches its mode to Keep Alive (or Keep Down when scaled to zero) so the autoscaler doesn’t immediately revert the change on its next cycle. Reset the scaling mode from the service’s Settings tab to return it to automatic scaling.

Single Service

From a service’s detail page, click Force Scale (hidden for core services) to open a dialog where you set a target replica count within the service’s configured bounds. Scaling a service to zero opens a confirmation dialog instead: you must type the service name to confirm, and scaling a stateful service (such as MinIO) to zero additionally requires acknowledging that its persistent volume data will be permanently deleted.

Multiple Services (Batch)

Services can be scaled together in a single operation:
  1. Select services using the checkboxes on their cards (core services are not selectable).
  2. Click Force scale selected in the actions bar that appears.
  3. Set a target replica count per service — the dialog previews whether each service will scale up, down, or stay unchanged. A reason is required for the audit trail.
  4. Confirm to apply all changes as one batch.
Results are reported per service; services that fail to scale stay selected so you can retry them. Each batch is assigned a correlation ID shown with the results, which links the individual scaling events in the All Events tab for auditing. Batch scaling requests are rate-limited to a few per minute — scaling actions take about a minute to materialize (pods must start), so rapid resubmission has no effect.

Deferred Scaling Conflicts

The platform serializes scaling operations per instance: only one resource can converge at a time. If you force-scale a service while another resource’s scaling operation is still in progress, the request is deferred rather than treated as a failure — you’ll see “A previous scaling operation is still in progress. Please try again in a few seconds.” in muted text rather than a red error, and no failed event is recorded. Wait a few seconds and retry.

Scaling Events

The All Events tab (global) and each service’s Events tab (per-service) show the scaling audit trail: timestamp, action (scale up/down, manual scale up/down/to-zero, or failed), replica change, who triggered it (or “Autoscaler” for automatic actions), and the reason given.
Use the correlation ID from a batch force-scale result to find every event that batch produced across services.