Prometheus Integration

OLake Go UI exposes a Prometheus compatible /metrics endpoint so you can monitor the health and performance of your sync jobs using standard tooling like Grafana, without parsing logs manually.

This is a pull based endpoint, point your existing Prometheus scrape config at it and metrics are available immediately. No agents, sidecars, or additional configuration are required on OLake Go's side.

Version Requirements

To use the /metrics endpoint, OLake UI version should be 0.5.0 or higher.

info

Metrics are scoped at the job level, reflecting each job's most recent sync run.

Enabling the Endpoint

The /metrics endpoint is served by the OLake Go UI backend on the same port as the UI itself at:

GET http://<your-olake-host>:8000/metrics

It's controlled by an environment variable on the OLake Go UI service:

Variable	Default	Description
METRICS_ENABLED	false	Set to true to enable the endpoint. When disabled, /metrics returns a 404.

info

• Docker Compose:
  To enable the /metrics endpoint, add METRICS_ENABLED=true under x-envs in your Docker Compose file. See Prometheus Metrics Configuration for setup details.
• Helm:
  To enable the /metrics endpoint, add METRICS_ENABLED: true under olakeUI in your values.yaml file. See Prometheus Metrics Configuration for setup details.
• Once enabled, metrics will be available from the next sync run. Historical metrics from previous runs are not available.

The endpoint is unauthenticated and sits outside the /api route group, so your Prometheus scraper doesn't need OLake Go UI credentials to reach it.

Metrics Specification

1. Sync Metrics

These metrics show what's happening with your latest sync for each job. Think of them as a dashboard that updates with fresh numbers every time a new sync starts or completes. When a new sync begins, it replaces the previous sync's numbers.

note

The endpoint exposes only the olake_* series listed below (plus the scrape-error counter). The default go_* and process_* runtime metrics are intentionally not included.

Metric	Description
olake_sync_status	Status of the latest sync run: 0 = running (a paused run also reports 0) 1 = succeeded 2 = failed (a cancelled sync is also reported as failed).
olake_sync_start_time_seconds	Unix timestamp of when the latest sync run started.
olake_sync_duration_seconds	Duration of the run in seconds. Updates live while the sync is running, and freezes once it completes.
olake_sync_records_ingested	Total records ingested during the run.
olake_sync_bytes_read	Total bytes read from the source during the run.

Every sync metric carries the following labels:

Label	Description
job_id	Unique identifier of the job.
job_name	Job name configured by the user.
source_name	Source name configured by the user.
destination_name	Destination name configured by the user.

2. System Metrics

Metric	Description
olake_process_cpu_usage_ratio	CPU utilization during the sync, as a ratio between 0 and 1.
olake_process_memory_usage_bytes	System memory in use during the sync, in bytes.

Available from v0.9.0

The following metrics require OLake Go v0.9.0 or later:

• olake_sync_bytes_read
• olake_process_cpu_usage_ratio
• olake_process_memory_usage_bytes

Data Collection

Each scrape derives its response from three sources, refreshed independently:

1. Job metadata: job, source, and destination names configured in OLake Go's Postgres database.
2. Run status, start time, and duration: read from Temporal's workflow visibility, refreshed at most every 10 seconds.
3. Records, bytes, CPU, and memory: read from the sync process's stats.json on the shared volume, refreshed at most every 2 seconds.

info

If a stats file scapre fails, the endpoint serves the previous values instead of erroring, and increments olake_metrics_scrape_errors_total on each failure. Stats refresh every 2 seconds, so this counter shows how stale the served metrics are for example, a count of 3 means the values are roughly 6 seconds old.

Stats Freshness

Changes reach Prometheus with a maximum delay of snapshot cadence + scrape interval (plus approximately 1 second Temporal lag):

• Snapshot cadence: OLake Go's internal refresh schedule that captures the current state. Status and duration metrics refresh every 10 seconds. Throughput metrics (records, bytes, CPU, memory) refresh every 2 seconds.
• Scrape interval: The frequency at which Prometheus scrapes the /metrics endpoint, configured in the Prometheus scrape config.

Visibility timeline:

• New job: Snapshot cadence is 10 seconds. Jobs that haven't run yet won't appear.
• Completion or cancellation: Snapshot cadence is 10 seconds.
• Granularity: Snapshot cadence is 2 seconds for throughput metrics (records, bytes, CPU, memory) and 10 seconds for status and duration metrics.

Limitations

• Metrics show only the latest sync run for each job. When a new sync starts, it replaces the numbers from the previous run.
• Metrics are available at the job level only. You won't see per-table breakdowns.
• This endpoint exposes metrics only. Logs and traces are not included.
• Prometheus and its compatible visualization tools require separate setup. OLake Go does not bundle or manage these tools.
• /metrics covers sync jobs only. It does not expose container, host, or Go runtime metrics. For infrastructure monitoring, use cAdvisor (built into the kubelet on Kubernetes).