OLake UI

OLake UI provides a complete Docker Compose stack for running the replication sync using different sources with orchestration.

Components

• OLake UI: Main web interface for job management and configuration
• Temporal Worker: Background worker for processing data replication jobs
• PostgreSQL: Primary database for storing job data, configurations, and sync state
• Temporal Server: Workflow orchestration engine for managing job execution
• Temporal UI: Web interface for monitoring workflows and debugging
• Elasticsearch: Search and indexing backend for Temporal workflow data
• Signup Init: One-time initialization service that creates the default admin user

Prerequisites

The following requirements must be met before starting:

• Docker installed (Docker Desktop recommended)
• Docker Compose (included with Docker Desktop)
• At least 4GB RAM available for Docker
• Port 8000 available on the system

Quick Start

One-Command Setup

The fastest way to get OLake UI running is with a single command:

curl -sSL https://raw.githubusercontent.com/datazip-inc/olake-ui/master/docker-compose.yml | docker compose -f - up -d

This command will:

• Download the latest docker-compose.yml file
• Pull all required Docker images
• Start all services in the background
• Create a default admin user automatically

Access the Application

• OLake UI: http://localhost:8000

Login

The default credentials are:

• Username: admin
• Password: password

Using OLake UI

Once the stack is running, the following operations can be performed:

1. Create Jobs: Data replication jobs can be set up between sources and destinations
2. Configure Sources: Connectors for databases like MongoDB, PostgreSQL, MySQL or Oracle DB can be established
3. Configure Destinations: Apache Iceberg or AWS S3 as targets can be configured
4. Monitor Jobs: Job execution can be tracked and logs can be viewed

For detailed job creation instructions, see Create Jobs.

Service Configuration

Changing Admin Credentials

The default admin user can be customized by editing the docker-compose.yml file before starting:

x-signup-defaults:
  username: &defaultUsername "your-username"
  password: &defaultPassword "your-secure-password"
  email: &defaultEmail "your-email@example.com"

Updating OLake UI Version

To update OLake UI to the latest version, use the following command:

curl -sSL https://raw.githubusercontent.com/datazip-inc/olake-ui/master/docker-compose.yml | docker compose -f - down && \
curl -sSL https://raw.githubusercontent.com/datazip-inc/olake-ui/master/docker-compose.yml | docker compose -f - up -d

Note: Your data and configurations will be preserved as they are stored in persistent volumes and the olake-data directory.

Encryption Key Configuration

OLake supports encryption of source and destination configurations stored in the database. Configure the encryption key in docker-compose.yml:

• Custom String: Provide any string (OLake generates SHA-256 hash):

  x-encryption:
    key: &encryptionKey "your-passphrase"

• AWS KMS: Use a AWS KMS key ARN (recommended for production):

  x-encryption:
    key: &encryptionKey "arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012"

• Disable Encryption: Use empty string:

  x-encryption:
    key: &encryptionKey ""  # No Encryption

Customizing Data Directory

The default data directory can be changed by modifying the host persistence path:

x-app-defaults:
  host_persistence_path: &hostPersistencePath /custom/path/to/olake-data
  worker_config_volume_details: &workerConfigVolumeDetails
    type: bind
    source: *hostPersistencePath
    target: /tmp/olake-config

This will create and use /custom/path/to/olake-data instead of the default ./olake-data directory.

Service Environment Variables

Key environment variables that can be customized within specific services:

services:
  olake-ui:
    environment:
      PERSISTENT_DIR: *hostPersistencePath
      POSTGRES_DB: "postgres://olake:olake@postgresql:5432/olakedb"
      OLAKE_SECRET_KEY: *encryptionKey

Data Persistence

The Docker Compose setup creates the following data storage:

OLake Config Directory

• olake-data: Local directory created in the current working directory
  • Contains streams configurations, connection settings, and sync state
  • Persists across container restarts and recreations
  • Used by OLake UI and Temporal Worker services

Docker Volumes

• temporal-postgresql-data: PostgreSQL database storage
  • Contains workflow execution history, job metadata and sync state
  • Used by the PostgreSQL service
• temporal-elasticsearch-data: Elasticsearch search index storage
  • Used by the Elasticsearch service

Monitoring and Debugging

Viewing Logs

Logs from all services can be viewed with:

docker compose logs -f

Logs from a specific service can be viewed with:

docker compose logs -f olake-ui
docker compose logs -f temporal-worker

Troubleshooting

Common Issues

Port Conflicts

If port binding errors occur:

1. Check what's using the ports: lsof -i :8000 (on macOS/Linux)
2. Stop conflicting services or change ports in docker-compose.yml

Database Connection Issues

• Ensure PostgreSQL container is healthy: docker compose ps
• Check PostgreSQL logs: docker compose logs postgresql

Memory Issues

• Ensure Docker has at least 4GB RAM allocated
• Check Docker resource usage: docker stats

Permission Issues

• On Linux, ensure the user is in the docker group
• Check file permissions for the ./olake-data/ directory (or custom path if modified)

Reset Everything

The stack can be completely reset with:

docker compose down -v  # Removes containers and volumes
docker compose up -d    # Fresh start

Warning: This will delete all job data and configurations.