Overview

A job in OLake is a single pipeline that defines how data should be synchronized from a source to a destination. It includes details such as the source and destination configurations, the streams to be synchronized, the replication frequency and other related job configurations.

There are 4 types of associated jobs, depending on the job status:

1. Active Jobs: These jobs are currently running or scheduled to run. Any newly created job will also appear on the Active tab. They are visible in the "Active Jobs" section of the OLake UI.
2. Inactive Jobs: These jobs are not currently running but can be resumed. A paused job will also be under Inactive Jobs tab. They are visible in the "Inactive Jobs" section of the OLake UI.
3. Saved Jobs: These jobs are saved configurations that can be used to create new job runs. A saved job when scheduled or run will appear under the Active Jobs tab. They are visible in the "Saved Jobs" section of the OLake UI.

4. Failed Jobs: These jobs have encountered errors during execution or failed sync execution. They are visible in the "Failed Jobs" section of the OLake UI.

OLake jobs can be created, edited, and managed through the OLake UI. You can also view complete job history, detailed logs for each job run, and other details to monitor the progress and status of your jobs.

Key Features

• Job Creation: Easily create jobs with source and destination configurations.
• Job Management: Start, pause, resume, and delete jobs as needed.
• Job Monitoring: View job history, logs, and status to track job progress.
• Job Scheduling & Configuration: Customize job settings such as replication frequency, run job at specific intervals (cron), job name, and more.

Logs

Logs are essential for debugging and monitoring the execution of OLake jobs. You can view logs for each job run to understand what happened during the execution, including any errors or warnings that may have occurred.

To view a job log:

1. Click on the "Job History" button for the job you want to inspect.
2. Select the specific job run you want to view logs for.
3. Click on the "View Logs" button to see the detailed logs for that job run.

Logs Features

1. You can search logs using the search bar at the top of the logs page.
2. You can hit on the refresh button to refresh the logs as they are being generated.

Things to Note:

1. An active and failed job will be shown in both the "Active Jobs" and "Failed Jobs" sections.
2. An paused job will be shown in the "Inactive Jobs" section.