Troubleshooting OLake

info

Read debug docs here

1. File name too long & FATAL error occurred while reading records: failed to finish backfill chunk 381: main writer closed

2025-02-17T07:03:00Z ERROR main writer closed, with error: failed to create parititon file: failed to create directories[output/otter_db/stream_8/H.

Solution: The max filename length is 255 bytes and this error shows that you have excceded that limit for file creation (might happen if you partion based on STRING field that contain values that are too large). Usually in a linux system, these limits are defined at:

cat /usr/include/linux/limits.h

...
#define NAME_MAX         255    /* # chars in a file name */
#define PATH_MAX        4096    /* # chars in a path name including nul */
...

2. Sync Failure Due to I/O Timeout

2025-06-30T14:12:18Z FATAL error occurred while reading records: error occurred while waiting for context groups: failed to get or split chunks: failed to get next row id: read tcp 192.xx.x.xxx:xxxxx->65.x.xxx.xxx:xxxx: i/o timeout

Cause: Oracle databases often enforce an idle timeout policy, typically between 10 to 15 minutes, after which inactive sessions are terminated. For large tables (e.g., with 10M+ rows), the chunk splitting and row-reading operations may take longer than this default threshold, leading to a connection timeout.

Solution: Set the TIMEOUT parameter (in seconds) to extend the idle timeout:

• Olake UI: Add it under the JDBC URL Parameters textbox while creating or editing the Oracle source.
• Olake CLI: Add it under the jdbc_url_params field in your config.json.

Example:

Olake UI

Olake CLI

"jdbc_url_params": {
    "TIMEOUT": "86400" /* Timeout in seconds; here, 24 hours (24 × 60 × 60) */
}