Troubleshooting OLake

info

Read debug docs here

1. File name too long & FATAL error occurred while reading records: failed to finish backfill chunk 381: main writer closed

2025-02-17T07:03:00Z ERROR main writer closed, with error: failed to create parititon file: failed to create directories[output/otter_db/stream_8/H.

Solution: The max filename length is 255 bytes and this error shows that you have excceded that limit for file creation (might happen if you partion based on STRING field that contain values that are too large). Usually in a linux system, these limits are defined at:

cat /usr/include/linux/limits.h

...
#define NAME_MAX         255    /* # chars in a file name */
#define PATH_MAX        4096    /* # chars in a path name including nul */
...

2. No encryption (SQLSTATE 28000)

FATAL failed to ping database: failed to connect to `host=product-team-testing.postgres.database.azure.com user=product_team database=postgres`: server error (FATAL: no p
g_hba.conf entry for host "4.240.65.100", user "product_team", database "postgres", no encryption (SQLSTATE 28000))

Solution: Change the ssl.mode: disable to ssl.mode: allow to resolve this issue. This is a known issue with Azure Postgres and the solution is to set the ssl.mode to allow in the connection string.

Unexpected EOF

Running Postgres Sync ran into following error: FATAL error occurred while reading records: failed to receive message from wal: receive message failed: unexpected EOF - Solution: Set the value of intial_wait_time: 10 in case if you have increased the value in source database config file. This could be because of network issues, try again in that case.