Contributing to OLake

Hi there! We're thrilled that you'd like to contribute to this project, thank you for your interest. Whether it's a bug report, new feature, correction, or additional documentation, we greatly value feedback and contributions from our community.

If you are a student contributor and new to data engineering domain, finish the below materials and then proceed to pick up issues:

• General Terminologies
• OLake Terminologies

For Contributing to OLake, refer the guide here (detailed Contributing guide comming soon)

A good start will be to explore:

• first good issues
• beginner issues

See the issues and if its looks interesting enough to you, comment below them and one of our maintainers will get in touch with you.

You can also refer to the projects section in GitHub to see who all are working on which issues.

Refer here to build mongodb replicaset.

info

Please join OLake slack community's #contributing-to-olake channel for all the information regarding contributions. Join OLake Community Slack here.

Clone the repo

git clone git@github.com:datazip-inc/olake.git

Create a Pull Request

Make A Branch

• Please create a separate branch for each issue that you're working on. Do not make changes to the default branch (e.g. master, staging) of your fork.

git checkout -b <BRANCH_NAME>

To check your current branch,

git branch

To get back to an existing branch

git checkout <EXISTING_BRANCH_NAME> 

or

git switch <EXISTING_BRANCH_NAME> 

Read more about common git commands: Beginners guide | Advanced git guide

Add, verify and push changes

1. Add your changes

git add . 

2. Verify

Perform git status or git diff to see what all changes occured, which all files are ready to be committed (or staged)

3. Commit

After you are done with all the changes, perform

git commit -am "write a descriptive message here about the nature of changes / addition of code"

info

We accept verified as well as un-verified Commits. You can verify your commits via signing gpg keys. Pull Requests with non-verified commits will be accepted as well. Follow the guide written by OLake DevRel or this guide by GitLab to set up commit signing.

Next, take a pull (in case any new code has been added, this will help you avoid merge conflicts) by doing git pull and then git push origin <YOUR_FEATURE_BRANCH_NAME>.

Push Your Code ASAP

• Push your code as soon as you can. Follow the "early and often" rule.
• Make a pull request as soon as you can and mark the title with a "[WIP]". You can create a draft pull request.

Describe Your Pull Request

• Use the format specified in pull request template for the repository. Populate the pull request descriptioncompletely for maximum verbosity.

  • Tag the actual issue number by replacing #[issue_number] e.g. #42. This closes the issue when your PR is merged.
  • Tag the actual issue author by replacing @[author] e.g. @issue_author. This brings the reporter of the issue into the conversation.
  • Mark the tasks off your checklist by adding an x in the [ ] e.g. [x]. This checks off the boxes in your to-do list. The more boxes you check, the better.

• Describe your change in detail. Too much detail is better than too little.

• Describe how you tested your change.

• Check the Preview tab to make sure the Markdown is correctly rendered and that all tags and references are linked. If not, go back and edit the Markdown.

Request Review

• Once your PR is ready, remove the "[WIP]" from the title and/or change it from a draft PR to a regular PR.
• If a specific reviewer is not assigned automatically, please request a review from the project maintainer.

Debugging OLake

Assumptions

1. You are using VSCode to run OLake locally.
2. You have cloned the project in a suitable directory.

git clone git@github.com:datazip-inc/olake.git

caution

Please read getting started with OLake guide before proceeding to follow along this guide

Steps to Debug

1. Make a directory .vscode (inside OLake project, at root location) if not already created.
2. Create a file named launch.json inside the .vscode directory and paste the beflow config.

.vscode/launch.json

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Launch Go Code",
            "type": "go",
            "request": "launch",
            "program": "{PATH_TO_UPDATE}/drivers/mongodb/main.go",
            "mode": "auto",
            "args": [
                "sync",
                "--config",
                "{PATH_TO_UPDATE}/drivers/mongodb/examples/source.json",
                "--catalog",
                "{PATH_TO_UPDATE}/drivers/mongodb/examples/streams.json",
                "--destination",
                "{PATH_TO_UPDATE}/drivers/mongodb/examples/destination.json",
                // "--state",
                // "{PATH_TO_UPDATE}/drivers/mongodb/examples/state.json",
            ]
        }
    ]
}

Params:

key	value(s)
mode	auto, debug
args	sync , discover

Update PATH_TO_UPDATE with the location where OLake project lives inside your system. For example:

"program": "/Users/john/Desktop/projects/olake/drivers/mongodb/main.go",
...
    "--config",
    "/Users/john/Desktop/projects/olake/drivers/mongodb/examples/source.json",
...          

Now, setup debug points in the codebase and click "Launch Go Code".

info

Refer here fir more details on how to debug OLake's Iceberg writer using VSCode.