Skip to content

Contributing guidelines

One-time configuration

The steps in this section only ever need to be done once on any particular system.

For Google Cloud configuration:

  1. Install Google Cloud SDK: https://cloud.google.com/sdk/docs/install.

  2. Log in to your work Google Account: run gcloud auth login and follow instructions.

  3. Obtain Google application credentials: run gcloud auth application-default login and follow instructions.

Check that you have the make utility installed, and if not (which is unlikely), install it using your system package manager.

Check that you have java installed. To be able to use all features including hail support use java 11.

Environment configuration

Run make setup-dev to install/update the necessary packages and activate the development environment. You need to do this every time you open a new shell.

It is recommended to use VS Code as an IDE for development.

How to create gentropy step

All gentropy steps can be invoked after successful environment configuration by running

poetry run gentropy step=<step_name>

  1. Create a new step config in the src/gentropy/config.py that inherits from StepConfig class.

  2. Register new step configuration to ConfigStore.

  3. Create a step class that holds the business logic in new file in the src/gentropy.

Contributing checklist

When making changes, and especially when implementing a new module or feature, it's essential to ensure that all relevant sections of the code base are modified.

  • Run make check. This will run the linter and formatter to ensure that the code is compliant with the project conventions.
  • Develop unit tests for your code and run make test. This will run all unit tests in the repository, including the examples appended in the docstrings of some methods.
  • Update the configuration if necessary.
  • Update the documentation and check it with make build-documentation. This will start a local server to browse it (URL will be printed, usually http://127.0.0.1:8000/)

For more details on each of these steps, see the sections below.

Documentation

  • If during development you had a question which wasn't covered in the documentation, and someone explained it to you, add it to the documentation. The same applies if you encountered any instructions in the documentation which were obsolete or incorrect.
  • Documentation autogeneration expressions start with :::. They will automatically generate sections of the documentation based on class and method docstrings. Be sure to update them for:
  • Datasource main page, for example: docs/python_api/datasources/finngen/_finngen.md
  • Dataset definitions, for example: docs/python_api/datasources/finngen/study_index.md
  • Step definition, for example: docs/python_api/steps/finngen_sumstat_preprocess.md

Configuration

  • step default configuration in the src/gentropy/config/ StepConfig derived classes.

Classes

  • Datasource init, for example: src/gentropy/datasource/finngen/__init__.py
  • Dataset classes, for example: src/gentropy/datasource/finngen/study_index.py → FinnGenStudyIndex
  • Step main running class, for example: src/gentropy/finngen_sumstat_preprocess.py

Tests

  • Test study fixture in tests/conftest.py, for example: mock_study_index_finngen in that module
  • Test sample data, for example: tests/gentropy/data_samples/finngen_studies_sample.json
  • Test definition, for example: tests/dataset/test_study_index.py → test_study_index_finngen_creation)

Airflow dags

  • 2023-05-30
  • 2024-09-12
  • Contributors
    • project-defiant
    • tskir
    • d0choa