validators package

This package contains the validators that are used to ensure the resorces obtained by tasks are correct. Any new validators should be added to this package.

validators.elasticsearch module

Validators for Elasticsearch.

pis.validators.elasticsearch.counts(url: str, index: str, local_path: Path) → bool

Check if the document counts at the remote and local locations match.

Parameters:

• url (str) – The URL of the ElasticSearch instance.

• index (str) – The index to scan.

• local_path (Path) – The path where the documents are stored locally.

Returns:

True if the document counts match, False otherwise.

Return type:

bool

validators.file module

Validators for files.

pis.validators.file.file_exists(path: Path) → bool

Check if a file exists.

Parameters:

path (Path) – The path to the file.

Returns:

True if the file exists, False otherwise.

Return type:

bool

pis.validators.file.file_size(source: str, destination: Path) → bool

Check if the file size of a remote file matches the local file.

Parameters:

• source (str) – The URL of the remote file.

• destination (Path) – The path to the local file.

Returns:

True if the file sizes match, False otherwise.

Return type:

bool

Module contents

Resource validators.

This module contains the validators for the resources generated by tasks.

pis.validators.v(func: Callable[[...], bool], *args, **kwargs) → None

Validate a resource.

v is the way to run a validator inside of the validate function of a Task. Validators are meant to return a boolean, and v will take care of handling it in a common way.

v will run the validator function passed as first parameter passing any args and kwargs to it.

Parameters:

func (Callable[..., bool]) – The validator function to run.

Raises:

ValidationError – If the validator returns False.