Storage

The otter.storage package contains classes for interacting with remote storage services.

storage.model module

Storage model definitions.

otter.storage.model.Revision = float | str | None

Type alias for file revision identifiers.

class otter.storage.model.StatResult(is_dir: bool, is_reg: bool, size: int | None = None, revision: float | str | None = None, mtime: float | None = None)[source]

Bases: object

Dataclass representing file metadata.

is_dir: bool

Whether the resource is a directory.

is_reg: bool

Whether the resource is a regular file.

size: int | None = None

The resource size in bytes, None if unknown.

revision: float | str | None = None

The resource revision identifier.

mtime: float | None = None

The resource modification time as a Unix timestamp, None if unknown.

storage.synchronous.handle module

Storage Handle class.

class otter.storage.synchronous.handle.StorageHandle(location: str | Path, config: Config | None = None, force_local: bool = False)[source]

Bases: object

Storage handle class.

A high-level interface to operate with storage resources (folders or prefixes, files and globs) in tasks. It abstracts the underlying storage and provides a unified API.

The StorageHandle uses the internal method _resolve in its initialization to resolve the absolute location of the resource.

The resolution chain is:

  1. If the location is absolute (remote url or uri), use it as is.

  2. If force_local is False and there is a remote_uri, prepend it.

  3. Otherwise prepend work_path.

When a new cloud storage interface is written, it must be registered in the storage_registry dictionary to be usable by the StorageHandle class. The key is the protocol prefix (e.g., ‘gs’ for Google Storage).

property storage: Storage

Get the storage backend for this handle.

Returns:

The storage backend instance.

Return type:

Storage

property absolute: str

Get the absolute location of this storage handle.

Returns:

The absolute location.

Return type:

str

property is_absolute: bool

Check if the location is absolute.

Returns:

True if the location is absolute, False otherwise.

Return type:

bool

stat() → StatResult[source]

Get metadata for this resource.

Returns:

A StatResult object containing the resource metadata.

Return type:

StatResult

Raises:

NotFoundError – If the resource does not exist.

glob(pattern: str) → list[str][source]

List resources matching a glob under this storage handle’s location.

Parameters:

pattern (str) – The pattern to match files against.

Returns:

A list of absolute locations for the matched resources.

Return type:

list[str]

open(mode: str = 'r') → IOBase[source]

Open this resource as a file-like object.

Parameters:

mode (str) – The file mode. Defaults to ‘r’ for reading.

Returns:

A file-like object for this resource.

read() → tuple[bytes, float | str | None][source]

Read the contents of this resource.

Returns:

The file contents as bytes.

Return type:

bytes

Raises:

  • NotFoundError – If the resource does not exist.

  • TimeoutError – If the read operation times out.

read_text(encoding: str = 'utf-8') → tuple[str, float | str | None][source]

Read the contents of this resource as text.

Parameters:

encoding (str) – The text encoding. Defaults to ‘utf-8’.

Returns:

The file contents as a string.

Return type:

str

Raises:

NotFoundError – If the resource does not exist.

write(data: bytes, *, expected_revision: float | str | None = None) → float | str | None[source]

Write data to this resource.

Optionally, an expected revision can be provided to fail the write if the current revision does not match.

Parameters:

data (bytes) – The data to write.

Returns:

The revision of the written resource.

Return type:

Revision

write_text(data: str, encoding: str = 'utf-8', *, expected_revision: float | str | None = None) → float | str | None[source]

Write text to this resource.

Optionally, an expected revision can be provided to fail the write if the current revision does not match.

Parameters:

  • data (str) – The text to write.

  • encoding (str) – The text encoding. Defaults to ‘utf-8’.

Returns:

The revision of the written resource.

Return type:

Revision

copy_to(dest: StorageHandle) → float | str | None[source]

Copy this resource to the destination handle.

If both source and destination are in the same storage backend, it attempts to use the backend’s native copy method (copy_within) for efficiency. Otherwise, it reads from source and writes to destination.

Parameters:

dest (StorageHandle) – The destination storage handle.

Returns:

The revision of the copied resource at the destination.

Return type:

Revision

Raises:

NotFoundError – If the source does not exist.

storage.synchronous.filesystem module

Local filesystem storage class.

class otter.storage.synchronous.filesystem.FilesystemStorage[source]

Bases: Storage

Local filesystem storage class.

property name: str

The name of the storage backend.

Returns:

The name of the storage backend.

Return type:

str

stat(location: str) → StatResult[source]

Get metadata for a resource.

Parameters:

location (str) – The path or uri to the resource.

Returns:

A StatResult object containing the resource metadata.

Return type:

StatResult

Raises:

NotFoundError – If the resource does not exist.

glob(location: str, pattern: str = '*') → list[str][source]

List resources matching a glob pattern under the given location.

Parameters:

  • location (str) – The base path or uri to search under.

  • pattern (str) – The pattern to match for.

Returns:

A list of absolute file paths or uris.

Return type:

list[str]

open(location: str, mode: str = 'r') → IOBase[source]

Open a file-like object for the given location.

Parameters:

  • location (str) – The path or uri to the file.

  • mode (str) – The file mode, e.g. Defaults to ‘r’ for reading.

Returns:

A file-like object.

Return type:

IOBase

Raises:

NotFoundError – If the file does not exist (in read mode).

read(location: str) → tuple[bytes, float | str | None][source]

Read the contents of a file.

Parameters:

location (str) – The path or uri to the file.

Returns:

A tuple of (file contents as bytes, file revision).

Return type:

tuple[bytes, Revision]

Raises:

  • NotFoundError – If the file does not exist.

  • TimeoutError – If the read operation times out.

read_text(location: str, encoding: str = 'utf-8') → tuple[str, float | str | None][source]

Read the contents of a file as text.

Parameters:

  • location (str) – The path or uri to the file.

  • encoding (str) – The text encoding. Defaults to ‘utf-8’.

Returns:

A tuple of (file contents as a string, file revision).

Return type:

tuple[str, Revision]

Raises:

NotFoundError – If the file does not exist.

write(location: str, data: bytes, *, expected_revision: float | str | None = None) → float | str | None[source]

Write data to a file.

Optionally, an expected revision can be provided to fail the write if the current revision does not match.

Note

The locking mechanism to enforce expected revision is backend-specific and may be blocking.

Parameters:

  • location (str) – The path or uri to the file.

  • data (bytes) – The data to write.

  • expected_revision (Revision) – (keyword-only) The expected target revision.

Returns:

The revision of the written file.

Return type:

Revision

Raises:

PreconditionFailedError – If expected revision does not match.

write_text(location: str, data: str, *, encoding: str = 'utf-8', expected_revision: float | str | None = None) → float | str | None[source]

Write text to a file.

Optionally, an expected revision can be provided to fail the write if the current revision does not match.

Note

The locking mechanism to enforce expected revision is backend-specific and may be blocking.

Parameters:

  • location (str) – The path or uri to the file.

  • data (str) – The text to write.

  • encoding (str) – (keyword-only) The text encoding. Defaults to ‘utf-8’.

  • expected_revision (Revision) – (keyword-only) The expected target revision.

Returns:

The revision of the written file.

Return type:

Revision

Raises:

PreconditionFailedError – If expected revision does not match.

copy_within(src: str, dst: str) → float | str | None[source]

Copy a file within the same storage backend.

This method allows for efficient copies without downloading/uploading.

Parameters:

  • src (str) – The source path of the file to copy.

  • dst (str) – The destination path to copy the file to.

Returns:

The revision of the copied file.

Return type:

Revision

Raises:

NotFoundError – If the source file does not exist.

storage.synchronous.http module

HTTP Storage class.

class otter.storage.synchronous.http.HTTPStorage[source]

Bases: Storage

HTTP Storage class.

This class implements the Storage interface for HTTP resources. Uses requests.Session for HTTP operations with connection pooling.

property name: str

The name of the storage backend.

Returns:

The name of the storage backend.

Return type:

str

stat(location: str) → StatResult[source]

Get metadata for a resource.

Parameters:

location (str) – The path or uri to the resource.

Returns:

A StatResult object containing the resource metadata.

Return type:

StatResult

Raises:

NotFoundError – If the resource does not exist.

open(location: str, mode: str = 'r') → IOBase[source]

Open is not supported for HTTP storage.

Raises:

NotImplementedError – Always, since HTTP storage does not support opening file-like objects.

glob(location: str, pattern: str) → list[str][source]

Glob is not supported for HTTP storage.

Raises:

NotImplementedError – Always, since HTTP storage does not support globbing.

read(location: str) → tuple[bytes, float | str | None][source]

Read the contents of a file.

Parameters:

location (str) – The path or uri to the file.

Returns:

A tuple of (file contents as bytes, file revision).

Return type:

tuple[bytes, Revision]

Raises:

  • NotFoundError – If the file does not exist.

  • TimeoutError – If the read operation times out.

read_text(location: str, encoding: str = 'utf-8') → tuple[str, float | str | None][source]

Read the contents of a file as text.

Parameters:

  • location (str) – The path or uri to the file.

  • encoding (str) – The text encoding. Defaults to ‘utf-8’.

Returns:

A tuple of (file contents as a string, file revision).

Return type:

tuple[str, Revision]

Raises:

NotFoundError – If the file does not exist.

write(location: str, data: bytes, *, encoding: str = 'utf-8', expected_revision: float | str | None = None) → float | str | None[source]

Writing is not supported for HTTP storage.

Raises:

NotImplementedError – Always, since HTTP storage is read-only.

write_text(location: str, data: str, *, encoding: str = 'utf-8', expected_revision: float | str | None = None) → float | str | None[source]

Writing is not supported for HTTP storage.

Raises:

NotImplementedError – Always, since HTTP storage is read-only.

copy_within(src: str, dst: str) → float | str | None[source]

Copying is not supported for HTTP storage.

Raises:

NotImplementedError – Always, since HTTP storage is read-only.

storage.synchronous.google module

Google Cloud Storage class.

class otter.storage.synchronous.google.GoogleStorage[source]

Bases: Storage

Google Cloud Storage class using google-cloud-storage for operations.

property name: str

The name of the storage backend.

Returns:

The name of the storage backend.

Return type:

str

stat(location: str) → StatResult[source]

Get metadata for a resource.

Parameters:

location (str) – The path or uri to the resource.

Returns:

A StatResult object containing the resource metadata.

Return type:

StatResult

Raises:

NotFoundError – If the resource does not exist.

glob(location: str, pattern: str = '*') → list[str][source]

List resources matching a glob pattern under the given location.

Parameters:

  • location (str) – The base path or uri to search under.

  • pattern (str) – The pattern to match for.

Returns:

A list of absolute file paths or uris.

Return type:

list[str]

open(location: str, mode: str = 'r') → IOBase[source]

Open a file-like object for the given location.

Parameters:

  • location (str) – The path or uri to the file.

  • mode (str) – The file mode, e.g. Defaults to ‘r’ for reading.

Returns:

A file-like object.

Return type:

IOBase

Raises:

NotFoundError – If the file does not exist (in read mode).

read(location: str) → tuple[bytes, float | str | None][source]

Read the contents of a file.

Parameters:

location (str) – The path or uri to the file.

Returns:

A tuple of (file contents as bytes, file revision).

Return type:

tuple[bytes, Revision]

Raises:

  • NotFoundError – If the file does not exist.

  • TimeoutError – If the read operation times out.

read_text(location: str, encoding: str = 'utf-8') → tuple[str, float | str | None][source]

Read the contents of a file as text.

Parameters:

  • location (str) – The path or uri to the file.

  • encoding (str) – The text encoding. Defaults to ‘utf-8’.

Returns:

A tuple of (file contents as a string, file revision).

Return type:

tuple[str, Revision]

Raises:

NotFoundError – If the file does not exist.

write(location: str, data: bytes, *, expected_revision: float | str | None = None) → float | str | None[source]

Write data to a file.

Optionally, an expected revision can be provided to fail the write if the current revision does not match.

Note

The locking mechanism to enforce expected revision is backend-specific and may be blocking.

Parameters:

  • location (str) – The path or uri to the file.

  • data (bytes) – The data to write.

  • expected_revision (Revision) – (keyword-only) The expected target revision.

Returns:

The revision of the written file.

Return type:

Revision

Raises:

PreconditionFailedError – If expected revision does not match.

write_text(location: str, data: str, *, encoding: str = 'utf-8', expected_revision: float | str | None = None) → float | str | None[source]

Write text to a file.

Optionally, an expected revision can be provided to fail the write if the current revision does not match.

Note

The locking mechanism to enforce expected revision is backend-specific and may be blocking.

Parameters:

  • location (str) – The path or uri to the file.

  • data (str) – The text to write.

  • encoding (str) – (keyword-only) The text encoding. Defaults to ‘utf-8’.

  • expected_revision (Revision) – (keyword-only) The expected target revision.

Returns:

The revision of the written file.

Return type:

Revision

Raises:

PreconditionFailedError – If expected revision does not match.

copy_within(src: str, dst: str) → float | str | None[source]

Copy a file within the same storage backend.

This method allows for efficient copies without downloading/uploading.

Parameters:

  • src (str) – The source path of the file to copy.

  • dst (str) – The destination path to copy the file to.

Returns:

The revision of the copied file.

Return type:

Revision

Raises:

NotFoundError – If the source file does not exist.

storage.synchronous.noop module

Noop storage class.

class otter.storage.synchronous.noop.NoopStorage[source]

Bases: Storage

No-op storage helper class.

Used as placeholder for unrecognized protocols.

property name: str

The name of the storage provider.

stat(location: str) → StatResult[source]

Get metadata for a resource.

Parameters:

location (str) – The path or uri to the resource.

Returns:

A StatResult object containing the resource metadata.

Return type:

StatResult

Raises:

NotFoundError – If the resource does not exist.

glob(location: str, pattern: str) → list[str][source]

List resources matching a glob pattern under the given location.

Parameters:

  • location (str) – The base path or uri to search under.

  • pattern (str) – The pattern to match for.

Returns:

A list of absolute file paths or uris.

Return type:

list[str]

open(location: str, mode: str = 'r') → IOBase[source]

Open a file-like object for the given location.

Parameters:

  • location (str) – The path or uri to the file.

  • mode (str) – The file mode, e.g. Defaults to ‘r’ for reading.

Returns:

A file-like object.

Return type:

IOBase

Raises:

NotFoundError – If the file does not exist (in read mode).

read(location: str) → tuple[bytes, float | str | None][source]

Read the contents of a file.

Parameters:

location (str) – The path or uri to the file.

Returns:

A tuple of (file contents as bytes, file revision).

Return type:

tuple[bytes, Revision]

Raises:

  • NotFoundError – If the file does not exist.

  • TimeoutError – If the read operation times out.

read_text(location: str, encoding: str = 'utf-8') → tuple[str, float | str | None][source]

Read the contents of a file as text.

Parameters:

  • location (str) – The path or uri to the file.

  • encoding (str) – The text encoding. Defaults to ‘utf-8’.

Returns:

A tuple of (file contents as a string, file revision).

Return type:

tuple[str, Revision]

Raises:

NotFoundError – If the file does not exist.

write(location: str, data: bytes, *, expected_revision: float | str | None = None) → float | str | None[source]

Write data to a file.

Optionally, an expected revision can be provided to fail the write if the current revision does not match.

Note

The locking mechanism to enforce expected revision is backend-specific and may be blocking.

Parameters:

  • location (str) – The path or uri to the file.

  • data (bytes) – The data to write.

  • expected_revision (Revision) – (keyword-only) The expected target revision.

Returns:

The revision of the written file.

Return type:

Revision

Raises:

PreconditionFailedError – If expected revision does not match.

write_text(location: str, data: str, *, encoding: str = 'utf-8', expected_revision: float | str | None = None) → float | str | None[source]

Write text to a file.

Optionally, an expected revision can be provided to fail the write if the current revision does not match.

Note

The locking mechanism to enforce expected revision is backend-specific and may be blocking.

Parameters:

  • location (str) – The path or uri to the file.

  • data (str) – The text to write.

  • encoding (str) – (keyword-only) The text encoding. Defaults to ‘utf-8’.

  • expected_revision (Revision) – (keyword-only) The expected target revision.

Returns:

The revision of the written file.

Return type:

Revision

Raises:

PreconditionFailedError – If expected revision does not match.

copy_within(src: str, dst: str) → float | str | None[source]

Copy a file within the same storage backend.

This method allows for efficient copies without downloading/uploading.

Parameters:

  • src (str) – The source path of the file to copy.

  • dst (str) – The destination path to copy the file to.

Returns:

The revision of the copied file.

Return type:

Revision

Raises:

NotFoundError – If the source file does not exist.

storage.asynchronous.handle module

Asynchronous Storage Handle class.

class otter.storage.asynchronous.handle.AsyncStorageHandle(location: str | Path, config: Config | None = None, force_local: bool = False)[source]

Bases: object

Asynchronous Storage handle class.

A high-level interface to operate with storage resources (folders or prefixes, files and globs) in tasks. It abstracts the underlying storage and provides a unified API.

The AsyncStorageHandle uses the internal method _resolve in its initialization to resolve the absolute location of the resource.

The resolution chain is:

  1. If the location is absolute (remote url or uri), use it as is.

  2. If force_local is False and there is a remote_uri, prepend it.

  3. Otherwise prepend work_path.

When a new cloud storage interface is written, it must be registered in the storage_registry dictionary to be usable by the AsyncStorageHandle class. The key is the protocol prefix (e.g., ‘gs’ for Google Storage).

property storage: AsyncStorage

Get the storage backend for this handle.

Returns:

The storage backend instance.

Return type:

AsyncStorage

property absolute: str

Get the absolute location of this storage handle.

Returns:

The absolute location.

Return type:

str

property is_absolute: bool

Check if the location is absolute.

Returns:

True if the location is absolute, False otherwise.

Return type:

bool

async stat() → StatResult[source]

Get metadata for this resource.

Returns:

A StatResult object containing the resource metadata.

Return type:

StatResult

Raises:

NotFoundError – If the resource does not exist.

async glob(pattern: str) → list[str][source]

List resources matching a glob under this storage handle’s location.

Parameters:

pattern (str) – The pattern to match files against.

Returns:

A list of absolute locations for the matched resources.

Return type:

list[str]

async read() → tuple[bytes, float | str | None][source]

Read the contents of this resource.

Returns:

The file contents as bytes.

Return type:

bytes

Raises:

  • NotFoundError – If the resource does not exist.

  • TimeoutError – If the read operation times out.

async read_text(encoding: str = 'utf-8') → tuple[str, float | str | None][source]

Read the contents of this resource as text.

Parameters:

encoding (str) – The text encoding. Defaults to ‘utf-8’.

Returns:

The file contents as a string.

Return type:

str

Raises:

NotFoundError – If the resource does not exist.

async write(data: bytes, *, expected_revision: float | str | None = None) → float | str | None[source]

Write data to this resource.

Optionally, an expected revision can be provided to fail the write if the current revision does not match.

Parameters:

data (bytes) – The data to write.

Returns:

The revision of the written resource.

Return type:

Revision

async write_text(data: str, encoding: str = 'utf-8', *, expected_revision: float | str | None = None) → float | str | None[source]

Write text to this resource.

Optionally, an expected revision can be provided to fail the write if the current revision does not match.

Parameters:

  • data (str) – The text to write.

  • encoding (str) – The text encoding. Defaults to ‘utf-8’.

Returns:

The revision of the written resource.

Return type:

Revision

async copy_to(dest: AsyncStorageHandle) → float | str | None[source]

Copy this resource to the destination handle.

If both source and destination are in the same storage backend, it attempts to use the backend’s native copy method (copy_within) for efficiency. Otherwise, it reads from source and writes to destination.

Parameters:

dest (AsyncStorageHandle) – The destination storage handle.

Returns:

The revision of the copied resource at the destination.

Return type:

Revision

Raises:

NotFoundError – If the source does not exist.

storage.asynchronous.filesystem module

Local filesystem storage class.

class otter.storage.asynchronous.filesystem.AsyncFilesystemStorage[source]

Bases: AsyncStorage

Local filesystem storage class.

property name: str

The name of the storage backend.

Returns:

The name of the storage backend.

Return type:

str

async stat(location: str) → StatResult[source]

Get metadata for a resource.

Parameters:

location (str) – The path or uri to the resource.

Returns:

A StatResult object containing the resource metadata.

Return type:

StatResult

Raises:

NotFoundError – If the resource does not exist.

async glob(location: str, pattern: str = '*') → list[str][source]

List resources matching a glob pattern under the given location.

Parameters:

  • location (str) – The base path or uri to search under.

  • pattern (str) – The pattern to match for.

Returns:

A list of absolute file paths or uris.

Return type:

list[str]

async read(location: str) → tuple[bytes, float | str | None][source]

Read the contents of a file.

Parameters:

location (str) – The path or uri to the file.

Returns:

A tuple of (file contents as bytes, file revision).

Return type:

tuple[bytes, Revision]

Raises:

  • NotFoundError – If the file does not exist.

  • TimeoutError – If the read operation times out.

async read_text(location: str, encoding: str = 'utf-8') → tuple[str, float | str | None][source]

Read the contents of a file as text.

Parameters:

  • location (str) – The path or uri to the file.

  • encoding (str) – The text encoding. Defaults to ‘utf-8’.

Returns:

A tuple of (file contents as a string, file revision).

Return type:

tuple[str, Revision]

Raises:

NotFoundError – If the file does not exist.

async write(location: str, data: bytes, *, expected_revision: float | str | None = None) → float | str | None[source]

Write data to a file.

Optionally, an expected revision can be provided to fail the write if the current revision does not match.

Note

The locking mechanism to enforce expected revision is backend-specific and may be blocking.

Parameters:

  • location (str) – The path or uri to the file.

  • data (bytes) – The data to write.

  • expected_revision (Revision) – (keyword-only) The expected target revision.

Returns:

The revision of the written file.

Return type:

Revision

Raises:

PreconditionFailedError – If expected revision does not match.

async write_text(location: str, data: str, *, encoding: str = 'utf-8', expected_revision: float | str | None = None) → float | str | None[source]

Write text to a file.

Optionally, an expected revision can be provided to fail the write if the current revision does not match.

Note

The locking mechanism to enforce expected revision is backend-specific and may be blocking.

Parameters:

  • location (str) – The path or uri to the file.

  • data (str) – The text to write.

  • encoding (str) – (keyword-only) The text encoding. Defaults to ‘utf-8’.

  • expected_revision (Revision) – (keyword-only) The expected target revision.

Returns:

The revision of the written file.

Return type:

Revision

Raises:

PreconditionFailedError – If expected revision does not match.

async copy_within(src: str, dst: str) → float | str | None[source]

Copy a file within the same storage backend.

This method allows for efficient copies without downloading/uploading.

Parameters:

  • src (str) – The source path of the file to copy.

  • dst (str) – The destination path to copy the file to.

Returns:

The revision of the copied file.

Return type:

Revision

Raises:

NotFoundError – If the source file does not exist.

storage.asynchronous.http module

HTTP Storage class.

class otter.storage.asynchronous.http.AsyncHTTPStorage[source]

Bases: AsyncStorage

HTTP Storage class.

This class implements the Storage interface for HTTP resources. Uses httpx.AsyncClient for async HTTP operations with connection pooling.

property name: str

The name of the storage backend.

Returns:

The name of the storage backend.

Return type:

str

async stat(location: str) → StatResult[source]

Get metadata for a resource.

Parameters:

location (str) – The path or uri to the resource.

Returns:

A StatResult object containing the resource metadata.

Return type:

StatResult

Raises:

NotFoundError – If the resource does not exist.

async glob(location: str, pattern: str) → list[str][source]

Glob is not supported for HTTP storage.

Raises:

NotImplementedError – Always, since HTTP storage does not support globbing.

async read(location: str) → tuple[bytes, float | str | None][source]

Read the contents of a file.

Parameters:

location (str) – The path or uri to the file.

Returns:

A tuple of (file contents as bytes, file revision).

Return type:

tuple[bytes, Revision]

Raises:

  • NotFoundError – If the file does not exist.

  • TimeoutError – If the read operation times out.

async read_text(location: str, encoding: str = 'utf-8') → tuple[str, float | str | None][source]

Read the contents of a file as text.

Parameters:

  • location (str) – The path or uri to the file.

  • encoding (str) – The text encoding. Defaults to ‘utf-8’.

Returns:

A tuple of (file contents as a string, file revision).

Return type:

tuple[str, Revision]

Raises:

NotFoundError – If the file does not exist.

async write(location: str, data: bytes, *, encoding: str = 'utf-8', expected_revision: float | str | None = None) → float | str | None[source]

Writing is not supported for HTTP storage.

Raises:

NotImplementedError – Always, since HTTP storage is read-only.

async write_text(location: str, data: str, *, encoding: str = 'utf-8', expected_revision: float | str | None = None) → float | str | None[source]

Writing is not supported for HTTP storage.

Raises:

NotImplementedError – Always, since HTTP storage is read-only.

async copy_within(src: str, dst: str) → float | str | None[source]

Copying is not supported for HTTP storage.

Raises:

NotImplementedError – Always, since HTTP storage is read-only.

storage.asynchronous.google module

Google Cloud Storage class.

class otter.storage.asynchronous.google.AsyncGoogleStorage[source]

Bases: AsyncStorage

Google Cloud Storage class using gcloud-aio-storage for async operations.

property name: str

The name of the storage backend.

Returns:

The name of the storage backend.

Return type:

str

async stat(location: str) → StatResult[source]

Get metadata for a resource.

Parameters:

location (str) – The path or uri to the resource.

Returns:

A StatResult object containing the resource metadata.

Return type:

StatResult

Raises:

NotFoundError – If the resource does not exist.

async glob(location: str, pattern: str = '*') → list[str][source]

List resources matching a glob pattern under the given location.

Parameters:

  • location (str) – The base path or uri to search under.

  • pattern (str) – The pattern to match for.

Returns:

A list of absolute file paths or uris.

Return type:

list[str]

async read(location: str) → tuple[bytes, float | str | None][source]

Read the contents of a file.

Parameters:

location (str) – The path or uri to the file.

Returns:

A tuple of (file contents as bytes, file revision).

Return type:

tuple[bytes, Revision]

Raises:

  • NotFoundError – If the file does not exist.

  • TimeoutError – If the read operation times out.

async read_text(location: str, encoding: str = 'utf-8') → tuple[str, float | str | None][source]

Read the contents of a file as text.

Parameters:

  • location (str) – The path or uri to the file.

  • encoding (str) – The text encoding. Defaults to ‘utf-8’.

Returns:

A tuple of (file contents as a string, file revision).

Return type:

tuple[str, Revision]

Raises:

NotFoundError – If the file does not exist.

async write(location: str, data: bytes, *, expected_revision: float | str | None = None) → float | str | None[source]

Write data to a file.

Optionally, an expected revision can be provided to fail the write if the current revision does not match.

Note

The locking mechanism to enforce expected revision is backend-specific and may be blocking.

Parameters:

  • location (str) – The path or uri to the file.

  • data (bytes) – The data to write.

  • expected_revision (Revision) – (keyword-only) The expected target revision.

Returns:

The revision of the written file.

Return type:

Revision

Raises:

PreconditionFailedError – If expected revision does not match.

async write_text(location: str, data: str, *, encoding: str = 'utf-8', expected_revision: float | str | None = None) → float | str | None[source]

Write text to a file.

Optionally, an expected revision can be provided to fail the write if the current revision does not match.

Note

The locking mechanism to enforce expected revision is backend-specific and may be blocking.

Parameters:

  • location (str) – The path or uri to the file.

  • data (str) – The text to write.

  • encoding (str) – (keyword-only) The text encoding. Defaults to ‘utf-8’.

  • expected_revision (Revision) – (keyword-only) The expected target revision.

Returns:

The revision of the written file.

Return type:

Revision

Raises:

PreconditionFailedError – If expected revision does not match.

async copy_within(src: str, dst: str) → float | str | None[source]

Copy a file within the same storage backend.

This method allows for efficient copies without downloading/uploading.

Parameters:

  • src (str) – The source path of the file to copy.

  • dst (str) – The destination path to copy the file to.

Returns:

The revision of the copied file.

Return type:

Revision

Raises:

NotFoundError – If the source file does not exist.

Module contents

Remote storage implementation classes.