Timenet.registry

timenet.registry ¶source

Dataset registries: serve compiled TimeF versions and search over metadata.

TIMENET_REGISTRY_URL `module-attribute` ¶source

TIMENET_REGISTRY_URL = 'https://registry.timenet.ai'

The hosted registry URL represented by timenet://.

BaseRegistry ¶source

Bases: ABC

A registry of TimeF manifests, files, and searchable dataset metadata.

:meth:open_release and :meth:download_release handle one release without its payload sources. :func:~timenet.registry.copy_release copies a release with its payload sources.

has_catalog `class-attribute` ¶source

has_catalog: bool = True

Whether :meth:list_datasets and :meth:search work. Without a catalog, both raise NotImplementedError.

location `abstractmethod` `property` ¶source

location: str

The registry's URL, path, or name.

read_only `property` ¶source

read_only: bool

Whether this registry refuses every write, set with read_only=True.

requires_open_access `property` ¶source

requires_open_access: bool

Whether stored or served releases must declare open access.

check_writable ¶source

check_writable(write: str) -> None

Refuse a write if this registry is read-only.

Parameters:

Name	Type	Description	Default
`write`	`str`	The write to refuse, for the message, such as `"put_dataset()"`.	required

Raises:

Type	Description
`TimeNetRegistryError`	If this registry is read-only.

download_release ¶source

download_release(
    reference: DatasetRef,
    directory: str | Path,
    *,
    force: bool = False,
    manifest: Manifest | None = None,
    progress_cb: ByteCountCallback | None = None,
) -> None

Download one release's files to a local directory.

Reuse a complete copy unless force is set. Replace directory only after every downloaded file matches its manifest record.

Parameters:

Name	Type	Description	Default
`reference`	`DatasetRef`	The exact release.	required
`directory`	`str \| Path`	The local release directory, `<root>/<dataset_id>/<version>`.	required
`force`	`bool`	Download again even if a complete copy exists.	`False`
`manifest`	`Manifest \| None`	The release's manifest, or `None` to fetch it from this registry.	`None`
`progress_cb`	`ByteCountCallback \| None`	Callback that receives each downloaded file's byte count, or `None`.	`None`

Raises:

Type	Description
`TimeNetFileError`	If a downloaded file's size or checksum differs from its manifest record.

find_manifest ¶source

find_manifest(release: DatasetRef) -> Manifest | None

Return the manifest of a release, or None if this registry does not hold it.

Parameters:

Name	Type	Description	Default
`release`	`DatasetRef`	The exact release.	required

Returns:

Type	Description
`Manifest \| None`	The manifest, or `None`.

Raises:

Type	Description
`TimeFFormatError`	If the stored manifest names another release.

get_manifest `abstractmethod` ¶source

get_manifest(
    dataset_id: str, version: str | None = None
) -> Manifest

Return the manifest of a dataset.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str \| None`	The version string, or `None` for the latest version.	`None`

Returns:

Name	Type	Description
`The`	`Manifest`	class:`~timenet.manifest.Manifest` of the dataset.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version is unknown.

list_datasets `abstractmethod` ¶source

list_datasets() -> list[DatasetMetadata]

Return the metadata of every dataset, using the latest version. The result is sorted by dataset id.

Returns:

Name	Type	Description
`One`	`list[DatasetMetadata]`	class:`~timenet.types.DatasetMetadata` object for each dataset.

open_file `abstractmethod` ¶source

open_file(
    dataset_id: str, version: str, relpath: str
) -> BinaryIO

Open one file of a dataset version for binary reading.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required
`relpath`	`str`	The file path relative to the version directory.	required

Returns:

Type	Description
`BinaryIO`	An open binary file object.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version is unknown.

open_release `abstractmethod` ¶source

open_release(
    reference: DatasetRef,
    *,
    manifest: Manifest | None = None,
) -> DatasetVersion

Open an exact release without attaching its payload sources.

Opening reads metadata only. Attach payload sources with :func:~timenet.registry.dependencies.attach_payload_sources.

Parameters:

Name	Type	Description	Default
`reference`	`DatasetRef`	The dataset ID and exact version to open.	required
`manifest`	`Manifest \| None`	A manifest already read from this registry, or `None` to fetch it.	`None`

Returns:

Type	Description
`DatasetVersion`	A handle to the release's manifest and files.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the registry lacks the release.
`TimeFFormatError`	If the manifest declares a different release.
`TimeNetFileError`	If a local file is missing or has an incorrect size.

resolve ¶source

resolve(
    dataset_id: str, version: Version | None
) -> ReleaseLookup

Check whether the registry serves a dataset release.

If :attr:requires_open_access is true, the registry serves a release only if its card and each card in its lineage declare open access.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset ID.	required
`version`	`Version \| None`	An exact version, or `None` for the latest version.	required

Returns:

Type	Description
`ReleaseLookup`	`FoundInRegistry` if served, `NotServed` if access is restricted, or `NotFound` if absent.

search ¶source

search(filter: DatasetFilter) -> list[DatasetMetadata]

Find datasets that match a filter.

A task or time_series_spec filter reads the manifest of a dataset only if its card matches.

Parameters:

Name	Type	Description	Default
`filter`	`DatasetFilter`	The filters and the limit.	required

Returns:

Type	Description
`list[DatasetMetadata]`	The matching dataset metadata, sorted by dataset ID, with at most `filter.limit` entries.

DatasetVersion `dataclass` ¶source

An opened release: its manifest, filesystem, and its already-opened payload sources.

filesystem `instance-attribute` ¶source

filesystem: FileSystem

manifest `instance-attribute` ¶source

manifest: Manifest

payload_sources `class-attribute` `instance-attribute` ¶source

payload_sources: dict[SourceKey, DatasetVersion] = field(
    default_factory=dict, compare=False, repr=False
)

The opened release behind each key of manifest.payload_sources. Each was checked against its pins when this release was opened, and its own payload sources are not opened.

root `instance-attribute` ¶source

root: str

fsspec_mapper ¶source

fsspec_mapper(relpath: str) -> FSMap

Return an fsspec mapping for a version-relative directory.

The mapping uses the prepared filesystem, including its credentials and caching. Consumers choose the store's read-only policy when opening it.

resolve_file_path ¶source

resolve_file_path(relative_path: str) -> str

Return a file's filesystem path without checking or downloading the file.

Parameters:

Name	Type	Description	Default
`relative_path`	`str`	A path within this version, such as `time_series/part-000.parquet`. Absolute paths and paths that escape the version directory are rejected.	required

Returns:

Type	Description
`str`	The path to use with :attr:`filesystem`.

resolve_payload_source ¶source

resolve_payload_source(
    source_key: SourceKey | None,
) -> DatasetVersion

Return the opened release that stores a payload file.

Parameters:

Name	Type	Description	Default
`source_key`	`SourceKey \| None`	The `source_key` of a catalog row, a key of `manifest.payload_sources`. `None` means this release stores the file.	required

Returns:

Type	Description
`DatasetVersion`	This release or one of its already-opened payload sources. No files are opened here.

Raises:

Type	Description
`TimeFFormatError`	If the catalog names a payload source the manifest does not list.

LocalRegistry ¶source

Bases: WritableRegistry

Read and write dataset versions in a local directory.

has_catalog `class-attribute` ¶source

has_catalog: bool = True

Whether :meth:list_datasets and :meth:search work. Without a catalog, both raise NotImplementedError.

location `property` ¶source

location: str

The registry's absolute root path.

read_only `property` ¶source

read_only: bool

Whether this registry refuses every write, set with read_only=True.

requires_clean_code_provenance `property` ¶source

requires_clean_code_provenance: bool

Whether this registry refuses releases built from locally modified TimeNet code.

Returns:

Type	Description
`bool`	`False`, since a local registry takes any release.

requires_open_access `property` ¶source

requires_open_access: bool

Whether local releases must declare open access.

Returns:

Type	Description
`bool`	`False`, since a local registry accepts any access level.

root `property` ¶source

root: Path

The directory this registry reads from and writes to.

check_no_linked_folders ¶source

check_no_linked_folders() -> None

Check that no dataset or release folder is a symlink, which :meth:releases would skip.

Raises:

Type	Description
`TimeFValidationError`	If a dataset or release folder is a symlink.
`OSError`	If a folder cannot be read.

check_writable ¶source

check_writable(write: str) -> None

Refuse a write if this registry is read-only.

Parameters:

Name	Type	Description	Default
`write`	`str`	The write to refuse, for the message, such as `"put_dataset()"`.	required

Raises:

Type	Description
`TimeNetRegistryError`	If this registry is read-only.

damage ¶source

damage(
    release: DatasetRef, manifest: Manifest | None = None
) -> TimeNetFileError | None

Return the first missing or resized file of a stored release, or None for a complete copy.

Parameters:

Name	Type	Description	Default
`release`	`DatasetRef`	The exact release.	required
`manifest`	`Manifest \| None`	Its manifest, or `None` to read the stored one.	`None`

Returns:

Type	Description
`TimeNetFileError \| None`	The error that names the file, or `None`.

download_release ¶source

download_release(
    reference: DatasetRef,
    directory: str | Path,
    *,
    force: bool = False,
    manifest: Manifest | None = None,
    progress_cb: ByteCountCallback | None = None,
) -> None

Download one release's files to a local directory.

Reuse a complete copy unless force is set. Replace directory only after every downloaded file matches its manifest record.

Parameters:

Name	Type	Description	Default
`reference`	`DatasetRef`	The exact release.	required
`directory`	`str \| Path`	The local release directory, `<root>/<dataset_id>/<version>`.	required
`force`	`bool`	Download again even if a complete copy exists.	`False`
`manifest`	`Manifest \| None`	The release's manifest, or `None` to fetch it from this registry.	`None`
`progress_cb`	`ByteCountCallback \| None`	Callback that receives each downloaded file's byte count, or `None`.	`None`

Raises:

Type	Description
`TimeNetFileError`	If a downloaded file's size or checksum differs from its manifest record.

exists ¶source

exists(dataset_id: str, version: str) -> bool

Check whether this registry holds a dataset version.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required

Returns:

Type	Description
`bool`	`True` if the version has a committed manifest, else `False`.

find_manifest ¶source

find_manifest(release: DatasetRef) -> Manifest | None

Return the manifest of a release, or None if this registry does not hold it.

Parameters:

Name	Type	Description	Default
`release`	`DatasetRef`	The exact release.	required

Returns:

Type	Description
`Manifest \| None`	The manifest, or `None`.

Raises:

Type	Description
`TimeFFormatError`	If the stored manifest names another release.

get_manifest ¶source

get_manifest(
    dataset_id: str, version: str | None = None
) -> Manifest

Return a dataset's manifest (latest version if unspecified).

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str \| None`	The version string, or `None` / `"latest"` for the latest.	`None`

Returns:

Type	Description
`Manifest`	The dataset's manifest.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version has no committed manifest.
`TimeFFormatError`	If the manifest is unreadable, invalid, or names a different dataset.

list_datasets ¶source

list_datasets() -> list[DatasetMetadata]

Return the latest-version metadata of every dataset, sorted by id.

Returns:

Name	Type	Description
`One`	`list[DatasetMetadata]`	class:`~timenet.types.DatasetMetadata` per dataset.

open_file ¶source

open_file(
    dataset_id: str, version: str, relpath: str
) -> BinaryIO

Open one file of a dataset version for binary reading.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required
`relpath`	`str`	The file path relative to the version directory.	required

Returns:

Type	Description
`BinaryIO`	An open binary file object.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset version directory does not exist.
`ValueError`	If `relpath` escapes the dataset version directory.

open_release ¶source

open_release(
    reference: DatasetRef,
    *,
    manifest: Manifest | None = None,
) -> DatasetVersion

Open a local release without attaching its payload sources.

This checks that every file exists with its recorded size.

Parameters:

Name	Type	Description	Default
`reference`	`DatasetRef`	The dataset ID and exact version to open.	required
`manifest`	`Manifest \| None`	A manifest already read from this registry, or `None` to fetch it.	`None`

Returns:

Type	Description
`DatasetVersion`	A handle to the release's directory on the local filesystem.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the registry lacks a committed manifest for the release.
`TimeFFormatError`	If the manifest declares a different release.
`TimeNetFileError`	If a file is missing or has an incorrect size.

put_dataset ¶source

put_dataset(
    dataset: TimeFDataset,
    *,
    force: bool = False,
    values_backend: str = "parquet",
    progress_cb: Callable[[WriteProgressEvent], None]
    | None = None,
    pinned_by: Mapping[DatasetRef, PayloadSource]
    | None = None,
) -> DatasetRef

Write a dataset to this local registry.

Return an existing release's reference unless force is set. Reused data must match the source releases stored in this registry.

Parameters:

Name	Type	Description	Default
`dataset`	`TimeFDataset`	The dataset to write.	required
`force`	`bool`	Replace an existing version.	`False`
`values_backend`	`str`	Storage format for values (`"parquet"` or `"zarr"`).	`'parquet'`
`progress_cb`	`Callable[[WriteProgressEvent], None] \| None`	Optional callback for write progress.	`None`
`pinned_by`	`Mapping[DatasetRef, PayloadSource] \| None`	Releases that already read this release's files, with the pins that the new files must match.	`None`

Returns:

Type	Description
`DatasetRef`	A reference to the stored release.

Raises:

Type	Description
`TimeNetRegistryError`	If this registry is read-only.
`TimeNetDatasetNotFoundError`	If a release that stores reused data is missing from this registry.
`TimeFFormatError`	If a source release or the new output fails a file pin check.

put_release ¶source

put_release(from_directory: Path) -> DatasetRef

Copy a release directory into this registry.

The directory must contain manifest.json and every file it lists. This method copies those files before replacing an existing version.

Parameters:

Name	Type	Description	Default
`from_directory`	`Path`	The release directory.	required

Returns:

Type	Description
`DatasetRef`	A reference to the published release.

Raises:

Type	Description
`TimeNetRegistryError`	If this registry is read-only.
`TimeFFormatError`	If `manifest.json` is missing, unreadable, or invalid.
`TimeNetFileError`	If a listed file is missing or its size differs from the manifest.
`OSError`	If copying or renaming fails, with the backup path if restoring the old directory also fails.

read_origin ¶source

read_origin(release: DatasetRef) -> Origin | None

Read the provenance record of a stored release.

Parameters:

Name	Type	Description	Default
`release`	`DatasetRef`	The release.	required

Returns:

Type	Description
`Origin \| None`	The record, or `None` if the release has none.

Raises:

Type	Description
`TimeNetOriginError`	If the record is invalid.

record_origin ¶source

record_origin(
    release: DatasetRef, kind: OriginKind, source: str
) -> None

Record how a stored release reached this registry, with the current time.

The record replaces an earlier one. A URL in source loses its user info, query and fragment, so no credential lands in the record.

Parameters:

Name	Type	Description	Default
`release`	`DatasetRef`	The release.	required
`kind`	`OriginKind`	How the release reached this registry.	required
`source`	`str`	The registry location, or the name and version of the backend package.	required

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If this registry does not hold the release.

release_at ¶source

release_at(path: Path) -> DatasetRef | None

Return the release whose folder holds a path.

Parameters:

Name	Type	Description	Default
`path`	`Path`	A path, whether or not it exists.	required

Returns:

Type	Description
`DatasetRef \| None`	The release, or `None` if no release folder of this registry holds the path.

releases ¶source

releases() -> list[DatasetRef]

List the committed releases below the root.

A release is a <org>/<name>/<version>/manifest.json outside hidden folders and unfinished writes.

Returns:

Type	Description
`list[DatasetRef]`	The releases, sorted by dataset ID and version.

Raises:

Type	Description
`OSError`	If a folder cannot be read.

remove_release ¶source

remove_release(release: DatasetRef) -> int

Delete a release and the dataset folders that this leaves empty.

Parameters:

Name	Type	Description	Default
`release`	`DatasetRef`	The release to delete.	required

Returns:

Type	Description
`int`	The bytes freed.

Raises:

Type	Description
`TimeNetRegistryError`	If the registry is read-only.
`OSError`	If the release folder cannot be removed completely.

require_clean_code_provenance ¶source

require_clean_code_provenance(manifest: Manifest) -> None

Raise TimeNetBuildError if this registry refuses releases built from locally modified code, and manifest records local changes.

Parameters:

Name	Type	Description	Default
`manifest`	`Manifest`	The manifest of the release to publish.	required

Raises:

Type	Description
`TimeNetBuildError`	If the manifest records local changes or no commit for a TimeNet package.

resolve ¶source

resolve(
    dataset_id: str, version: Version | None
) -> ReleaseLookup

Check whether the local registry holds a dataset release.

Local releases are available at any access level.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset ID.	required
`version`	`Version \| None`	An exact version, or `None` for the latest version.	required

Returns:

Type	Description
`ReleaseLookup`	`FoundInRegistry` with the committed version, or `NotFound` if the release is absent.

Raises:

Type	Description
`TimeFValidationError`	If `dataset_id` is invalid.

search ¶source

search(filter: DatasetFilter) -> list[DatasetMetadata]

Find datasets that match a filter.

A task or time_series_spec filter reads the manifest of a dataset only if its card matches.

Parameters:

Name	Type	Description	Default
`filter`	`DatasetFilter`	The filters and the limit.	required

Returns:

Type	Description
`list[DatasetMetadata]`	The matching dataset metadata, sorted by dataset ID, with at most `filter.limit` entries.

RemoteRegistry ¶source

Bases: LazyReadRegistry

A TimeNet registry that reads and publishes datasets over HTTP(S).

has_catalog `class-attribute` ¶source

has_catalog: bool = True

Whether :meth:list_datasets and :meth:search work. Without a catalog, both raise NotImplementedError.

location `property` ¶source

location: str

The registry service's base URL.

read_only `property` ¶source

read_only: bool

Whether this registry refuses every write, set with read_only=True.

requires_clean_code_provenance `property` ¶source

requires_clean_code_provenance: bool

Whether this registry refuses releases built from locally modified TimeNet code.

requires_open_access `property` ¶source

requires_open_access: bool

Whether stored or served releases must declare open access.

check_writable ¶source

check_writable(write: str) -> None

Refuse a write if this registry is read-only.

Parameters:

Name	Type	Description	Default
`write`	`str`	The write to refuse, for the message, such as `"put_dataset()"`.	required

Raises:

Type	Description
`TimeNetRegistryError`	If this registry is read-only.

download_release ¶source

download_release(
    reference: DatasetRef,
    directory: str | Path,
    *,
    force: bool = False,
    manifest: Manifest | None = None,
    progress_cb: ByteCountCallback | None = None,
) -> None

Download one release's files to a local directory.

Parameters:

Name	Type	Description	Default
`reference`	`DatasetRef`	The exact release.	required
`directory`	`str \| Path`	The local release directory, `<root>/<dataset_id>/<version>`.	required
`force`	`bool`	Download again even if a complete copy exists.	`False`
`manifest`	`Manifest \| None`	The release's manifest, or `None` to fetch it from this registry.	`None`
`progress_cb`	`ByteCountCallback \| None`	Callback that receives each downloaded chunk's byte count, or `None`.	`None`

exists ¶source

exists(dataset_id: str, version: str) -> bool

Check whether this registry holds a dataset version.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required

Returns:

Type	Description
`bool`	`True` if the version has a committed manifest, else `False`.

find_manifest ¶source

find_manifest(release: DatasetRef) -> Manifest | None

Return the manifest of a release, or None if this registry does not hold it.

Parameters:

Name	Type	Description	Default
`release`	`DatasetRef`	The exact release.	required

Returns:

Type	Description
`Manifest \| None`	The manifest, or `None`.

Raises:

Type	Description
`TimeFFormatError`	If the stored manifest names another release.

get_manifest ¶source

get_manifest(
    dataset_id: str, version: str | None = None
) -> Manifest

Return a dataset's manifest (latest version if unspecified).

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The `org/name` id.	required
`version`	`str \| None`	The version string, or `None` / `"latest"` for the latest.	`None`

Returns:

Type	Description
`Manifest`	The dataset's manifest.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version is unknown.
`TimeFFormatError`	If the served manifest is not valid.

list_datasets ¶source

list_datasets() -> list[DatasetMetadata]

Return the latest-version metadata of every dataset, sorted by id.

Returns:

Name	Type	Description
`One`	`list[DatasetMetadata]`	class:`~timenet.types.DatasetMetadata` per dataset.

open_file ¶source

open_file(
    dataset_id: str, version: str, relpath: str
) -> BinaryIO

Open one file of a dataset version for binary reading.

Streams the presigned URL into a spooled temporary file, so the returned handle is seekable and self-contained (the caller closes it).

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The `org/name` id.	required
`version`	`str`	The version string.	required
`relpath`	`str`	The version-relative file path.	required

Returns:

Type	Description
`BinaryIO`	An open, seekable binary file positioned at the start.

open_release ¶source

open_release(
    reference: DatasetRef,
    *,
    manifest: Manifest | None = None,
) -> DatasetVersion

Open a remote release without attaching its payload sources.

Parameters:

Name	Type	Description	Default
`reference`	`DatasetRef`	The dataset ID and exact version to open.	required
`manifest`	`Manifest \| None`	A manifest already read from this registry, or `None` to fetch it.	`None`

Returns:

Type	Description
`DatasetVersion`	A handle to the remote files of the release.

Raises:

Type	Description
`TimeFFormatError`	If the manifest declares a different release.

put_dataset ¶source

put_dataset(
    dataset: TimeFDataset,
    *,
    force: bool = False,
    values_backend: str = "parquet",
    progress_cb: Callable[[WriteProgressEvent], None]
    | None = None,
    pinned_by: Mapping[DatasetRef, PayloadSource]
    | None = None,
) -> DatasetRef

Write a dataset locally and publish it to this registry.

If the version already exists, return its reference. A published version never changes.

Parameters:

Name	Type	Description	Default
`dataset`	`TimeFDataset`	The dataset to write.	required
`force`	`bool`	Must be false, because this registry never writes a published version again.	`False`
`values_backend`	`str`	Storage format for values (`"parquet"` or `"zarr"`).	`'parquet'`
`progress_cb`	`Callable[[WriteProgressEvent], None] \| None`	Optional callback for write progress.	`None`
`pinned_by`	`Mapping[DatasetRef, PayloadSource] \| None`	Releases that already read this release's files, with the pins that the new files must match.	`None`

Returns:

Type	Description
`DatasetRef`	A reference to the stored release.

Raises:

Type	Description
`TimeNetRegistryError`	If this registry is read-only.
`TimeFValidationError`	If `force` is set.

put_release ¶source

put_release(from_directory: Path) -> DatasetRef

Publish a release directory to the remote registry.

The directory must contain manifest.json and every file it lists.

Parameters:

Name	Type	Description	Default
`from_directory`	`Path`	The release directory.	required

Returns:

Type	Description
`DatasetRef`	A reference to the published release.

Raises:

Type	Description
`TimeFFormatError`	If `manifest.json` is missing, unreadable, or invalid.
`TimeNetFileError`	If a listed file is missing or its size differs from the manifest.
`TimeNetBuildError`	If the manifest records TimeNet code with local changes.
`TimeNetRegistryError`	If this registry is read-only, or the service skips a listed file or requests a missing file.

require_clean_code_provenance ¶source

require_clean_code_provenance(manifest: Manifest) -> None

Raise TimeNetBuildError if this registry refuses releases built from locally modified code, and manifest records local changes.

Parameters:

Name	Type	Description	Default
`manifest`	`Manifest`	The manifest of the release to publish.	required

Raises:

Type	Description
`TimeNetBuildError`	If the manifest records local changes or no commit for a TimeNet package.

resolve ¶source

resolve(
    dataset_id: str, version: Version | None
) -> ReleaseLookup

Check whether the registry serves a dataset release.

If :attr:requires_open_access is true, the registry serves a release only if its card and each card in its lineage declare open access.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset ID.	required
`version`	`Version \| None`	An exact version, or `None` for the latest version.	required

Returns:

Type	Description
`ReleaseLookup`	`FoundInRegistry` if served, `NotServed` if access is restricted, or `NotFound` if absent.

search ¶source

search(filter: DatasetFilter) -> list[DatasetMetadata]

Find datasets that match a filter.

A task or time_series_spec filter reads the manifest of a dataset only if its card matches.

Parameters:

Name	Type	Description	Default
`filter`	`DatasetFilter`	The filters and the limit.	required

Returns:

Type	Description
`list[DatasetMetadata]`	The matching dataset metadata, sorted by dataset ID, with at most `filter.limit` entries.

S3Registry ¶source

Bases: LazyReadRegistry

A registry that reads and publishes datasets under an S3 bucket or prefix.

has_catalog `class-attribute` ¶source

has_catalog: bool = False

location `property` ¶source

location: str

The registry's root URL as s3://bucket/prefix.

read_only `property` ¶source

read_only: bool

Whether this registry refuses every write, set with read_only=True.

requires_clean_code_provenance `property` ¶source

requires_clean_code_provenance: bool

Whether this registry refuses releases built from locally modified TimeNet code.

requires_open_access `property` ¶source

requires_open_access: bool

Whether stored or served releases must declare open access.

check_writable ¶source

check_writable(write: str) -> None

Refuse a write if this registry is read-only.

Parameters:

Name	Type	Description	Default
`write`	`str`	The write to refuse, for the message, such as `"put_dataset()"`.	required

Raises:

Type	Description
`TimeNetRegistryError`	If this registry is read-only.

download_release ¶source

download_release(
    reference: DatasetRef,
    directory: str | Path,
    *,
    force: bool = False,
    manifest: Manifest | None = None,
    progress_cb: ByteCountCallback | None = None,
) -> None

Download one release's files to a local directory.

Reuse a complete copy unless force is set. Replace directory only after every downloaded file matches its manifest record.

Parameters:

Name	Type	Description	Default
`reference`	`DatasetRef`	The exact release.	required
`directory`	`str \| Path`	The local release directory, `<root>/<dataset_id>/<version>`.	required
`force`	`bool`	Download again even if a complete copy exists.	`False`
`manifest`	`Manifest \| None`	The release's manifest, or `None` to fetch it from this registry.	`None`
`progress_cb`	`ByteCountCallback \| None`	Callback that receives each downloaded file's byte count, or `None`.	`None`

Raises:

Type	Description
`TimeNetFileError`	If a downloaded file's size or checksum differs from its manifest record.

exists ¶source

exists(dataset_id: str, version: str) -> bool

Check whether this registry holds a dataset version.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required

Returns:

Type	Description
`bool`	`True` if the version has a committed manifest, else `False`.

find_manifest ¶source

find_manifest(release: DatasetRef) -> Manifest | None

Return the manifest of a release, or None if this registry does not hold it.

Parameters:

Name	Type	Description	Default
`release`	`DatasetRef`	The exact release.	required

Returns:

Type	Description
`Manifest \| None`	The manifest, or `None`.

Raises:

Type	Description
`TimeFFormatError`	If the stored manifest names another release.

get_manifest ¶source

get_manifest(
    dataset_id: str, version: str | None = None
) -> Manifest

Return a dataset's manifest (latest version if unspecified).

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The `org/name` id.	required
`version`	`str \| None`	The version string, or `None` / `"latest"` for the latest.	`None`

Returns:

Type	Description
`Manifest`	The dataset's manifest.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version has no manifest in the bucket.
`TimeFFormatError`	If the stored manifest is not valid.

list_datasets ¶source

list_datasets() -> list[DatasetMetadata]

Unsupported: the S3 backend has no catalog.

Raises:

Type	Description
`NotImplementedError`	Always.

open_file ¶source

open_file(
    dataset_id: str, version: str, relpath: str
) -> BinaryIO

Open one file of a dataset version for binary reading.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The `org/name` id.	required
`version`	`str`	The version string.	required
`relpath`	`str`	The version-relative file path.	required

Returns:

Type	Description
`BinaryIO`	An open, streaming binary file object.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the object does not exist.

open_release ¶source

open_release(
    reference: DatasetRef,
    *,
    manifest: Manifest | None = None,
) -> DatasetVersion

Open a remote release without attaching its payload sources.

Parameters:

Name	Type	Description	Default
`reference`	`DatasetRef`	The dataset ID and exact version to open.	required
`manifest`	`Manifest \| None`	A manifest already read from this registry, or `None` to fetch it.	`None`

Returns:

Type	Description
`DatasetVersion`	A handle to the remote files of the release.

Raises:

Type	Description
`TimeFFormatError`	If the manifest declares a different release.

put_dataset ¶source

put_dataset(
    dataset: TimeFDataset,
    *,
    force: bool = False,
    values_backend: str = "parquet",
    progress_cb: Callable[[WriteProgressEvent], None]
    | None = None,
    pinned_by: Mapping[DatasetRef, PayloadSource]
    | None = None,
) -> DatasetRef

Write a dataset locally and publish it to this registry.

If the version already exists, return its reference. A published version never changes.

Parameters:

Name	Type	Description	Default
`dataset`	`TimeFDataset`	The dataset to write.	required
`force`	`bool`	Must be false, because this registry never writes a published version again.	`False`
`values_backend`	`str`	Storage format for values (`"parquet"` or `"zarr"`).	`'parquet'`
`progress_cb`	`Callable[[WriteProgressEvent], None] \| None`	Optional callback for write progress.	`None`
`pinned_by`	`Mapping[DatasetRef, PayloadSource] \| None`	Releases that already read this release's files, with the pins that the new files must match.	`None`

Returns:

Type	Description
`DatasetRef`	A reference to the stored release.

Raises:

Type	Description
`TimeNetRegistryError`	If this registry is read-only.
`TimeFValidationError`	If `force` is set.

put_release ¶source

put_release(from_directory: Path) -> DatasetRef

Upload a release directory to S3.

The directory must contain manifest.json and every file it lists. This method uploads the listed files and publishes manifest.json last.

Parameters:

Name	Type	Description	Default
`from_directory`	`Path`	The release directory.	required

Returns:

Type	Description
`DatasetRef`	A reference to the published release.

Raises:

Type	Description
`TimeNetRegistryError`	If this registry is read-only.
`TimeFFormatError`	If `manifest.json` is missing, unreadable, or invalid.
`TimeNetFileError`	If a listed file is missing or its size differs from the manifest.
`TimeNetBuildError`	If the manifest records TimeNet code with local changes.

require_clean_code_provenance ¶source

require_clean_code_provenance(manifest: Manifest) -> None

Raise TimeNetBuildError if this registry refuses releases built from locally modified code, and manifest records local changes.

Parameters:

Name	Type	Description	Default
`manifest`	`Manifest`	The manifest of the release to publish.	required

Raises:

Type	Description
`TimeNetBuildError`	If the manifest records local changes or no commit for a TimeNet package.

resolve ¶source

resolve(
    dataset_id: str, version: Version | None
) -> ReleaseLookup

Check whether the registry serves a dataset release.

If :attr:requires_open_access is true, the registry serves a release only if its card and each card in its lineage declare open access.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset ID.	required
`version`	`Version \| None`	An exact version, or `None` for the latest version.	required

Returns:

Type	Description
`ReleaseLookup`	`FoundInRegistry` if served, `NotServed` if access is restricted, or `NotFound` if absent.

search ¶source

search(filter: DatasetFilter) -> list[DatasetMetadata]

Unsupported: the S3 backend has no catalog.

Raises:

Type	Description
`NotImplementedError`	Always.

WritableRegistry ¶source

Bases: BaseRegistry, ABC

A registry that can also publish datasets and release directories.

has_catalog `class-attribute` ¶source

has_catalog: bool = True

Whether :meth:list_datasets and :meth:search work. Without a catalog, both raise NotImplementedError.

location `abstractmethod` `property` ¶source

location: str

The registry's URL, path, or name.

read_only `property` ¶source

read_only: bool

Whether this registry refuses every write, set with read_only=True.

requires_clean_code_provenance `property` ¶source

requires_clean_code_provenance: bool

Whether this registry refuses releases built from locally modified TimeNet code.

requires_open_access `property` ¶source

requires_open_access: bool

Whether stored or served releases must declare open access.

check_writable ¶source

check_writable(write: str) -> None

Refuse a write if this registry is read-only.

Parameters:

Name	Type	Description	Default
`write`	`str`	The write to refuse, for the message, such as `"put_dataset()"`.	required

Raises:

Type	Description
`TimeNetRegistryError`	If this registry is read-only.

download_release ¶source

download_release(
    reference: DatasetRef,
    directory: str | Path,
    *,
    force: bool = False,
    manifest: Manifest | None = None,
    progress_cb: ByteCountCallback | None = None,
) -> None

Download one release's files to a local directory.

Reuse a complete copy unless force is set. Replace directory only after every downloaded file matches its manifest record.

Parameters:

Name	Type	Description	Default
`reference`	`DatasetRef`	The exact release.	required
`directory`	`str \| Path`	The local release directory, `<root>/<dataset_id>/<version>`.	required
`force`	`bool`	Download again even if a complete copy exists.	`False`
`manifest`	`Manifest \| None`	The release's manifest, or `None` to fetch it from this registry.	`None`
`progress_cb`	`ByteCountCallback \| None`	Callback that receives each downloaded file's byte count, or `None`.	`None`

Raises:

Type	Description
`TimeNetFileError`	If a downloaded file's size or checksum differs from its manifest record.

exists ¶source

exists(dataset_id: str, version: str) -> bool

Check whether this registry holds a dataset version.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required

Returns:

Type	Description
`bool`	`True` if the version has a committed manifest, else `False`.

find_manifest ¶source

find_manifest(release: DatasetRef) -> Manifest | None

Return the manifest of a release, or None if this registry does not hold it.

Parameters:

Name	Type	Description	Default
`release`	`DatasetRef`	The exact release.	required

Returns:

Type	Description
`Manifest \| None`	The manifest, or `None`.

Raises:

Type	Description
`TimeFFormatError`	If the stored manifest names another release.

get_manifest `abstractmethod` ¶source

get_manifest(
    dataset_id: str, version: str | None = None
) -> Manifest

Return the manifest of a dataset.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str \| None`	The version string, or `None` for the latest version.	`None`

Returns:

Name	Type	Description
`The`	`Manifest`	class:`~timenet.manifest.Manifest` of the dataset.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version is unknown.

list_datasets `abstractmethod` ¶source

list_datasets() -> list[DatasetMetadata]

Return the metadata of every dataset, using the latest version. The result is sorted by dataset id.

Returns:

Name	Type	Description
`One`	`list[DatasetMetadata]`	class:`~timenet.types.DatasetMetadata` object for each dataset.

open_file `abstractmethod` ¶source

open_file(
    dataset_id: str, version: str, relpath: str
) -> BinaryIO

Open one file of a dataset version for binary reading.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required
`relpath`	`str`	The file path relative to the version directory.	required

Returns:

Type	Description
`BinaryIO`	An open binary file object.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version is unknown.

open_release `abstractmethod` ¶source

open_release(
    reference: DatasetRef,
    *,
    manifest: Manifest | None = None,
) -> DatasetVersion

Open an exact release without attaching its payload sources.

Opening reads metadata only. Attach payload sources with :func:~timenet.registry.dependencies.attach_payload_sources.

Parameters:

Name	Type	Description	Default
`reference`	`DatasetRef`	The dataset ID and exact version to open.	required
`manifest`	`Manifest \| None`	A manifest already read from this registry, or `None` to fetch it.	`None`

Returns:

Type	Description
`DatasetVersion`	A handle to the release's manifest and files.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the registry lacks the release.
`TimeFFormatError`	If the manifest declares a different release.
`TimeNetFileError`	If a local file is missing or has an incorrect size.

put_dataset `abstractmethod` ¶source

put_dataset(
    dataset: TimeFDataset,
    *,
    force: bool = False,
    values_backend: str = "parquet",
    progress_cb: Callable[[WriteProgressEvent], None]
    | None = None,
    pinned_by: Mapping[DatasetRef, PayloadSource]
    | None = None,
) -> DatasetRef

Write a dataset to this registry.

This method derives a schema if the dataset has none. It returns an existing version's reference unless force is set.

Parameters:

Name	Type	Description	Default
`dataset`	`TimeFDataset`	The dataset to write.	required
`force`	`bool`	Replace an existing version.	`False`
`values_backend`	`str`	Storage format for values (`"parquet"` or `"zarr"`).	`'parquet'`
`progress_cb`	`Callable[[WriteProgressEvent], None] \| None`	Optional callback for write progress.	`None`
`pinned_by`	`Mapping[DatasetRef, PayloadSource] \| None`	Releases that already read this release's files, with the pins that the new files must match.	`None`

Returns:

Type	Description
`DatasetRef`	A reference to the stored release.

put_release `abstractmethod` ¶source

put_release(from_directory: Path) -> DatasetRef

Publish an existing release to this registry.

The directory must contain manifest.json and every file it lists. The method publishes only those files and leaves the directory unchanged.

Parameters:

Name	Type	Description	Default
`from_directory`	`Path`	The release directory.	required

Returns:

Type	Description
`DatasetRef`	A reference to the published release.

Raises:

Type	Description
`TimeFFormatError`	If `manifest.json` is missing, unreadable, or invalid.
`TimeNetFileError`	If a listed file is missing or its size differs from the manifest.
`TimeNetBuildError`	If this registry refuses releases built from locally modified code, and the manifest records local changes.

require_clean_code_provenance ¶source

require_clean_code_provenance(manifest: Manifest) -> None

Raise TimeNetBuildError if this registry refuses releases built from locally modified code, and manifest records local changes.

Parameters:

Name	Type	Description	Default
`manifest`	`Manifest`	The manifest of the release to publish.	required

Raises:

Type	Description
`TimeNetBuildError`	If the manifest records local changes or no commit for a TimeNet package.

resolve ¶source

resolve(
    dataset_id: str, version: Version | None
) -> ReleaseLookup

Check whether the registry serves a dataset release.

If :attr:requires_open_access is true, the registry serves a release only if its card and each card in its lineage declare open access.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset ID.	required
`version`	`Version \| None`	An exact version, or `None` for the latest version.	required

Returns:

Type	Description
`ReleaseLookup`	`FoundInRegistry` if served, `NotServed` if access is restricted, or `NotFound` if absent.

search ¶source

search(filter: DatasetFilter) -> list[DatasetMetadata]

Find datasets that match a filter.

A task or time_series_spec filter reads the manifest of a dataset only if its card matches.

Parameters:

Name	Type	Description	Default
`filter`	`DatasetFilter`	The filters and the limit.	required

Returns:

Type	Description
`list[DatasetMetadata]`	The matching dataset metadata, sorted by dataset ID, with at most `filter.limit` entries.

copy_release ¶source

copy_release(
    reference: DatasetRef,
    source: BaseRegistry,
    dest: WritableRegistry,
    *,
    with_payload_sources: bool = True,
    force: bool = False,
    progress: ProgressCallback | None = None,
) -> tuple[DatasetRef, ...]

Copy a release and its payload sources to another registry.

Copy payload sources first, reuse complete copies, and repair damaged local copies. Check all releases before copying any files.

Parameters:

Name	Type	Description	Default
`reference`	`DatasetRef`	The exact release to copy.	required
`source`	`BaseRegistry`	The registry to copy from.	required
`dest`	`WritableRegistry`	The registry to copy to.	required
`with_payload_sources`	`bool`	Also copy the payload sources that the manifest of the release lists.	`True`
`force`	`bool`	Fetch `reference` again even if the local destination has a complete copy.	`False`
`progress`	`ProgressCallback \| None`	Receives step events for each copy and byte counts for local destinations.	`None`

Returns:

Type	Description
`tuple[DatasetRef, ...]`	The copied or repaired releases, in copy order.

Raises:

Type	Description
`TimeNetRegistryError`	If `dest` is read-only.
`TimeFValidationError`	If `force` is set and `dest` is not local.
`TimeNetVersionConflictError`	If `dest` holds the same release with other files or another card.
`TimeNetAccessError`	If a release lacks open access and either registry requires it.
`TimeNetBuildError`	If `dest` is not local and a copied release records TimeNet code with local changes.
`TimeNetDatasetNotFoundError`	If `source` lacks the release.
`TimeNetMissingParentError`	If neither registry holds a complete payload source.
`TimeFFormatError`	If a manifest names the wrong release or a pin fails.

local_registry_path ¶source

local_registry_path(
    uri: str | Path, *, label: str = "registry"
) -> Path

Resolve a local registry address to a directory path.

Parameters:

Name	Type	Description	Default
`uri`	`str \| Path`	A `file://` URI or local path.	required
`label`	`str`	The name of the value in an error message, such as `sources[0]`.	`'registry'`

Returns:

Type	Description
`Path`	The local directory path with `~` expanded.

Raises:

Type	Description
`TimeNetRegistryError`	If the value is blank, remote or an invalid local registry URI.

open_local_version ¶source

open_local_version(
    path: str | Path,
    *,
    sources_root: str | Path | None = None,
) -> DatasetVersion

Open a local release directory and its payload sources.

Opening reads only metadata. The manifest must be valid, with a supported format version, and every file it lists must be present with its recorded size. Each payload source must be found under the sources root, match the files the release pins, and hold each of them with its recorded size.

Parameters:

Name	Type	Description	Default
`path`	`str \| Path`	The release directory.	required
`sources_root`	`str \| Path \| None`	The registry root to find payload sources under. By default this is the root that holds `path` as `<root>/<org>/<name>/<version>`, so a release and its sources can move together. A release without payload sources opens from any directory.	`None`

Returns:

Type	Description
`DatasetVersion`	A handle with each payload source opened. No catalog or values file is opened.

Raises:

Type	Description
`FileNotFoundError`	If `path` has no manifest.
`TimeFValidationError`	If the release has payload sources, is not stored as `<root>/<org>/<name>/<version>`, and no `sources_root` is given.

open_registry ¶source

open_registry(
    uri: str | Path, *, label: str = "registry"
) -> WritableRegistry

Open a registry from a URI or path.

The scheme selects the backend. http(s):// and timenet:// open a :class:RemoteRegistry, and timenet:// is an alias for the hosted :data:TIMENET_REGISTRY_URL. s3:// opens an :class:S3Registry. file:// or a plain path opens a :class:LocalRegistry.

Parameters:

Name	Type	Description	Default
`uri`	`str \| Path`	A URL, `timenet://` / `s3://` / `file://` URI, or local path.	required
`label`	`str`	The name of the value in an error message, such as `sources[0]`.	`'registry'`

Returns:

Type	Description
`WritableRegistry`	The matching registry backend.

Raises:

Type	Description
`TimeNetRegistryError`	If `uri` is blank or carries a scheme no backend handles, if a `timenet://` URI carries a path, or if `uri` is a `file://` URI with a host component (the host would be dropped without notice) or without a path.

Timenet.registry