Registry

timenet.registry ¶source

Dataset registries: serve compiled TimeF versions and search over metadata.

TIMENET_REGISTRY_URL `module-attribute` ¶source

TIMENET_REGISTRY_URL = 'https://registry.timenet.ai'

The hosted TimeNet registry that the timenet:// scheme is an alias for.

BaseRegistry ¶source

Bases: ABC

A source of TimeF datasets: their manifests, files, and searchable metadata.

local_root `abstractmethod` `property` ¶source

local_root: Path

The local registry or cache directory that holds prepared releases.

download_version ¶source

download_version(
    dataset_id: str,
    version: str,
    dest_dir: str | Path,
    *,
    force: bool = False,
    manifest: Manifest | None = None,
    progress_cb: ProgressCallback | None = None,
) -> None

Download complete local copies of a release and of each of its payload sources.

Only the release's own payload sources are downloaded: it lists every release whose files it reads. Each is checked against the files the release pins.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset ID.	required
`version`	`str`	The exact release version.	required
`dest_dir`	`str \| Path`	The local `<registry>/<org>/<name>/<version>` directory.	required
`force`	`bool`	Download the requested release again.	`False`
`manifest`	`Manifest \| None`	An already-loaded manifest for the requested release.	`None`
`progress_cb`	`ProgressCallback \| None`	Report transferred bytes.	`None`

Raises:

Type	Description
`TimeFValidationError`	If the destination does not identify the requested release.

get_manifest `abstractmethod` ¶source

get_manifest(
    dataset_id: str, version: str | None = None
) -> Manifest

Return the manifest of a dataset.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str \| None`	The version string, or `None` for the latest version.	`None`

Returns:

Name	Type	Description
`The`	`Manifest`	class:`~timenet.manifest.Manifest` of the dataset.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version is unknown.

list_datasets `abstractmethod` ¶source

list_datasets() -> list[DatasetMetadata]

Return the metadata of every dataset, using the latest version. The result is sorted by dataset id.

Returns:

Name	Type	Description
`One`	`list[DatasetMetadata]`	class:`~timenet.types.DatasetMetadata` object for each dataset.

open_file `abstractmethod` ¶source

open_file(
    dataset_id: str, version: str, relpath: str
) -> BinaryIO

Open one file of a dataset version for binary reading.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required
`relpath`	`str`	The file path relative to the version directory.	required

Returns:

Type	Description
`BinaryIO`	An open binary file object.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version is unknown.

open_version `abstractmethod` ¶source

open_version(
    dataset_id: str, version: str | None = None
) -> DatasetVersion

Open a committed dataset version as a random-access handle.

The returned :class:~timenet.registry.version.DatasetVersion object bundles the parsed manifest with a handle to the files of the version. The handle is rooted in a filesystem. A reader built from this object does not re-open the registry. It also does not re-read manifest.json. The filesystem of the handle MUST serve range reads. The reader uses it to read Parquet footers and value slices out of order.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str \| None`	The version string, or `None` for the latest version.	`None`

Returns:

Type	Description
`DatasetVersion`	A handle to the manifest and files of the committed version.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version is unknown.

search ¶source

search(
    *,
    query: str | list[str] | None = None,
    domain: Domain | list[Domain] | None = None,
    task: type[Task] | list[type[Task]] | None = None,
    license: License | list[License] | None = None,
    time_series_spec: str | list[str] | None = None,
    dataset_id: str | list[str] | None = None,
    tag: str | list[str] | None = None,
    limit: int = 100,
) -> list[DatasetMetadata]

Filter datasets by any combination of criteria.

Each filter accepts a single value or a list of values. The method ignores filters set to None. It combines all other filters with AND logic. The filters query, domain, task, license, and dataset_id match a dataset when any of their values match. The filters time_series_spec and tag match a dataset only when all of their values match. The type filters task and time_series_spec read the manifest schema of each dataset.

Parameters:

Name	Type	Description	Default
`query`	`str \| list[str] \| None`	Free-text terms. The method matches each term as a case-insensitive substring in the dataset name, description, or tags.	`None`
`domain`	`Domain \| list[Domain] \| None`	Keep datasets that share any of these domains.	`None`
`task`	`type[Task] \| list[type[Task]] \| None`	Keep datasets whose schema includes any of these task classes.	`None`
`license`	`License \| list[License] \| None`	Keep datasets that have any of these licenses.	`None`
`time_series_spec`	`str \| list[str] \| None`	Keep datasets that declare all of these `spec_type` values.	`None`
`dataset_id`	`str \| list[str] \| None`	Keep only these ids.	`None`
`tag`	`str \| list[str] \| None`	Keep datasets that declare all of these tags.	`None`
`limit`	`int`	The maximum number of results. The default is 100.	`100`

Returns:

Type	Description
`list[DatasetMetadata]`	The matching dataset metadata. The list has at most `limit` entries.

Raises:

Type	Description
`ValueError`	If `limit` is negative.

DatasetVersion `dataclass` ¶source

An opened release: its manifest, filesystem, and its already-opened payload sources.

filesystem `instance-attribute` ¶source

filesystem: FileSystem

manifest `instance-attribute` ¶source

manifest: Manifest

payload_sources `class-attribute` `instance-attribute` ¶source

payload_sources: dict[SourceKey, DatasetVersion] = field(
    default_factory=dict, compare=False, repr=False
)

The opened release behind each key of manifest.payload_sources. Each was checked against its pins when this release was opened, and its own payload sources are not opened.

root `instance-attribute` ¶source

root: str

fsspec_mapper ¶source

fsspec_mapper(relpath: str) -> FSMap

Return an fsspec mapping for a version-relative directory.

The mapping uses the prepared filesystem, including its credentials and caching. Consumers choose the store's read-only policy when opening it.

resolve_file_path ¶source

resolve_file_path(relative_path: str) -> str

Return a file's filesystem path without checking or downloading the file.

Parameters:

Name	Type	Description	Default
`relative_path`	`str`	A path within this version, such as `time_series/part-000.parquet`. Absolute paths and paths that escape the version directory are rejected.	required

Returns:

Type	Description
`str`	The path to use with :attr:`filesystem`.

resolve_payload_source ¶source

resolve_payload_source(
    source_key: SourceKey | None,
) -> DatasetVersion

Return the opened release that stores a payload file.

Parameters:

Name	Type	Description	Default
`source_key`	`SourceKey \| None`	The `source_key` of a catalog row, a key of `manifest.payload_sources`. `None` means this release stores the file.	required

Returns:

Type	Description
`DatasetVersion`	This release or one of its already-opened payload sources. No files are opened here.

Raises:

Type	Description
`TimeFFormatError`	If the catalog names a payload source the manifest does not list.

LocalRegistry ¶source

Bases: WritableRegistry

Serve datasets from a local directory. The build output is itself a valid registry.

local_root `property` ¶source

local_root: Path

The directory that holds complete local releases.

root `property` ¶source

root: Path

The directory this registry reads from and writes to.

download_version ¶source

download_version(
    dataset_id: str,
    version: str,
    dest_dir: str | Path,
    *,
    force: bool = False,
    manifest: Manifest | None = None,
    progress_cb: ProgressCallback | None = None,
) -> None

Download complete local copies of a release and of each of its payload sources.

Only the release's own payload sources are downloaded: it lists every release whose files it reads. Each is checked against the files the release pins.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset ID.	required
`version`	`str`	The exact release version.	required
`dest_dir`	`str \| Path`	The local `<registry>/<org>/<name>/<version>` directory.	required
`force`	`bool`	Download the requested release again.	`False`
`manifest`	`Manifest \| None`	An already-loaded manifest for the requested release.	`None`
`progress_cb`	`ProgressCallback \| None`	Report transferred bytes.	`None`

Raises:

Type	Description
`TimeFValidationError`	If the destination does not identify the requested release.

exists ¶source

exists(dataset_id: str, version: str) -> bool

Return whether a committed version already exists in this registry.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required

Returns:

Type	Description
`bool`	`True` if the version has a committed manifest, else `False`.

get_manifest ¶source

get_manifest(
    dataset_id: str, version: str | None = None
) -> Manifest

Return a dataset's manifest (latest version if unspecified).

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str \| None`	The version string, or `None` / `"latest"` for the latest.	`None`

Returns:

Type	Description
`Manifest`	The dataset's manifest.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version has no committed manifest.
`TimeFFormatError`	If the stored manifest declares a dataset id that differs from its directory. This means a misplaced or corrupt artifact.

list_datasets ¶source

list_datasets() -> list[DatasetMetadata]

Return the latest-version metadata of every dataset, sorted by id.

Search at any depth. This finds both flat (hello_world) and namespaced (org/name) layouts. A dataset id is the path from the root to a version directory's parent.

Returns:

Name	Type	Description
`One`	`list[DatasetMetadata]`	class:`~timenet.types.DatasetMetadata` per dataset.

open_file ¶source

open_file(
    dataset_id: str, version: str, relpath: str
) -> BinaryIO

Open one file of a dataset version for binary reading.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required
`relpath`	`str`	The file path relative to the version directory.	required

Returns:

Type	Description
`BinaryIO`	An open binary file object.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset version directory does not exist.
`ValueError`	If `relpath` escapes the dataset version directory.

open_version ¶source

open_version(
    dataset_id: str, version: str | None = None
) -> DatasetVersion

Open a committed version as a local, random-access handle.

Reuses the manifest :meth:get_manifest already parsed and validated, so a missing version or a misplaced artifact surfaces there. Roots the handle at the version's directory over a :class:pyarrow.fs.LocalFileSystem, which is zero network and already seekable. Opening reads only metadata, as :func:~timenet.registry.open_local_version does, and finds payload sources in this registry.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str \| None`	The version string, or `None` for the latest.	`None`

Returns:

Type	Description
`DatasetVersion`	A handle to the committed version's manifest and files.

search ¶source

search(
    *,
    query: str | list[str] | None = None,
    domain: Domain | list[Domain] | None = None,
    task: type[Task] | list[type[Task]] | None = None,
    license: License | list[License] | None = None,
    time_series_spec: str | list[str] | None = None,
    dataset_id: str | list[str] | None = None,
    tag: str | list[str] | None = None,
    limit: int = 100,
) -> list[DatasetMetadata]

Filter datasets by any combination of criteria.

Each filter accepts a single value or a list of values. The method ignores filters set to None. It combines all other filters with AND logic. The filters query, domain, task, license, and dataset_id match a dataset when any of their values match. The filters time_series_spec and tag match a dataset only when all of their values match. The type filters task and time_series_spec read the manifest schema of each dataset.

Parameters:

Name	Type	Description	Default
`query`	`str \| list[str] \| None`	Free-text terms. The method matches each term as a case-insensitive substring in the dataset name, description, or tags.	`None`
`domain`	`Domain \| list[Domain] \| None`	Keep datasets that share any of these domains.	`None`
`task`	`type[Task] \| list[type[Task]] \| None`	Keep datasets whose schema includes any of these task classes.	`None`
`license`	`License \| list[License] \| None`	Keep datasets that have any of these licenses.	`None`
`time_series_spec`	`str \| list[str] \| None`	Keep datasets that declare all of these `spec_type` values.	`None`
`dataset_id`	`str \| list[str] \| None`	Keep only these ids.	`None`
`tag`	`str \| list[str] \| None`	Keep datasets that declare all of these tags.	`None`
`limit`	`int`	The maximum number of results. The default is 100.	`100`

Returns:

Type	Description
`list[DatasetMetadata]`	The matching dataset metadata. The list has at most `limit` entries.

Raises:

Type	Description
`ValueError`	If `limit` is negative.

store ¶source

store(
    dataset: TimeFDataset,
    *,
    force: bool = False,
    values_backend: str = "parquet",
    progress_cb: Callable[[WriteProgressEvent], None]
    | None = None,
    pinned_by: Mapping[DatasetRef, PayloadSource]
    | None = None,
) -> str

Compile a dataset and write it into this registry's directory tree.

Stream the dataset through a :class:~timenet.writer.TimeFWriter. The writer stages under <version>.tmp-* and publishes with a single atomic rename. It skips an already-committed version unless the caller sets force, which replaces it only once the new release is ready, so a failed write keeps the old one. The writer checks every payload source of the release against this registry, since its readers find them here.

Parameters:

Name	Type	Description	Default
`dataset`	`TimeFDataset`	The populated dataset to store.	required
`force`	`bool`	Replace an already-committed version instead of skipping it.	`False`
`values_backend`	`str`	Storage backend for the values plane (`"parquet"` or `"zarr"`).	`'parquet'`
`progress_cb`	`Callable[[WriteProgressEvent], None] \| None`	Optional writer progress callback.	`None`
`pinned_by`	`Mapping[DatasetRef, PayloadSource] \| None`	The releases that already read files of this release, with their pins. The writer publishes only if it reproduced each pinned file.	`None`

Returns:

Type	Description
`str`	The stored version string.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset reuses payloads stored by a release that this registry does not hold.
`TimeFFormatError`	If this registry holds such a release in another form, or a release in `pinned_by` pins a file that the write did not reproduce.

RemoteRegistry ¶source

Bases: LazyReadRegistry

Serves datasets from the hosted TimeNet registry over HTTP(S).

local_root `property` ¶source

local_root: Path

The directory that holds complete local releases.

download_version ¶source

download_version(
    dataset_id: str,
    version: str,
    dest_dir: str | Path,
    *,
    force: bool = False,
    manifest: Manifest | None = None,
    progress_cb: ProgressCallback | None = None,
) -> None

Download complete local copies of a release and of each of its payload sources.

Only the release's own payload sources are downloaded: it lists every release whose files it reads. Each is checked against the files the release pins.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset ID.	required
`version`	`str`	The exact release version.	required
`dest_dir`	`str \| Path`	The local `<registry>/<org>/<name>/<version>` directory.	required
`force`	`bool`	Download the requested release again.	`False`
`manifest`	`Manifest \| None`	An already-loaded manifest for the requested release.	`None`
`progress_cb`	`ProgressCallback \| None`	Report transferred bytes.	`None`

Raises:

Type	Description
`TimeFValidationError`	If the destination does not identify the requested release.

exists ¶source

exists(dataset_id: str, version: str) -> bool

Return whether a committed version already exists in this registry.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required

Returns:

Type	Description
`bool`	`True` if the version has a committed manifest, else `False`.

get_manifest ¶source

get_manifest(
    dataset_id: str, version: str | None = None
) -> Manifest

Return a dataset's manifest (latest version if unspecified).

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The `org/name` id.	required
`version`	`str \| None`	The version string, or `None` / `"latest"` for the latest.	`None`

Returns:

Type	Description
`Manifest`	The dataset's manifest.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version is unknown.

list_datasets ¶source

list_datasets() -> list[DatasetMetadata]

Return the latest-version metadata of every dataset, sorted by id.

Returns:

Name	Type	Description
`One`	`list[DatasetMetadata]`	class:`~timenet.types.DatasetMetadata` per dataset.

open_file ¶source

open_file(
    dataset_id: str, version: str, relpath: str
) -> BinaryIO

Open one file of a dataset version for binary reading.

Streams the presigned URL into a spooled temporary file, so the returned handle is seekable and self-contained (the caller closes it).

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The `org/name` id.	required
`version`	`str`	The version string.	required
`relpath`	`str`	The version-relative file path.	required

Returns:

Type	Description
`BinaryIO`	An open, seekable binary file positioned at the start.

open_version ¶source

open_version(
    dataset_id: str, version: str | None = None
) -> DatasetVersion

Open a release and its payload sources, then read remote files on demand.

Opening reads only manifests. Each payload source must be in this registry and match the files the release pins.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset ID.	required
`version`	`str \| None`	The release version, or `None` for the latest.	`None`

Returns:

Type	Description
`DatasetVersion`	A handle that prefers complete cached releases and otherwise uses remote range reads.

search ¶source

search(
    *,
    query: str | list[str] | None = None,
    domain: Domain | list[Domain] | None = None,
    task: type[Task] | list[type[Task]] | None = None,
    license: License | list[License] | None = None,
    time_series_spec: str | list[str] | None = None,
    dataset_id: str | list[str] | None = None,
    tag: str | list[str] | None = None,
    limit: int = 100,
) -> list[DatasetMetadata]

Filter datasets by any combination of criteria.

Each filter accepts a single value or a list of values. The method ignores filters set to None. It combines all other filters with AND logic. The filters query, domain, task, license, and dataset_id match a dataset when any of their values match. The filters time_series_spec and tag match a dataset only when all of their values match. The type filters task and time_series_spec read the manifest schema of each dataset.

Parameters:

Name	Type	Description	Default
`query`	`str \| list[str] \| None`	Free-text terms. The method matches each term as a case-insensitive substring in the dataset name, description, or tags.	`None`
`domain`	`Domain \| list[Domain] \| None`	Keep datasets that share any of these domains.	`None`
`task`	`type[Task] \| list[type[Task]] \| None`	Keep datasets whose schema includes any of these task classes.	`None`
`license`	`License \| list[License] \| None`	Keep datasets that have any of these licenses.	`None`
`time_series_spec`	`str \| list[str] \| None`	Keep datasets that declare all of these `spec_type` values.	`None`
`dataset_id`	`str \| list[str] \| None`	Keep only these ids.	`None`
`tag`	`str \| list[str] \| None`	Keep datasets that declare all of these tags.	`None`
`limit`	`int`	The maximum number of results. The default is 100.	`100`

Returns:

Type	Description
`list[DatasetMetadata]`	The matching dataset metadata. The list has at most `limit` entries.

Raises:

Type	Description
`ValueError`	If `limit` is negative.

store ¶source

store(
    dataset: TimeFDataset,
    *,
    force: bool = False,
    values_backend: str = "parquet",
    progress_cb: Callable[[WriteProgressEvent], None]
    | None = None,
    pinned_by: Mapping[DatasetRef, PayloadSource]
    | None = None,
) -> str

Compile a dataset locally and publish it to the remote registry.

Compiles to a temporary directory, then runs the service publish flow: POST the manifest, PUT each declared file to its presigned URL, and finalize. An already-committed version is skipped unless force.

Parameters:

Name	Type	Description	Default
`dataset`	`TimeFDataset`	The populated dataset to store.	required
`force`	`bool`	Publish even if the version is already committed.	`False`
`values_backend`	`str`	Storage backend for the values plane (`"parquet"` or `"zarr"`).	`'parquet'`
`progress_cb`	`Callable[[WriteProgressEvent], None] \| None`	Optional writer progress callback.	`None`
`pinned_by`	`Mapping[DatasetRef, PayloadSource] \| None`	The releases that already read files of this release, with their pins. The writer publishes only if it reproduced each pinned file.	`None`

Returns:

Type	Description
`str`	The stored version string.

Raises:

Type	Description
`TimeNetRegistryError`	If the service requests a file that was not produced locally.

S3Registry ¶source

Bases: LazyReadRegistry

Serves and publishes datasets under an s3://bucket/prefix root.

local_root `property` ¶source

local_root: Path

The directory that holds complete local releases.

download_version ¶source

download_version(
    dataset_id: str,
    version: str,
    dest_dir: str | Path,
    *,
    force: bool = False,
    manifest: Manifest | None = None,
    progress_cb: ProgressCallback | None = None,
) -> None

Download complete local copies of a release and of each of its payload sources.

Only the release's own payload sources are downloaded: it lists every release whose files it reads. Each is checked against the files the release pins.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset ID.	required
`version`	`str`	The exact release version.	required
`dest_dir`	`str \| Path`	The local `<registry>/<org>/<name>/<version>` directory.	required
`force`	`bool`	Download the requested release again.	`False`
`manifest`	`Manifest \| None`	An already-loaded manifest for the requested release.	`None`
`progress_cb`	`ProgressCallback \| None`	Report transferred bytes.	`None`

Raises:

Type	Description
`TimeFValidationError`	If the destination does not identify the requested release.

exists ¶source

exists(dataset_id: str, version: str) -> bool

Return whether a committed version already exists in this registry.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required

Returns:

Type	Description
`bool`	`True` if the version has a committed manifest, else `False`.

get_manifest ¶source

get_manifest(
    dataset_id: str, version: str | None = None
) -> Manifest

Return a dataset's manifest (latest version if unspecified).

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The `org/name` id.	required
`version`	`str \| None`	The version string, or `None` / `"latest"` for the latest.	`None`

Returns:

Type	Description
`Manifest`	The dataset's manifest.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version has no manifest in the bucket.

list_datasets ¶source

list_datasets() -> list[DatasetMetadata]

Unsupported: the S3 backend has no catalog.

Raises:

Type	Description
`NotImplementedError`	Always.

open_file ¶source

open_file(
    dataset_id: str, version: str, relpath: str
) -> BinaryIO

Open one file of a dataset version for binary reading.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The `org/name` id.	required
`version`	`str`	The version string.	required
`relpath`	`str`	The version-relative file path.	required

Returns:

Type	Description
`BinaryIO`	An open, streaming binary file object.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the object does not exist.

open_version ¶source

open_version(
    dataset_id: str, version: str | None = None
) -> DatasetVersion

Open a release and its payload sources, then read remote files on demand.

Opening reads only manifests. Each payload source must be in this registry and match the files the release pins.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset ID.	required
`version`	`str \| None`	The release version, or `None` for the latest.	`None`

Returns:

Type	Description
`DatasetVersion`	A handle that prefers complete cached releases and otherwise uses remote range reads.

search ¶source

search(**_kwargs: Any) -> list[DatasetMetadata]

Unsupported: the S3 backend has no catalog.

Raises:

Type	Description
`NotImplementedError`	Always.

store ¶source

store(
    dataset: TimeFDataset,
    *,
    force: bool = False,
    values_backend: str = "parquet",
    progress_cb: Callable[[WriteProgressEvent], None]
    | None = None,
    pinned_by: Mapping[DatasetRef, PayloadSource]
    | None = None,
) -> str

Compile a dataset locally and upload it under this registry's prefix.

Uploads every file to a temporary <version>.tmp-<uuid> prefix first. Then moves each object into the final <version>/ prefix with a server-side copy, writing manifest.json last as the commit marker. A failed publish then never leaves a partial version a reader can trust. An already-committed version is skipped unless force.

Parameters:

Name	Type	Description	Default
`dataset`	`TimeFDataset`	The populated dataset to store.	required
`force`	`bool`	Publish even if the version is already committed.	`False`
`values_backend`	`str`	Storage backend for the values plane (`"parquet"` or `"zarr"`).	`'parquet'`
`progress_cb`	`Callable[[WriteProgressEvent], None] \| None`	Optional writer progress callback.	`None`
`pinned_by`	`Mapping[DatasetRef, PayloadSource] \| None`	The releases that already read files of this release, with their pins. The writer publishes only if it reproduced each pinned file.	`None`

Returns:

Type	Description
`str`	The stored version string.

WritableRegistry ¶source

Bases: BaseRegistry, ABC

A registry that build can publish datasets into, not just read from.

local_root `abstractmethod` `property` ¶source

local_root: Path

The local registry or cache directory that holds prepared releases.

download_version ¶source

download_version(
    dataset_id: str,
    version: str,
    dest_dir: str | Path,
    *,
    force: bool = False,
    manifest: Manifest | None = None,
    progress_cb: ProgressCallback | None = None,
) -> None

Download complete local copies of a release and of each of its payload sources.

Only the release's own payload sources are downloaded: it lists every release whose files it reads. Each is checked against the files the release pins.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset ID.	required
`version`	`str`	The exact release version.	required
`dest_dir`	`str \| Path`	The local `<registry>/<org>/<name>/<version>` directory.	required
`force`	`bool`	Download the requested release again.	`False`
`manifest`	`Manifest \| None`	An already-loaded manifest for the requested release.	`None`
`progress_cb`	`ProgressCallback \| None`	Report transferred bytes.	`None`

Raises:

Type	Description
`TimeFValidationError`	If the destination does not identify the requested release.

exists ¶source

exists(dataset_id: str, version: str) -> bool

Return whether a committed version already exists in this registry.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required

Returns:

Type	Description
`bool`	`True` if the version has a committed manifest, else `False`.

get_manifest `abstractmethod` ¶source

get_manifest(
    dataset_id: str, version: str | None = None
) -> Manifest

Return the manifest of a dataset.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str \| None`	The version string, or `None` for the latest version.	`None`

Returns:

Name	Type	Description
`The`	`Manifest`	class:`~timenet.manifest.Manifest` of the dataset.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version is unknown.

list_datasets `abstractmethod` ¶source

list_datasets() -> list[DatasetMetadata]

Return the metadata of every dataset, using the latest version. The result is sorted by dataset id.

Returns:

Name	Type	Description
`One`	`list[DatasetMetadata]`	class:`~timenet.types.DatasetMetadata` object for each dataset.

open_file `abstractmethod` ¶source

open_file(
    dataset_id: str, version: str, relpath: str
) -> BinaryIO

Open one file of a dataset version for binary reading.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required
`relpath`	`str`	The file path relative to the version directory.	required

Returns:

Type	Description
`BinaryIO`	An open binary file object.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version is unknown.

open_version `abstractmethod` ¶source

open_version(
    dataset_id: str, version: str | None = None
) -> DatasetVersion

Open a committed dataset version as a random-access handle.

The returned :class:~timenet.registry.version.DatasetVersion object bundles the parsed manifest with a handle to the files of the version. The handle is rooted in a filesystem. A reader built from this object does not re-open the registry. It also does not re-read manifest.json. The filesystem of the handle MUST serve range reads. The reader uses it to read Parquet footers and value slices out of order.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str \| None`	The version string, or `None` for the latest version.	`None`

Returns:

Type	Description
`DatasetVersion`	A handle to the manifest and files of the committed version.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version is unknown.

search ¶source

search(
    *,
    query: str | list[str] | None = None,
    domain: Domain | list[Domain] | None = None,
    task: type[Task] | list[type[Task]] | None = None,
    license: License | list[License] | None = None,
    time_series_spec: str | list[str] | None = None,
    dataset_id: str | list[str] | None = None,
    tag: str | list[str] | None = None,
    limit: int = 100,
) -> list[DatasetMetadata]

Filter datasets by any combination of criteria.

Each filter accepts a single value or a list of values. The method ignores filters set to None. It combines all other filters with AND logic. The filters query, domain, task, license, and dataset_id match a dataset when any of their values match. The filters time_series_spec and tag match a dataset only when all of their values match. The type filters task and time_series_spec read the manifest schema of each dataset.

Parameters:

Name	Type	Description	Default
`query`	`str \| list[str] \| None`	Free-text terms. The method matches each term as a case-insensitive substring in the dataset name, description, or tags.	`None`
`domain`	`Domain \| list[Domain] \| None`	Keep datasets that share any of these domains.	`None`
`task`	`type[Task] \| list[type[Task]] \| None`	Keep datasets whose schema includes any of these task classes.	`None`
`license`	`License \| list[License] \| None`	Keep datasets that have any of these licenses.	`None`
`time_series_spec`	`str \| list[str] \| None`	Keep datasets that declare all of these `spec_type` values.	`None`
`dataset_id`	`str \| list[str] \| None`	Keep only these ids.	`None`
`tag`	`str \| list[str] \| None`	Keep datasets that declare all of these tags.	`None`
`limit`	`int`	The maximum number of results. The default is 100.	`100`

Returns:

Type	Description
`list[DatasetMetadata]`	The matching dataset metadata. The list has at most `limit` entries.

Raises:

Type	Description
`ValueError`	If `limit` is negative.

store `abstractmethod` ¶source

store(
    dataset: TimeFDataset,
    *,
    force: bool = False,
    values_backend: str = "parquet",
    progress_cb: Callable[[WriteProgressEvent], None]
    | None = None,
    pinned_by: Mapping[DatasetRef, PayloadSource]
    | None = None,
) -> str

Compile a dataset and publish it to this registry.

If the dataset has no schema, this method derives one first. If a version is already committed, this method skips it unless force is set.

Parameters:

Name	Type	Description	Default
`dataset`	`TimeFDataset`	The populated dataset to store.	required
`force`	`bool`	Overwrite an already-committed version instead of skipping it.	`False`
`values_backend`	`str`	Storage backend for the values plane (`"parquet"` or `"zarr"`).	`'parquet'`
`progress_cb`	`Callable[[WriteProgressEvent], None] \| None`	Optional writer progress callback.	`None`
`pinned_by`	`Mapping[DatasetRef, PayloadSource] \| None`	The releases that already read files of this release, with their pins. The writer publishes only if it reproduced each pinned file.	`None`

Returns:

Type	Description
`str`	The stored version string.

default_registry_path ¶source

default_registry_path() -> Path

Resolve the local registry directory a build writes to (and the SDK reads from) by default.

If $TIMENET_REGISTRY names a local directory, use it. Otherwise use the default <TIMENET_HOME>/registry. The build CLI and the timenet_connectors build and load helpers all call this function, so producer and consumer agree on where a dataset lands. If $TIMENET_REGISTRY names a remote registry, this function propagates the :class:~timenet.errors.TimeNetRegistryError from :func:local_registry_path, because a build cannot write to a remote registry.

Returns:

Type	Description
`Path`	The local registry directory.

local_registry_path ¶source

local_registry_path(uri: str | Path) -> Path

Resolve a registry URI to the local directory it names, for build to write into.

Every backend is a :class:WritableRegistry, so :func:open_writable_registry cannot separate a directory the engine can write to from a remote stub. This function can.

Parameters:

Name	Type	Description	Default
`uri`	`str \| Path`	A `file://` URI or local path.	required

Returns:

Type	Description
`Path`	The local directory the URI names, with `~` expanded.

Raises:

Type	Description
`TimeNetRegistryError`	If the URI names a remote backend or carries an unsupported scheme, which build cannot write to.

open_local_version ¶source

open_local_version(
    path: str | Path,
    *,
    sources_root: str | Path | None = None,
) -> DatasetVersion

Open a local release directory and its payload sources.

Opening reads only metadata. The manifest must be valid, with a supported format version, and every file it lists must be present with its recorded size. Each payload source must be found under the sources root, match the files the release pins, and hold each of them with its recorded size.

Parameters:

Name	Type	Description	Default
`path`	`str \| Path`	The release directory.	required
`sources_root`	`str \| Path \| None`	The registry root to find payload sources under. By default this is the root that holds `path` as `<root>/<org>/<name>/<version>`, so a release and its sources can move together. A release without payload sources opens from any directory.	`None`

Returns:

Type	Description
`DatasetVersion`	A handle with each payload source opened. No catalog or values file is opened.

Raises:

Type	Description
`FileNotFoundError`	If `path` has no manifest.
`TimeFValidationError`	If the release has payload sources, is not stored as `<root>/<org>/<name>/<version>`, and no `sources_root` is given.

open_registry ¶source

open_registry(
    uri: str | Path, *, cache_dir: str | Path | None = None
) -> BaseRegistry

Open a registry from a URI or path.

The scheme selects the backend. http(s):// and timenet:// open a :class:RemoteRegistry, and timenet:// is an alias for the hosted :data:TIMENET_REGISTRY_URL. s3:// opens an :class:S3Registry. file:// or a plain path opens a :class:LocalRegistry.

Parameters:

Name	Type	Description	Default
`uri`	`str \| Path`	A URL, `timenet://` / `s3://` / `file://` URI, or local path.	required
`cache_dir`	`str \| Path \| None`	Where a remote backend caches downloads. The local backend ignores it. Defaults to the configured storage directory when `None`.	`None`

Returns:

Type	Description
`BaseRegistry`	The matching registry backend.

Raises:

Type	Description
`ValueError`	If `uri` carries a scheme no backend handles, if a `timenet://` URI carries a path, or if `uri` is a `file://` URI with a host component (the host would be dropped without notice).

open_writable_registry ¶source

open_writable_registry(uri: str | Path) -> WritableRegistry

Open a registry that supports :meth:~WritableRegistry.store, for build to publish into.

Parameters:

Name	Type	Description	Default
`uri`	`str \| Path`	A URL, `timenet://` / `s3://` / `file://` URI, or local path.	required

Returns:

Type	Description
`WritableRegistry`	The matching writable registry backend.

Raises:

Type	Description
`ValueError`	If the resolved backend does not support writing.

Registry

timenet.registry ¶source

TIMENET_REGISTRY_URL module-attribute ¶source

BaseRegistry ¶source

local_root abstractmethod property ¶source

download_version ¶source

get_manifest abstractmethod ¶source

list_datasets abstractmethod ¶source

open_file abstractmethod ¶source

open_version abstractmethod ¶source

search ¶source

DatasetVersion dataclass ¶source

filesystem instance-attribute ¶source

manifest instance-attribute ¶source

payload_sources class-attribute instance-attribute ¶source

root instance-attribute ¶source

fsspec_mapper ¶source

resolve_file_path ¶source

resolve_payload_source ¶source

LocalRegistry ¶source

local_root property ¶source

root property ¶source

download_version ¶source

exists ¶source

get_manifest ¶source

list_datasets ¶source

open_file ¶source

open_version ¶source

search ¶source

store ¶source

RemoteRegistry ¶source

local_root property ¶source

download_version ¶source

exists ¶source

get_manifest ¶source

list_datasets ¶source

open_file ¶source

open_version ¶source

search ¶source

store ¶source

S3Registry ¶source

local_root property ¶source

download_version ¶source

exists ¶source

get_manifest ¶source

list_datasets ¶source

open_file ¶source

open_version ¶source

search ¶source

store ¶source

WritableRegistry ¶source

local_root abstractmethod property ¶source

download_version ¶source

exists ¶source

get_manifest abstractmethod ¶source

list_datasets abstractmethod ¶source

open_file abstractmethod ¶source

open_version abstractmethod ¶source

search ¶source

store abstractmethod ¶source

default_registry_path ¶source

local_registry_path ¶source

open_local_version ¶source

open_registry ¶source

open_writable_registry ¶source

TIMENET_REGISTRY_URL `module-attribute` ¶source

local_root `abstractmethod` `property` ¶source

get_manifest `abstractmethod` ¶source

list_datasets `abstractmethod` ¶source

open_file `abstractmethod` ¶source

open_version `abstractmethod` ¶source

DatasetVersion `dataclass` ¶source

filesystem `instance-attribute` ¶source

manifest `instance-attribute` ¶source

payload_sources `class-attribute` `instance-attribute` ¶source

root `instance-attribute` ¶source

local_root `property` ¶source

root `property` ¶source

local_root `property` ¶source

local_root `property` ¶source

local_root `abstractmethod` `property` ¶source

get_manifest `abstractmethod` ¶source

list_datasets `abstractmethod` ¶source

open_file `abstractmethod` ¶source

open_version `abstractmethod` ¶source

store `abstractmethod` ¶source