Registry

timenet.registry ¶source

Dataset registries: serve compiled TimeF versions and search over metadata.

TIMENET_REGISTRY_URL `module-attribute` ¶source

TIMENET_REGISTRY_URL = 'https://registry.timenet.ai'

The hosted TimeNet registry that the timenet:// scheme is an alias for.

BaseRegistry ¶source

Bases: ABC

A source of TimeF datasets: their manifests, files, and searchable metadata.

download_version ¶source

download_version(
    dataset_id: str,
    version: str,
    dest_dir: str | Path,
    *,
    force: bool = False,
    manifest: Manifest | None = None,
    progress_cb: ProgressCallback | None = None,
) -> None

Download a version's files into dest_dir, swapping the directory in atomically.

Fetches every file through :meth:open_file into a sibling <version>.tmp-* staging directory, with manifest.json last as the commit marker. It then renames the staging directory over dest_dir in one step. An interrupted download never leaves a half-written copy in place. Subclasses can override this method with a faster path, such as concurrent streaming from a remote service.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required
`dest_dir`	`str \| Path`	The target `<...>/<id>/<version>` directory.	required
`force`	`bool`	Re-download even if a copy already exists.	`False`
`manifest`	`Manifest \| None`	The parsed manifest, passed to avoid re-fetching it. Fetched if `None`.	`None`
`progress_cb`	`ProgressCallback \| None`	Called with each staged file's byte count, for a progress display.	`None`

get_manifest `abstractmethod` ¶source

get_manifest(
    dataset_id: str, version: str | None = None
) -> Manifest

Return the manifest of a dataset.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str \| None`	The version string, or `None` for the latest version.	`None`

Returns:

Name	Type	Description
`The`	`Manifest`	class:`~timenet.manifest.Manifest` of the dataset.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version is unknown.

list_datasets `abstractmethod` ¶source

list_datasets() -> list[DatasetMetadata]

Return the metadata of every dataset, using the latest version. The result is sorted by dataset id.

Returns:

Name	Type	Description
`One`	`list[DatasetMetadata]`	class:`~timenet.types.DatasetMetadata` object for each dataset.

open_file `abstractmethod` ¶source

open_file(
    dataset_id: str, version: str, relpath: str
) -> BinaryIO

Open one file of a dataset version for binary reading.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required
`relpath`	`str`	The file path relative to the version directory.	required

Returns:

Type	Description
`BinaryIO`	An open binary file object.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version is unknown.

open_version `abstractmethod` ¶source

open_version(
    dataset_id: str, version: str | None = None
) -> DatasetVersion

Open a committed dataset version as a random-access handle.

The returned :class:~timenet.registry.version.DatasetVersion object bundles the parsed manifest with a handle to the files of the version. The handle is rooted in a filesystem. A reader built from this object does not re-open the registry. It also does not re-read manifest.json. The filesystem of the handle MUST serve range reads. The reader uses it to read Parquet footers and value slices out of order.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str \| None`	The version string, or `None` for the latest version.	`None`

Returns:

Type	Description
`DatasetVersion`	A handle to the manifest and files of the committed version.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version is unknown.

search ¶source

search(
    *,
    query: str | list[str] | None = None,
    domain: Domain | list[Domain] | None = None,
    task: type[Task] | list[type[Task]] | None = None,
    license: License | list[License] | None = None,
    time_series_spec: str | list[str] | None = None,
    dataset_id: str | list[str] | None = None,
    tag: str | list[str] | None = None,
    limit: int = 100,
) -> list[DatasetMetadata]

Filter datasets by any combination of criteria.

Each filter accepts a single value or a list of values. The method ignores filters set to None. It combines all other filters with AND logic. The filters query, domain, task, license, and dataset_id match a dataset when any of their values match. The filters time_series_spec and tag match a dataset only when all of their values match. The type filters task and time_series_spec read the manifest schema of each dataset.

Parameters:

Name	Type	Description	Default
`query`	`str \| list[str] \| None`	Free-text terms. The method matches each term as a case-insensitive substring in the dataset name, description, or tags.	`None`
`domain`	`Domain \| list[Domain] \| None`	Keep datasets that share any of these domains.	`None`
`task`	`type[Task] \| list[type[Task]] \| None`	Keep datasets whose schema includes any of these task classes.	`None`
`license`	`License \| list[License] \| None`	Keep datasets that have any of these licenses.	`None`
`time_series_spec`	`str \| list[str] \| None`	Keep datasets that declare all of these `spec_type` values.	`None`
`dataset_id`	`str \| list[str] \| None`	Keep only these ids.	`None`
`tag`	`str \| list[str] \| None`	Keep datasets that declare all of these tags.	`None`
`limit`	`int`	The maximum number of results. The default is 100.	`100`

Returns:

Type	Description
`list[DatasetMetadata]`	The matching dataset metadata. The list has at most `limit` entries.

Raises:

Type	Description
`ValueError`	If `limit` is negative.

DatasetVersion `dataclass` ¶source

An opened dataset version: its manifest plus a filesystem-rooted handle to its files.

filesystem `instance-attribute` ¶source

filesystem: FileSystem

The filesystem the version's files live on (LocalFileSystem now, S3FileSystem later). It MUST serve range reads. The reader pulls Parquet footers and value slices out of order through open_input_file, so a forward-only download stream does not qualify.

manifest `instance-attribute` ¶source

manifest: Manifest

The version's parsed manifest. The reader trusts it rather than re-reading manifest.json.

root `instance-attribute` ¶source

root: str

The version's root prefix on :attr:filesystem. An example is /abs/ds/1.0.0.

open_local `classmethod` ¶source

open_local(root: str | Path) -> DatasetVersion

Open a committed version directory on the local filesystem.

Reads the version's manifest.json and pairs it with a :class:pyarrow.fs.LocalFileSystem. A caller builds this handle when it already holds a version directory on disk, such as build's copy-on-write edit or a downloaded copy. It does not go through a registry's open_version.

Parameters:

Name	Type	Description	Default
`root`	`str \| Path`	The version directory (`<...>/<dataset_id>/<version>`).	required

Returns:

Type	Description
`DatasetVersion`	A handle rooted at `root`.

Raises:

Type	Description
`FileNotFoundError`	If `root` has no `manifest.json`.

path ¶source

path(relpath: str) -> str

Return the filesystem path of a version-relative file.

Parameters:

Name	Type	Description	Default
`relpath`	`str`	A path relative to the version root.	required

Returns:

Type	Description
`str`	The path to hand a pyarrow reader alongside :attr:`filesystem`.

store_uri ¶source

store_uri(relpath: str) -> str

Return a version-relative file as a store location for a store-oriented backend.

The Parquet backend reads through (filesystem, path). The Zarr backend opens a store instead, which is what this method hands it. For a local version, the store is the plain filesystem path, which Zarr opens as a LocalStore. An object-store version needs a scheme-qualified URI, such as s3://…, because Zarr uses only this value and ignores :attr:filesystem. The S3 backend will supply that scheme once it exists. The two accessors stay distinct even though they return the same value for a local version.

Parameters:

Name	Type	Description	Default
`relpath`	`str`	A path relative to the version root.	required

Returns:

Type	Description
`str`	The store location for `relpath`.

LocalRegistry ¶source

Bases: WritableRegistry

Serve datasets from a local directory. The build output is itself a valid registry.

root `property` ¶source

root: Path

The directory this registry reads from and writes to.

download_version ¶source

download_version(
    dataset_id: str,
    version: str,
    dest_dir: str | Path,
    *,
    force: bool = False,
    manifest: Manifest | None = None,
    progress_cb: ProgressCallback | None = None,
) -> None

Download a version's files into dest_dir, swapping the directory in atomically.

Fetches every file through :meth:open_file into a sibling <version>.tmp-* staging directory, with manifest.json last as the commit marker. It then renames the staging directory over dest_dir in one step. An interrupted download never leaves a half-written copy in place. Subclasses can override this method with a faster path, such as concurrent streaming from a remote service.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required
`dest_dir`	`str \| Path`	The target `<...>/<id>/<version>` directory.	required
`force`	`bool`	Re-download even if a copy already exists.	`False`
`manifest`	`Manifest \| None`	The parsed manifest, passed to avoid re-fetching it. Fetched if `None`.	`None`
`progress_cb`	`ProgressCallback \| None`	Called with each staged file's byte count, for a progress display.	`None`

exists ¶source

exists(dataset_id: str, version: str) -> bool

Return whether a committed version already exists in this registry.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required

Returns:

Type	Description
`bool`	`True` if the version has a committed manifest, else `False`.

get_manifest ¶source

get_manifest(
    dataset_id: str, version: str | None = None
) -> Manifest

Return a dataset's manifest (latest version if unspecified).

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str \| None`	The version string, or `None` / `"latest"` for the latest.	`None`

Returns:

Type	Description
`Manifest`	The dataset's manifest.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version has no committed manifest.
`TimeFFormatError`	If the stored manifest declares a dataset id that differs from its directory. This means a misplaced or corrupt artifact.

list_datasets ¶source

list_datasets() -> list[DatasetMetadata]

Return the latest-version metadata of every dataset, sorted by id.

Search at any depth. This finds both flat (hello_world) and namespaced (org/name) layouts. A dataset id is the path from the root to a version directory's parent.

Returns:

Name	Type	Description
`One`	`list[DatasetMetadata]`	class:`~timenet.types.DatasetMetadata` per dataset.

open_file ¶source

open_file(
    dataset_id: str, version: str, relpath: str
) -> BinaryIO

Open one file of a dataset version for binary reading.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required
`relpath`	`str`	The file path relative to the version directory.	required

Returns:

Type	Description
`BinaryIO`	An open binary file object.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset version directory does not exist.
`ValueError`	If `relpath` escapes the dataset version directory.

open_version ¶source

open_version(
    dataset_id: str, version: str | None = None
) -> DatasetVersion

Open a committed version as a local, random-access handle.

Reuses the manifest :meth:get_manifest already parsed and validated, so a missing version or a misplaced artifact surfaces there. Roots the handle at the version's directory over a :class:pyarrow.fs.LocalFileSystem, which is zero network and already seekable.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str \| None`	The version string, or `None` for the latest.	`None`

Returns:

Type	Description
`DatasetVersion`	A handle to the committed version's manifest and files.

search ¶source

search(
    *,
    query: str | list[str] | None = None,
    domain: Domain | list[Domain] | None = None,
    task: type[Task] | list[type[Task]] | None = None,
    license: License | list[License] | None = None,
    time_series_spec: str | list[str] | None = None,
    dataset_id: str | list[str] | None = None,
    tag: str | list[str] | None = None,
    limit: int = 100,
) -> list[DatasetMetadata]

Filter datasets by any combination of criteria.

Each filter accepts a single value or a list of values. The method ignores filters set to None. It combines all other filters with AND logic. The filters query, domain, task, license, and dataset_id match a dataset when any of their values match. The filters time_series_spec and tag match a dataset only when all of their values match. The type filters task and time_series_spec read the manifest schema of each dataset.

Parameters:

Name	Type	Description	Default
`query`	`str \| list[str] \| None`	Free-text terms. The method matches each term as a case-insensitive substring in the dataset name, description, or tags.	`None`
`domain`	`Domain \| list[Domain] \| None`	Keep datasets that share any of these domains.	`None`
`task`	`type[Task] \| list[type[Task]] \| None`	Keep datasets whose schema includes any of these task classes.	`None`
`license`	`License \| list[License] \| None`	Keep datasets that have any of these licenses.	`None`
`time_series_spec`	`str \| list[str] \| None`	Keep datasets that declare all of these `spec_type` values.	`None`
`dataset_id`	`str \| list[str] \| None`	Keep only these ids.	`None`
`tag`	`str \| list[str] \| None`	Keep datasets that declare all of these tags.	`None`
`limit`	`int`	The maximum number of results. The default is 100.	`100`

Returns:

Type	Description
`list[DatasetMetadata]`	The matching dataset metadata. The list has at most `limit` entries.

Raises:

Type	Description
`ValueError`	If `limit` is negative.

store ¶source

store(
    dataset: TimeFDataset,
    *,
    force: bool = False,
    values_backend: str = "parquet",
    progress_cb: Callable[[WriteProgressEvent], None]
    | None = None,
) -> str

Compile a dataset and write it into this registry's directory tree.

Stream the dataset through a :class:~timenet.writer.TimeFWriter. The writer stages under <version>.tmp-* and publishes with a single atomic rename. It skips an already-committed version unless the caller sets force.

Parameters:

Name	Type	Description	Default
`dataset`	`TimeFDataset`	The populated dataset to store.	required
`force`	`bool`	Overwrite an already-committed version instead of skipping it.	`False`
`values_backend`	`str`	Storage backend for the values plane (`"parquet"` or `"zarr"`).	`'parquet'`
`progress_cb`	`Callable[[WriteProgressEvent], None] \| None`	Optional writer progress callback.	`None`

Returns:

Type	Description
`str`	The stored version string.

RemoteRegistry ¶source

Bases: WritableRegistry

Serves datasets from the hosted TimeNet registry over HTTP(S).

download_version ¶source

download_version(
    dataset_id: str,
    version: str,
    dest_dir: str | Path,
    *,
    force: bool = False,
    manifest: Manifest | None = None,
    progress_cb: ProgressCallback | None = None,
) -> None

Download a version's files into dest_dir (used by TimeNet.download for remotes).

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The `org/name` id.	required
`version`	`str`	The version string.	required
`dest_dir`	`str \| Path`	The target `<...>/<id>/<version>` directory.	required
`force`	`bool`	Re-download even if a copy already exists.	`False`
`manifest`	`Manifest \| None`	The already-parsed manifest, passed to avoid re-fetching it; fetched if `None`.	`None`
`progress_cb`	`ProgressCallback \| None`	Called with each chunk's byte count as it is written, for a progress display.	`None`

exists ¶source

exists(dataset_id: str, version: str) -> bool

Return whether a committed version already exists in this registry.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required

Returns:

Type	Description
`bool`	`True` if the version has a committed manifest, else `False`.

get_manifest ¶source

get_manifest(
    dataset_id: str, version: str | None = None
) -> Manifest

Return a dataset's manifest (latest version if unspecified).

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The `org/name` id.	required
`version`	`str \| None`	The version string, or `None` / `"latest"` for the latest.	`None`

Returns:

Type	Description
`Manifest`	The dataset's manifest.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version is unknown.

list_datasets ¶source

list_datasets() -> list[DatasetMetadata]

Return the latest-version metadata of every dataset, sorted by id.

Returns:

Name	Type	Description
`One`	`list[DatasetMetadata]`	class:`~timenet.types.DatasetMetadata` per dataset.

open_file ¶source

open_file(
    dataset_id: str, version: str, relpath: str
) -> BinaryIO

Open one file of a dataset version for binary reading.

Streams the presigned URL into a spooled temporary file, so the returned handle is seekable and self-contained (the caller closes it).

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The `org/name` id.	required
`version`	`str`	The version string.	required
`relpath`	`str`	The version-relative file path.	required

Returns:

Type	Description
`BinaryIO`	An open, seekable binary file positioned at the start.

open_version ¶source

open_version(
    dataset_id: str, version: str | None = None
) -> DatasetVersion

Open a committed version as a local handle, materializing it on first use.

A remote version is downloaded in full and then read from local disk, like the local and S3 backends. On-demand range reads are a separate follow-up.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The `org/name` id.	required
`version`	`str \| None`	The version string, or `None` for the latest.	`None`

Returns:

Type	Description
`DatasetVersion`	A handle to the committed version's manifest and files.

search ¶source

search(
    *,
    query: str | list[str] | None = None,
    domain: Domain | list[Domain] | None = None,
    task: type[Task] | list[type[Task]] | None = None,
    license: License | list[License] | None = None,
    time_series_spec: str | list[str] | None = None,
    dataset_id: str | list[str] | None = None,
    tag: str | list[str] | None = None,
    limit: int = 100,
) -> list[DatasetMetadata]

Filter datasets by any combination of criteria.

Each filter accepts a single value or a list of values. The method ignores filters set to None. It combines all other filters with AND logic. The filters query, domain, task, license, and dataset_id match a dataset when any of their values match. The filters time_series_spec and tag match a dataset only when all of their values match. The type filters task and time_series_spec read the manifest schema of each dataset.

Parameters:

Name	Type	Description	Default
`query`	`str \| list[str] \| None`	Free-text terms. The method matches each term as a case-insensitive substring in the dataset name, description, or tags.	`None`
`domain`	`Domain \| list[Domain] \| None`	Keep datasets that share any of these domains.	`None`
`task`	`type[Task] \| list[type[Task]] \| None`	Keep datasets whose schema includes any of these task classes.	`None`
`license`	`License \| list[License] \| None`	Keep datasets that have any of these licenses.	`None`
`time_series_spec`	`str \| list[str] \| None`	Keep datasets that declare all of these `spec_type` values.	`None`
`dataset_id`	`str \| list[str] \| None`	Keep only these ids.	`None`
`tag`	`str \| list[str] \| None`	Keep datasets that declare all of these tags.	`None`
`limit`	`int`	The maximum number of results. The default is 100.	`100`

Returns:

Type	Description
`list[DatasetMetadata]`	The matching dataset metadata. The list has at most `limit` entries.

Raises:

Type	Description
`ValueError`	If `limit` is negative.

store ¶source

store(
    dataset: TimeFDataset,
    *,
    force: bool = False,
    values_backend: str = "parquet",
    progress_cb: Callable[[WriteProgressEvent], None]
    | None = None,
) -> str

Compile a dataset locally and publish it to the remote registry.

Compiles to a temporary directory, then runs the service publish flow: POST the manifest, PUT each declared file to its presigned URL, and finalize. An already-committed version is skipped unless force.

Parameters:

Name	Type	Description	Default
`dataset`	`TimeFDataset`	The populated dataset to store.	required
`force`	`bool`	Publish even if the version is already committed.	`False`
`values_backend`	`str`	Storage backend for the values plane (`"parquet"` or `"zarr"`).	`'parquet'`
`progress_cb`	`Callable[[WriteProgressEvent], None] \| None`	Optional writer progress callback.	`None`

Returns:

Type	Description
`str`	The stored version string.

Raises:

Type	Description
`TimeNetRegistryError`	If the service requests a file that was not produced locally.

S3Registry ¶source

Bases: WritableRegistry

Serves and publishes datasets under an s3://bucket/prefix root.

download_version ¶source

download_version(
    dataset_id: str,
    version: str,
    dest_dir: str | Path,
    *,
    force: bool = False,
    manifest: Manifest | None = None,
    progress_cb: ProgressCallback | None = None,
) -> None

Download a version's files into dest_dir, swapping the directory in atomically.

Fetches every file through :meth:open_file into a sibling <version>.tmp-* staging directory, with manifest.json last as the commit marker. It then renames the staging directory over dest_dir in one step. An interrupted download never leaves a half-written copy in place. Subclasses can override this method with a faster path, such as concurrent streaming from a remote service.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required
`dest_dir`	`str \| Path`	The target `<...>/<id>/<version>` directory.	required
`force`	`bool`	Re-download even if a copy already exists.	`False`
`manifest`	`Manifest \| None`	The parsed manifest, passed to avoid re-fetching it. Fetched if `None`.	`None`
`progress_cb`	`ProgressCallback \| None`	Called with each staged file's byte count, for a progress display.	`None`

exists ¶source

exists(dataset_id: str, version: str) -> bool

Return whether a committed version already exists in this registry.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required

Returns:

Type	Description
`bool`	`True` if the version has a committed manifest, else `False`.

get_manifest ¶source

get_manifest(
    dataset_id: str, version: str | None = None
) -> Manifest

Return a dataset's manifest (latest version if unspecified).

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The `org/name` id.	required
`version`	`str \| None`	The version string, or `None` / `"latest"` for the latest.	`None`

Returns:

Type	Description
`Manifest`	The dataset's manifest.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version has no manifest in the bucket.

list_datasets ¶source

list_datasets() -> list[DatasetMetadata]

Unsupported: the S3 backend has no catalog.

Raises:

Type	Description
`NotImplementedError`	Always.

open_file ¶source

open_file(
    dataset_id: str, version: str, relpath: str
) -> BinaryIO

Open one file of a dataset version for binary reading.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The `org/name` id.	required
`version`	`str`	The version string.	required
`relpath`	`str`	The version-relative file path.	required

Returns:

Type	Description
`BinaryIO`	An open, streaming binary file object.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the object does not exist.

open_version ¶source

open_version(
    dataset_id: str, version: str | None = None
) -> DatasetVersion

Open a committed version as a random-access handle.

A version already downloaded to the local cache is served from disk. Otherwise the handle reads lazily from S3 through a :class:pyarrow.fs.S3FileSystem (range reads, no whole-version fetch).

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The `org/name` id.	required
`version`	`str \| None`	The version string, or `None` for the latest.	`None`

Returns:

Type	Description
`DatasetVersion`	A handle to the committed version's manifest and files.

search ¶source

search(**_kwargs: Any) -> list[DatasetMetadata]

Unsupported: the S3 backend has no catalog.

Raises:

Type	Description
`NotImplementedError`	Always.

store ¶source

store(
    dataset: TimeFDataset,
    *,
    force: bool = False,
    values_backend: str = "parquet",
    progress_cb: Callable[[WriteProgressEvent], None]
    | None = None,
) -> str

Compile a dataset locally and upload it under this registry's prefix.

Uploads every file to a temporary <version>.tmp-<uuid> prefix first. Then moves each object into the final <version>/ prefix with a server-side copy, writing manifest.json last as the commit marker. A failed publish then never leaves a partial version a reader can trust. An already-committed version is skipped unless force.

Parameters:

Name	Type	Description	Default
`dataset`	`TimeFDataset`	The populated dataset to store.	required
`force`	`bool`	Publish even if the version is already committed.	`False`
`values_backend`	`str`	Storage backend for the values plane (`"parquet"` or `"zarr"`).	`'parquet'`
`progress_cb`	`Callable[[WriteProgressEvent], None] \| None`	Optional writer progress callback.	`None`

Returns:

Type	Description
`str`	The stored version string.

WritableRegistry ¶source

Bases: BaseRegistry, ABC

A registry that build can publish datasets into, not just read from.

download_version ¶source

download_version(
    dataset_id: str,
    version: str,
    dest_dir: str | Path,
    *,
    force: bool = False,
    manifest: Manifest | None = None,
    progress_cb: ProgressCallback | None = None,
) -> None

Download a version's files into dest_dir, swapping the directory in atomically.

Fetches every file through :meth:open_file into a sibling <version>.tmp-* staging directory, with manifest.json last as the commit marker. It then renames the staging directory over dest_dir in one step. An interrupted download never leaves a half-written copy in place. Subclasses can override this method with a faster path, such as concurrent streaming from a remote service.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required
`dest_dir`	`str \| Path`	The target `<...>/<id>/<version>` directory.	required
`force`	`bool`	Re-download even if a copy already exists.	`False`
`manifest`	`Manifest \| None`	The parsed manifest, passed to avoid re-fetching it. Fetched if `None`.	`None`
`progress_cb`	`ProgressCallback \| None`	Called with each staged file's byte count, for a progress display.	`None`

exists ¶source

exists(dataset_id: str, version: str) -> bool

Return whether a committed version already exists in this registry.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required

Returns:

Type	Description
`bool`	`True` if the version has a committed manifest, else `False`.

get_manifest `abstractmethod` ¶source

get_manifest(
    dataset_id: str, version: str | None = None
) -> Manifest

Return the manifest of a dataset.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str \| None`	The version string, or `None` for the latest version.	`None`

Returns:

Name	Type	Description
`The`	`Manifest`	class:`~timenet.manifest.Manifest` of the dataset.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version is unknown.

list_datasets `abstractmethod` ¶source

list_datasets() -> list[DatasetMetadata]

Return the metadata of every dataset, using the latest version. The result is sorted by dataset id.

Returns:

Name	Type	Description
`One`	`list[DatasetMetadata]`	class:`~timenet.types.DatasetMetadata` object for each dataset.

open_file `abstractmethod` ¶source

open_file(
    dataset_id: str, version: str, relpath: str
) -> BinaryIO

Open one file of a dataset version for binary reading.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required
`relpath`	`str`	The file path relative to the version directory.	required

Returns:

Type	Description
`BinaryIO`	An open binary file object.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version is unknown.

open_version `abstractmethod` ¶source

open_version(
    dataset_id: str, version: str | None = None
) -> DatasetVersion

Open a committed dataset version as a random-access handle.

The returned :class:~timenet.registry.version.DatasetVersion object bundles the parsed manifest with a handle to the files of the version. The handle is rooted in a filesystem. A reader built from this object does not re-open the registry. It also does not re-read manifest.json. The filesystem of the handle MUST serve range reads. The reader uses it to read Parquet footers and value slices out of order.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str \| None`	The version string, or `None` for the latest version.	`None`

Returns:

Type	Description
`DatasetVersion`	A handle to the manifest and files of the committed version.

Raises:

Type	Description
`TimeNetDatasetNotFoundError`	If the dataset id or version is unknown.

search ¶source

search(
    *,
    query: str | list[str] | None = None,
    domain: Domain | list[Domain] | None = None,
    task: type[Task] | list[type[Task]] | None = None,
    license: License | list[License] | None = None,
    time_series_spec: str | list[str] | None = None,
    dataset_id: str | list[str] | None = None,
    tag: str | list[str] | None = None,
    limit: int = 100,
) -> list[DatasetMetadata]

Filter datasets by any combination of criteria.

Each filter accepts a single value or a list of values. The method ignores filters set to None. It combines all other filters with AND logic. The filters query, domain, task, license, and dataset_id match a dataset when any of their values match. The filters time_series_spec and tag match a dataset only when all of their values match. The type filters task and time_series_spec read the manifest schema of each dataset.

Parameters:

Name	Type	Description	Default
`query`	`str \| list[str] \| None`	Free-text terms. The method matches each term as a case-insensitive substring in the dataset name, description, or tags.	`None`
`domain`	`Domain \| list[Domain] \| None`	Keep datasets that share any of these domains.	`None`
`task`	`type[Task] \| list[type[Task]] \| None`	Keep datasets whose schema includes any of these task classes.	`None`
`license`	`License \| list[License] \| None`	Keep datasets that have any of these licenses.	`None`
`time_series_spec`	`str \| list[str] \| None`	Keep datasets that declare all of these `spec_type` values.	`None`
`dataset_id`	`str \| list[str] \| None`	Keep only these ids.	`None`
`tag`	`str \| list[str] \| None`	Keep datasets that declare all of these tags.	`None`
`limit`	`int`	The maximum number of results. The default is 100.	`100`

Returns:

Type	Description
`list[DatasetMetadata]`	The matching dataset metadata. The list has at most `limit` entries.

Raises:

Type	Description
`ValueError`	If `limit` is negative.

store `abstractmethod` ¶source

store(
    dataset: TimeFDataset,
    *,
    force: bool = False,
    values_backend: str = "parquet",
    progress_cb: Callable[[WriteProgressEvent], None]
    | None = None,
) -> str

Compile a dataset and publish it to this registry.

If the dataset has no schema, this method derives one first. If a version is already committed, this method skips it unless force is set.

Parameters:

Name	Type	Description	Default
`dataset`	`TimeFDataset`	The populated dataset to store.	required
`force`	`bool`	Overwrite an already-committed version instead of skipping it.	`False`
`values_backend`	`str`	Storage backend for the values plane (`"parquet"` or `"zarr"`).	`'parquet'`
`progress_cb`	`Callable[[WriteProgressEvent], None] \| None`	Optional writer progress callback.	`None`

Returns:

Type	Description
`str`	The stored version string.

default_registry_path ¶source

default_registry_path() -> Path

Resolve the local registry directory a build writes to (and the SDK reads from) by default.

If $TIMENET_REGISTRY names a local directory, use it. Otherwise use the default <TIMENET_HOME>/registry. The build CLI and the timenet_connectors build and load helpers all call this function, so producer and consumer agree on where a dataset lands. If $TIMENET_REGISTRY names a remote registry, this function propagates the :class:~timenet.errors.TimeNetRegistryError from :func:local_registry_path, because a build cannot write to a remote registry.

Returns:

Type	Description
`Path`	The local registry directory.

local_registry_path ¶source

local_registry_path(uri: str | Path) -> Path

Resolve a registry URI to the local directory it names, for build to write into.

Every backend is a :class:WritableRegistry, so :func:open_writable_registry cannot separate a directory the engine can write to from a remote stub. This function can.

Parameters:

Name	Type	Description	Default
`uri`	`str \| Path`	A `file://` URI or local path.	required

Returns:

Type	Description
`Path`	The local directory the URI names, with `~` expanded.

Raises:

Type	Description
`TimeNetRegistryError`	If the URI names a remote backend or carries an unsupported scheme, which build cannot write to.

open_registry ¶source

open_registry(
    uri: str | Path, *, cache_dir: str | Path | None = None
) -> BaseRegistry

Open a registry from a URI or path.

The scheme selects the backend. http(s):// and timenet:// open a :class:RemoteRegistry, and timenet:// is an alias for the hosted :data:TIMENET_REGISTRY_URL. s3:// opens an :class:S3Registry. file:// or a plain path opens a :class:LocalRegistry.

Parameters:

Name	Type	Description	Default
`uri`	`str \| Path`	A URL, `timenet://` / `s3://` / `file://` URI, or local path.	required
`cache_dir`	`str \| Path \| None`	Where a remote backend caches downloads. The local backend ignores it. Defaults to the configured storage directory when `None`.	`None`

Returns:

Type	Description
`BaseRegistry`	The matching registry backend.

Raises:

Type	Description
`ValueError`	If `uri` carries a scheme no backend handles, if a `timenet://` URI carries a path, or if `uri` is a `file://` URI with a host component (the host would be dropped without notice).

open_writable_registry ¶source

open_writable_registry(uri: str | Path) -> WritableRegistry

Open a registry that supports :meth:~WritableRegistry.store, for build to publish into.

Parameters:

Name	Type	Description	Default
`uri`	`str \| Path`	A URL, `timenet://` / `s3://` / `file://` URI, or local path.	required

Returns:

Type	Description
`WritableRegistry`	The matching writable registry backend.

Raises:

Type	Description
`ValueError`	If the resolved backend does not support writing.

Registry

timenet.registry ¶source

TIMENET_REGISTRY_URL module-attribute ¶source

BaseRegistry ¶source

download_version ¶source

get_manifest abstractmethod ¶source

list_datasets abstractmethod ¶source

open_file abstractmethod ¶source

open_version abstractmethod ¶source

search ¶source

DatasetVersion dataclass ¶source

filesystem instance-attribute ¶source

manifest instance-attribute ¶source

root instance-attribute ¶source

open_local classmethod ¶source

path ¶source

store_uri ¶source

LocalRegistry ¶source

root property ¶source

download_version ¶source

exists ¶source

get_manifest ¶source

list_datasets ¶source

open_file ¶source

open_version ¶source

search ¶source

store ¶source

RemoteRegistry ¶source

download_version ¶source

exists ¶source

get_manifest ¶source

list_datasets ¶source

open_file ¶source

open_version ¶source

search ¶source

store ¶source

S3Registry ¶source

download_version ¶source

exists ¶source

get_manifest ¶source

list_datasets ¶source

open_file ¶source

open_version ¶source

search ¶source

store ¶source

WritableRegistry ¶source

download_version ¶source

exists ¶source

get_manifest abstractmethod ¶source

list_datasets abstractmethod ¶source

open_file abstractmethod ¶source

open_version abstractmethod ¶source

search ¶source

store abstractmethod ¶source

default_registry_path ¶source

local_registry_path ¶source

open_registry ¶source

open_writable_registry ¶source

TIMENET_REGISTRY_URL `module-attribute` ¶source

get_manifest `abstractmethod` ¶source

list_datasets `abstractmethod` ¶source

open_file `abstractmethod` ¶source

open_version `abstractmethod` ¶source

DatasetVersion `dataclass` ¶source

filesystem `instance-attribute` ¶source

manifest `instance-attribute` ¶source

root `instance-attribute` ¶source

open_local `classmethod` ¶source

root `property` ¶source

get_manifest `abstractmethod` ¶source

list_datasets `abstractmethod` ¶source

open_file `abstractmethod` ¶source

open_version `abstractmethod` ¶source

store `abstractmethod` ¶source