Registry

timenet.registry ¶source

Dataset registries: serve manifests and parquet, and search over metadata.

TIMENET_REGISTRY_URL `module-attribute` ¶source

TIMENET_REGISTRY_URL = 'https://registry.timenet.ai'

The hosted TimeNet registry that the timenet:// scheme is an alias for.

BaseRegistry ¶source

Bases: ABC

A source of TimeF datasets: their manifests, files, and searchable metadata.

get_manifest `abstractmethod` ¶source

get_manifest(
    dataset_id: str, version: str | None = None
) -> Manifest

Return a dataset's manifest.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str \| None`	The version string, or `None` for the latest.	`None`

Returns:

Type	Description
`Manifest`	The dataset's :class:`~timenet.manifest.Manifest`.

Raises:

Type	Description
`DatasetNotFoundError`	If the dataset id or version is unknown.

list_datasets `abstractmethod` ¶source

list_datasets() -> list[DatasetMetadata]

Return the metadata of every dataset (latest version), sorted by id.

Returns:

Name	Type	Description
`One`	`list[DatasetMetadata]`	class:`~timenet.types.DatasetMetadata` per dataset.

open_file `abstractmethod` ¶source

open_file(
    dataset_id: str, version: str, relpath: str
) -> BinaryIO

Open one file of a dataset version for binary reading.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required
`relpath`	`str`	The file path relative to the version directory.	required

Returns:

Type	Description
`BinaryIO`	An open binary file object.

Raises:

Type	Description
`DatasetNotFoundError`	If the dataset id or version is unknown.

search ¶source

search(
    *,
    query: str | list[str] | None = None,
    domain: Domain | list[Domain] | None = None,
    task: type[Task] | list[type[Task]] | None = None,
    license: License | list[License] | None = None,
    time_series_spec: str | list[str] | None = None,
    dataset_id: str | list[str] | None = None,
    tag: str | list[str] | None = None,
    limit: int = 100,
) -> list[DatasetMetadata]

Filter datasets by any combination of criteria.

Each filter accepts a scalar or a list; None filters are ignored and non-None filters are ANDed. query/domain/task/license/dataset_id match any of their values; time_series_spec/tag require all of theirs. Type-filters (task, time_series_spec) read each dataset's manifest schema.

Parameters:

Name	Type	Description	Default
`query`	`str \| list[str] \| None`	Free-text terms matched (case-insensitive substring) against name/description/tags.	`None`
`domain`	`Domain \| list[Domain] \| None`	Keep datasets sharing any of these domains.	`None`
`task`	`type[Task] \| list[type[Task]] \| None`	Keep datasets whose schema includes any of these task classes.	`None`
`license`	`License \| list[License] \| None`	Keep datasets with any of these licenses.	`None`
`time_series_spec`	`str \| list[str] \| None`	Keep datasets declaring all of these `spec_type` values.	`None`
`dataset_id`	`str \| list[str] \| None`	Keep only these ids.	`None`
`tag`	`str \| list[str] \| None`	Keep datasets declaring all of these tags.	`None`
`limit`	`int`	Maximum number of results (default 100).	`100`

Returns:

Type	Description
`list[DatasetMetadata]`	The matching dataset metadata, at most `limit` entries.

LocalRegistry ¶source

Bases: WritableRegistry

Serves datasets from a local directory (the output of curation is itself a valid one).

exists ¶source

exists(dataset_id: str, version: str) -> bool

Return whether a committed version already exists in this registry.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required

Returns:

Type	Description
`bool`	`True` if the version has a committed manifest, else `False`.

get_manifest ¶source

get_manifest(
    dataset_id: str, version: str | None = None
) -> Manifest

Return a dataset's manifest (latest version if unspecified).

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str \| None`	The version string, or `None` / `"latest"` for the latest.	`None`

Returns:

Type	Description
`Manifest`	The dataset's manifest.

Raises:

Type	Description
`DatasetNotFoundError`	If the dataset id or version has no committed manifest.

list_datasets ¶source

list_datasets() -> list[DatasetMetadata]

Return the latest-version metadata of every dataset, sorted by id.

Discovers datasets depth-agnostically so both flat (hello_world) and namespaced (org/name) layouts are found. A dataset id is the path from the root to a version directory's parent.

Returns:

Name	Type	Description
`One`	`list[DatasetMetadata]`	class:`~timenet.types.DatasetMetadata` per dataset.

open_file ¶source

open_file(
    dataset_id: str, version: str, relpath: str
) -> BinaryIO

Open one file of a dataset version for binary reading.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required
`relpath`	`str`	The file path relative to the version directory.	required

Returns:

Type	Description
`BinaryIO`	An open binary file object.

Raises:

Type	Description
`DatasetNotFoundError`	If the dataset version directory does not exist.
`ValueError`	If `relpath` escapes the dataset version directory.

search ¶source

search(
    *,
    query: str | list[str] | None = None,
    domain: Domain | list[Domain] | None = None,
    task: type[Task] | list[type[Task]] | None = None,
    license: License | list[License] | None = None,
    time_series_spec: str | list[str] | None = None,
    dataset_id: str | list[str] | None = None,
    tag: str | list[str] | None = None,
    limit: int = 100,
) -> list[DatasetMetadata]

Filter datasets by any combination of criteria.

Each filter accepts a scalar or a list; None filters are ignored and non-None filters are ANDed. query/domain/task/license/dataset_id match any of their values; time_series_spec/tag require all of theirs. Type-filters (task, time_series_spec) read each dataset's manifest schema.

Parameters:

Name	Type	Description	Default
`query`	`str \| list[str] \| None`	Free-text terms matched (case-insensitive substring) against name/description/tags.	`None`
`domain`	`Domain \| list[Domain] \| None`	Keep datasets sharing any of these domains.	`None`
`task`	`type[Task] \| list[type[Task]] \| None`	Keep datasets whose schema includes any of these task classes.	`None`
`license`	`License \| list[License] \| None`	Keep datasets with any of these licenses.	`None`
`time_series_spec`	`str \| list[str] \| None`	Keep datasets declaring all of these `spec_type` values.	`None`
`dataset_id`	`str \| list[str] \| None`	Keep only these ids.	`None`
`tag`	`str \| list[str] \| None`	Keep datasets declaring all of these tags.	`None`
`limit`	`int`	Maximum number of results (default 100).	`100`

Returns:

Type	Description
`list[DatasetMetadata]`	The matching dataset metadata, at most `limit` entries.

store ¶source

store(
    dataset: TimeFDataset,
    *,
    force: bool = False,
    progress_cb: Callable[[WriteProgressEvent], None]
    | None = None,
) -> str

Compile a dataset and write it into this registry's directory tree.

Streams the dataset through a :class:~timenet.writer.TimeFWriter, which stages under <version>.tmp-* and publishes with a single atomic rename. An already-committed version is skipped unless force is set.

Parameters:

Name	Type	Description	Default
`dataset`	`TimeFDataset`	The populated dataset to store.	required
`force`	`bool`	Overwrite an already-committed version instead of skipping it.	`False`
`progress_cb`	`Callable[[WriteProgressEvent], None] \| None`	Optional writer progress callback.	`None`

Returns:

Type	Description
`str`	The stored version string.

RemoteRegistry ¶source

Bases: WritableRegistry

Placeholder for an HTTP(S)-backed registry (not yet implemented).

exists ¶source

exists(dataset_id: str, version: str) -> bool

Return whether a committed version already exists in this registry.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required

Returns:

Type	Description
`bool`	`True` if the version has a committed manifest, else `False`.

get_manifest ¶source

get_manifest(
    dataset_id: str, version: str | None = None
) -> Manifest

Not yet implemented.

Raises:

Type	Description
`NotImplementedError`	Always.

list_datasets ¶source

list_datasets() -> list[DatasetMetadata]

Not yet implemented.

Raises:

Type	Description
`NotImplementedError`	Always.

open_file ¶source

open_file(
    dataset_id: str, version: str, relpath: str
) -> BinaryIO

Not yet implemented.

Raises:

Type	Description
`NotImplementedError`	Always.

search ¶source

search(
    *,
    query: str | list[str] | None = None,
    domain: Domain | list[Domain] | None = None,
    task: type[Task] | list[type[Task]] | None = None,
    license: License | list[License] | None = None,
    time_series_spec: str | list[str] | None = None,
    dataset_id: str | list[str] | None = None,
    tag: str | list[str] | None = None,
    limit: int = 100,
) -> list[DatasetMetadata]

Filter datasets by any combination of criteria.

Each filter accepts a scalar or a list; None filters are ignored and non-None filters are ANDed. query/domain/task/license/dataset_id match any of their values; time_series_spec/tag require all of theirs. Type-filters (task, time_series_spec) read each dataset's manifest schema.

Parameters:

Name	Type	Description	Default
`query`	`str \| list[str] \| None`	Free-text terms matched (case-insensitive substring) against name/description/tags.	`None`
`domain`	`Domain \| list[Domain] \| None`	Keep datasets sharing any of these domains.	`None`
`task`	`type[Task] \| list[type[Task]] \| None`	Keep datasets whose schema includes any of these task classes.	`None`
`license`	`License \| list[License] \| None`	Keep datasets with any of these licenses.	`None`
`time_series_spec`	`str \| list[str] \| None`	Keep datasets declaring all of these `spec_type` values.	`None`
`dataset_id`	`str \| list[str] \| None`	Keep only these ids.	`None`
`tag`	`str \| list[str] \| None`	Keep datasets declaring all of these tags.	`None`
`limit`	`int`	Maximum number of results (default 100).	`100`

Returns:

Type	Description
`list[DatasetMetadata]`	The matching dataset metadata, at most `limit` entries.

store ¶source

store(
    dataset: TimeFDataset,
    *,
    force: bool = False,
    progress_cb: Callable[[WriteProgressEvent], None]
    | None = None,
) -> str

Not yet implemented.

Raises:

Type	Description
`NotImplementedError`	Always.

S3Registry ¶source

Bases: WritableRegistry

Placeholder for an S3-backed registry (not yet implemented).

exists ¶source

exists(dataset_id: str, version: str) -> bool

Return whether a committed version already exists in this registry.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required

Returns:

Type	Description
`bool`	`True` if the version has a committed manifest, else `False`.

get_manifest ¶source

get_manifest(
    dataset_id: str, version: str | None = None
) -> Manifest

Not yet implemented.

Raises:

Type	Description
`NotImplementedError`	Always.

list_datasets ¶source

list_datasets() -> list[DatasetMetadata]

Not yet implemented.

Raises:

Type	Description
`NotImplementedError`	Always.

open_file ¶source

open_file(
    dataset_id: str, version: str, relpath: str
) -> BinaryIO

Not yet implemented.

Raises:

Type	Description
`NotImplementedError`	Always.

search ¶source

search(
    *,
    query: str | list[str] | None = None,
    domain: Domain | list[Domain] | None = None,
    task: type[Task] | list[type[Task]] | None = None,
    license: License | list[License] | None = None,
    time_series_spec: str | list[str] | None = None,
    dataset_id: str | list[str] | None = None,
    tag: str | list[str] | None = None,
    limit: int = 100,
) -> list[DatasetMetadata]

Filter datasets by any combination of criteria.

Each filter accepts a scalar or a list; None filters are ignored and non-None filters are ANDed. query/domain/task/license/dataset_id match any of their values; time_series_spec/tag require all of theirs. Type-filters (task, time_series_spec) read each dataset's manifest schema.

Parameters:

Name	Type	Description	Default
`query`	`str \| list[str] \| None`	Free-text terms matched (case-insensitive substring) against name/description/tags.	`None`
`domain`	`Domain \| list[Domain] \| None`	Keep datasets sharing any of these domains.	`None`
`task`	`type[Task] \| list[type[Task]] \| None`	Keep datasets whose schema includes any of these task classes.	`None`
`license`	`License \| list[License] \| None`	Keep datasets with any of these licenses.	`None`
`time_series_spec`	`str \| list[str] \| None`	Keep datasets declaring all of these `spec_type` values.	`None`
`dataset_id`	`str \| list[str] \| None`	Keep only these ids.	`None`
`tag`	`str \| list[str] \| None`	Keep datasets declaring all of these tags.	`None`
`limit`	`int`	Maximum number of results (default 100).	`100`

Returns:

Type	Description
`list[DatasetMetadata]`	The matching dataset metadata, at most `limit` entries.

store ¶source

store(
    dataset: TimeFDataset,
    *,
    force: bool = False,
    progress_cb: Callable[[WriteProgressEvent], None]
    | None = None,
) -> str

Not yet implemented.

Raises:

Type	Description
`NotImplementedError`	Always.

WritableRegistry ¶source

Bases: BaseRegistry, ABC

A registry that curation can publish datasets into, not just read from.

exists ¶source

exists(dataset_id: str, version: str) -> bool

Return whether a committed version already exists in this registry.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required

Returns:

Type	Description
`bool`	`True` if the version has a committed manifest, else `False`.

get_manifest `abstractmethod` ¶source

get_manifest(
    dataset_id: str, version: str | None = None
) -> Manifest

Return a dataset's manifest.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str \| None`	The version string, or `None` for the latest.	`None`

Returns:

Type	Description
`Manifest`	The dataset's :class:`~timenet.manifest.Manifest`.

Raises:

Type	Description
`DatasetNotFoundError`	If the dataset id or version is unknown.

list_datasets `abstractmethod` ¶source

list_datasets() -> list[DatasetMetadata]

Return the metadata of every dataset (latest version), sorted by id.

Returns:

Name	Type	Description
`One`	`list[DatasetMetadata]`	class:`~timenet.types.DatasetMetadata` per dataset.

open_file `abstractmethod` ¶source

open_file(
    dataset_id: str, version: str, relpath: str
) -> BinaryIO

Open one file of a dataset version for binary reading.

Parameters:

Name	Type	Description	Default
`dataset_id`	`str`	The dataset id.	required
`version`	`str`	The version string.	required
`relpath`	`str`	The file path relative to the version directory.	required

Returns:

Type	Description
`BinaryIO`	An open binary file object.

Raises:

Type	Description
`DatasetNotFoundError`	If the dataset id or version is unknown.

search ¶source

search(
    *,
    query: str | list[str] | None = None,
    domain: Domain | list[Domain] | None = None,
    task: type[Task] | list[type[Task]] | None = None,
    license: License | list[License] | None = None,
    time_series_spec: str | list[str] | None = None,
    dataset_id: str | list[str] | None = None,
    tag: str | list[str] | None = None,
    limit: int = 100,
) -> list[DatasetMetadata]

Filter datasets by any combination of criteria.

Each filter accepts a scalar or a list; None filters are ignored and non-None filters are ANDed. query/domain/task/license/dataset_id match any of their values; time_series_spec/tag require all of theirs. Type-filters (task, time_series_spec) read each dataset's manifest schema.

Parameters:

Name	Type	Description	Default
`query`	`str \| list[str] \| None`	Free-text terms matched (case-insensitive substring) against name/description/tags.	`None`
`domain`	`Domain \| list[Domain] \| None`	Keep datasets sharing any of these domains.	`None`
`task`	`type[Task] \| list[type[Task]] \| None`	Keep datasets whose schema includes any of these task classes.	`None`
`license`	`License \| list[License] \| None`	Keep datasets with any of these licenses.	`None`
`time_series_spec`	`str \| list[str] \| None`	Keep datasets declaring all of these `spec_type` values.	`None`
`dataset_id`	`str \| list[str] \| None`	Keep only these ids.	`None`
`tag`	`str \| list[str] \| None`	Keep datasets declaring all of these tags.	`None`
`limit`	`int`	Maximum number of results (default 100).	`100`

Returns:

Type	Description
`list[DatasetMetadata]`	The matching dataset metadata, at most `limit` entries.

store `abstractmethod` ¶source

store(
    dataset: TimeFDataset,
    *,
    force: bool = False,
    progress_cb: Callable[[WriteProgressEvent], None]
    | None = None,
) -> str

Compile a dataset and publish it to this registry.

Derives the dataset's schema first if it has none. An already-committed version is skipped unless force is set.

Parameters:

Name	Type	Description	Default
`dataset`	`TimeFDataset`	The populated dataset to store.	required
`force`	`bool`	Overwrite an already-committed version instead of skipping it.	`False`
`progress_cb`	`Callable[[WriteProgressEvent], None] \| None`	Optional writer progress callback.	`None`

Returns:

Type	Description
`str`	The stored version string.

default_registry_path ¶source

default_registry_path() -> Path

Resolve the local registry directory a build writes to (and the SDK reads from) by default.

Honors $TIMENET_REGISTRY when it names a local directory, otherwise falls back to the default <TIMENET_HOME>/registry. The single source of truth shared by the curate CLI and the timenet_connectors build/load helpers, so producer and consumer never disagree on where a dataset lands. Propagates :class:~timenet.errors.RegistryError from :func:local_registry_path when $TIMENET_REGISTRY names a remote registry, which cannot be built into.

Returns:

Type	Description
`Path`	The local registry directory.

local_registry_path ¶source

local_registry_path(uri: str | Path) -> Path

Resolve a registry URI to the local directory it names, for curation to write into.

Every backend is a :class:WritableRegistry, so :func:open_writable_registry cannot tell a directory the engine can write to from a remote stub. This can.

Parameters:

Name	Type	Description	Default
`uri`	`str \| Path`	A `file://` URI or local path.	required

Returns:

Type	Description
`Path`	The local directory the URI names, with `~` expanded.

Raises:

Type	Description
`RegistryError`	If the URI names a remote backend, which curation cannot write to.

open_registry ¶source

open_registry(uri: str | Path) -> BaseRegistry

Open a registry from a URI or path.

Dispatches by scheme: http(s):// and timenet:// open a :class:RemoteRegistry (timenet:// is an alias for the hosted :data:TIMENET_REGISTRY_URL), s3:// opens an :class:S3Registry, and file:// or a plain path opens a :class:LocalRegistry.

Parameters:

Name	Type	Description	Default
`uri`	`str \| Path`	A URL, `timenet://` / `s3://` / `file://` URI, or local path.	required

Returns:

Type	Description
`BaseRegistry`	The matching registry backend.

open_writable_registry ¶source

open_writable_registry(uri: str | Path) -> WritableRegistry

Open a registry that supports :meth:~WritableRegistry.store, for curation to publish into.

Parameters:

Name	Type	Description	Default
`uri`	`str \| Path`	A URL, `timenet://` / `s3://` / `file://` URI, or local path.	required

Returns:

Type	Description
`WritableRegistry`	The matching writable registry backend.

Raises:

Type	Description
`ValueError`	If the resolved backend does not support writing.

Registry

timenet.registry ¶source

TIMENET_REGISTRY_URL module-attribute ¶source

BaseRegistry ¶source

get_manifest abstractmethod ¶source

list_datasets abstractmethod ¶source

open_file abstractmethod ¶source

search ¶source

LocalRegistry ¶source

exists ¶source

get_manifest ¶source

list_datasets ¶source

open_file ¶source

search ¶source

store ¶source

RemoteRegistry ¶source

exists ¶source

get_manifest ¶source

list_datasets ¶source

open_file ¶source

search ¶source

store ¶source

S3Registry ¶source

exists ¶source

get_manifest ¶source

list_datasets ¶source

open_file ¶source

search ¶source

store ¶source

WritableRegistry ¶source

exists ¶source

get_manifest abstractmethod ¶source

list_datasets abstractmethod ¶source

open_file abstractmethod ¶source

search ¶source

store abstractmethod ¶source

default_registry_path ¶source

local_registry_path ¶source

open_registry ¶source

open_writable_registry ¶source

TIMENET_REGISTRY_URL `module-attribute` ¶source

get_manifest `abstractmethod` ¶source

list_datasets `abstractmethod` ¶source

open_file `abstractmethod` ¶source

get_manifest `abstractmethod` ¶source

list_datasets `abstractmethod` ¶source

open_file `abstractmethod` ¶source

store `abstractmethod` ¶source