Registry¶

A registry serves compiled TimeF versions to the SDK. A compiled TimeF version has manifests, Parquet control tables, and a values plane in Parquet or Zarr format. A registry never runs connector code. The registry code lives in timenet.registry.

You can have several registries: a public registry, private internal registries, or a local directory. The output of build is itself a valid local registry.

Choosing a registry¶

from timenet.registry import open_registry

registry = open_registry("./local_registry")

open_registry(uri) dispatches by scheme:

An unrecognized scheme, for example gs:// or az://, raises ValueError instead of becoming a local path. open_registry expands ~ in a local path or a file:// URI.

Remote registries¶

http(s):// and timenet:// both select RemoteRegistry. timenet:// is a shorthand for the hosted https://registry.timenet.ai service root and takes no path: dataset ids are passed to the client methods, not folded into the registry URI. An http(s):// URI is used verbatim as the service root, which is how you point at a private or a dev deployment.

The HTTP transport authenticates with a bearer token read from $TIMENET_TOKEN. When it is unset the SDK talks to the registry anonymously (no Authorization header), which is enough for public reads. Set it to reach private datasets or to write:

export TIMENET_TOKEN=your-token

S3 registries¶

An s3://bucket/prefix URI opens an S3Registry. boto3 reads credentials, region, and an optional endpoint override from the environment (AWS_* vars, AWS_PROFILE, AWS_ENDPOINT_URL), so nothing is hardcoded. Install the extra:

pip install 'timenet[s3]'

The layout under the prefix mirrors the local one, org/name/version/… with manifest.json beside the data files. store uploads to a temporary version.tmp-<uuid> prefix and moves each object into place (manifest.json last), so a failed publish never leaves a partial version. load reads lazily through a pyarrow S3FileSystem (range reads, no whole-version download), or from the local cache when a prior download populated it. The backend has no catalog, so list and search are unsupported: publish and fetch by an explicit org/name@version.

BaseRegistry¶

Every backend implements this contract: four data-access methods and one shared search method.

LocalRegistry serves a <root>/<dataset_id>/<version>/ tree. RemoteRegistry serves the same layout over the versioned REST contract (GET /api/v1/datasets, /api/v1/datasets/{id}/{version}/manifest, .../download/{relpath}); S3Registry serves the same layout under an s3://bucket/prefix root. The S3 backend has no catalog, so list_datasets and search raise NotImplementedError.

A dataset id is an org/name pair, for example chengsenwang/tsqa. The id nests one level deep on disk, at <root>/chengsenwang/tsqa/<version>/. list_datasets finds these ids at any depth. Use lowercase ids to avoid casing clashes on case-insensitive filesystems.

Writing to a registry¶

A WritableRegistry adds one write method to the read contract. As a result, build can publish into any backend, not only a local directory:

LocalRegistry implements store by streaming the dataset through a TimeFWriter, which stages under <version>.tmp-* and publishes with a single atomic rename. RemoteRegistry runs the publish flow below. S3Registry uploads to a temporary prefix and moves each object into place (manifest.json last). open_writable_registry(uri) resolves a URI like open_registry but returns a WritableRegistry.

from timenet.registry import open_writable_registry

registry = open_writable_registry("./local_registry")
version = registry.store(dataset)   # schema derived if needed, atomic commit

Every backend is a WritableRegistry. As a result, this type alone does not show if a backend is a directory the engine can write to or a remote stub. local_registry_path(uri) gives this information. It returns the directory that a file:// URI or a plain path names. It raises TimeNetRegistryError for a remote scheme. default_registry_path() uses local_registry_path to find the default local registry. It uses $TIMENET_REGISTRY when its value is a local path. Otherwise, it uses <home>/registry. This is how timenet-build build resolves its output when --out is absent. The timenet_connectors.builder and load helpers use it too.

Remote publishing¶

RemoteRegistry.store compiles the dataset locally, then hands the artifacts to the service over the REST contract. Dataset bytes never pass through the API: the service issues a presigned PUT per file and the SDK uploads straight to the object store. The token must be a writer token ($TIMENET_TOKEN); an anonymous or read-only caller is rejected by the service.

1. Compile into a temporary staging directory with a TimeFWriter, deriving the schema first if the dataset has none. An already-committed version returns early unless force.
2. POST /datasets/{id}/{version}/publish with the compiled manifest.json as the body. The service registers the pending version and returns the list of files it expects.
3. For each file, POST /datasets/{id}/{version}/publish/upload-url with {"path": relpath} to get a presigned grant ({"url", "headers"}), then PUT the file's bytes to that URL with the returned headers.
4. POST /datasets/{id}/{version}/finalize to commit the version. The staging directory is removed whether or not the publish succeeds.

from timenet.registry import open_writable_registry

registry = open_writable_registry("timenet://")  # needs $TIMENET_TOKEN
version = registry.store(dataset)                 # compile, upload, finalize

search¶

registry.search(
    query=None, domain=None, task=None, license=None,
    time_series_spec=None, dataset_id=None, tag=None, limit=100,
)

Every filter takes a scalar value or a list. The registry ignores None filters and combines all non-None filters with AND logic. The consumer CLI, timenet search, mirrors this behavior exactly. limit caps the number of results, with a default of 100. limit=0 returns no results. A negative limit raises ValueError.

The type-filters, task and time_series_spec, resolve each dataset's schema from its committed manifest. The manifest always carries the derived schema, so the registry does not need a precomputed_schema.

See the API reference for timenet.registry for the full symbol listing.