Manifest¶

The Dataset Manifest (manifest.json) is the entry point for a TimeF version. It contains the card metadata, derived schema, counts, and file descriptors. The writer writes it last, so its presence marks a committed version. The reader reads it before opening control.duckdb or Signal values. Manifest types live in timenet.manifest.

The DatasetSchema type already holds flat descriptors. The manifest's schema block is a direct serialization of this type. The manifest has no separate "entry" types to keep in sync.

The packaged manifest.schema.json (JSON Schema draft 2020-12) pins the on-disk shape. This file is the formal contract for external consumers. It is published as manifest.schema.json and is available in Python as timenet.schemas.MANIFEST_SCHEMA. A test validates the output of to_dict() against this schema.

Manifest¶

from timenet.manifest import Manifest, ManifestCounts, ManifestFiles

Manifest(
    dataset_id="physionet/ecg-qa-cot",
    metadata=metadata,          # DatasetMetadata
    files=files,                # ManifestFiles (required)
    dataset_schema=schema,      # Serialized as the `schema` block
    counts=counts,              # ManifestCounts (default: empty)
    values_backend="parquet",   # "parquet" (default) or "zarr"
    value_encoding={},          # spec_type -> the encoding its shards carry
    build_env={},               # {"python", "packages"} (default: empty)
    timef_format_version=1,     # required; validated against the supported set {1}
)

values_backend names the values backend that wrote files.time_series. The reader uses this value to select the backend. If the key is absent, the reader uses "parquet".

value_encoding gives the values encoding that wrote the shards of each spec type. No code reads this field. Parquet records the applied encoding in the footer of each file, so the reader does not need it. The field lets a builder see which encoding a build selected. The field is empty for a backend that has no such choice.

build_env records the environment that produced the version. It gives the interpreter version and every installed package with its version. timenet.provenance.build_env collects this data. Like value_encoding, build_env is provenance only. No code reads it to interpret the data.

The values locator is backend-neutral. One schema covers both scalar and multidimensional specs. A multidimensional spec records its shape in value_shape and dimension_names. It does not need a separate format version.

A spec records its nullable flag. Parquet stores nullability in Arrow validity bitmaps. Zarr stores it in validity arrays next to the values.

Direct construction and parsing with an unsupported timef_format_version raise Pydantic's ValidationError.

Codec¶

model_validate requires timef_format_version, dataset_id, metadata, and files. schema and counts default to empty. Unknown fields are rejected. A malformed block raises Pydantic's ValidationError with its native field locations and error details.

Serialization notes¶

• Known units serialize to their pint names ("hertz", "millivolt", "dimensionless"). An unknown unit serializes as null. The field remains required.
• Device information belongs to a Source, in its name, metadata, or annotations. The time origin belongs to the Record's start_time. Neither belongs to TimeSeriesSpec.
• Tasks serialize as {"task_type": ...}. On read, the reader resolves them against the built-in TASKS registry. An unknown task_type raises Pydantic's ValidationError. The annotation value_type round-trips as a string. The reader uses it to decode values.

ManifestCounts¶

The fields count records, sources, signals, axes, reusable annotation_contents, annotation_occurrences, and signal_chunks. tasks maps task type to count, and signals_by_spec maps specification type to Signal count. All fields default to 0 or {}.

ManifestFiles¶

ManifestFiles lists the single control.duckdb artifact, the time_series values-plane artifacts, and packed payloads files for large text and irregular time offsets. A reader uses this list and never uses a directory glob.

Each entry is a FilePart. A FilePart carries the file's path (version-relative), its checksum (with the sha256: prefix), and its size in bytes. So the path and the digest never live in separate structures.

all_files() returns every descriptor. all_parts() returns only the paths.

Parent and storage pins¶

metadata.parents declares the exact parent releases of a child, by parent alias. parent_cards holds a copy of the card of each parent release. lineage holds the card of each ancestor release, the parents included. These cards record where the child comes from. A read does not need them.

payload_sources maps each source key, org/name@version, to a release that stores payload files that this release reads. It pins each of these files as a copy of the FilePart record of that release (see Shared payloads). A payload source can be a parent, or another ancestor that stores the file. A registry never changes a published release. So opening compares the pins with the manifest of the source, and hashes no file. files.payloads lists the packed text and time-offset files that this release stores itself.

A build needs the parent releases. A read needs only the payload sources. TimeNet.open(), TimeNet.load(), TimeNet.open_torch(), and TimeFReader.open() resolve the payload sources before the first read. A read never opens the catalog of a parent. download() copies the release and its payload sources. See Payload sources.

See the API reference for timenet.manifest for the full symbol listing.