Skip to content

Timenet.manifest

timenet.manifest source

The dataset manifest: a frozen :class:Manifest and its JSON codec.

GitSourceMap module-attribute source

GitSourceMap = dict[PackageName, GitSource]

The commit of each installed TimeNet package, keyed by package name.

OLDEST_READABLE_TIMEF_FORMAT_VERSION module-attribute source

OLDEST_READABLE_TIMEF_FORMAT_VERSION = 2

The oldest TimeF format version that this TimeNet reads.

Version 2 is the format of TimeNet 0.12. Version 1 names layouts used during early development, and no TimeNet reads them.

PackageName module-attribute source

PackageName = StrictStr

The name pip installs, for example timenet-connectors (not the import name timenet_connectors).

TIMEF_FORMAT_VERSION module-attribute source

TIMEF_FORMAT_VERSION = 2

The maximum TimeF format version this TimeNet package supports, and the version that it writes.

It versions every file of a release except control.duckdb, which :data:~timenet.format.duckdb.CONTROL_SCHEMA_VERSION versions.

BuildEnvironment source

Bases: TimeFModel

The Python version, the installed packages, and the TimeNet commits that produced a version.

git instance-attribute source

git: GitSourceMap

The commit of each installed TimeNet package. A package that is not installed is absent.

model_config class-attribute instance-attribute source

model_config = ConfigDict(
    extra="forbid",
    frozen=True,
    revalidate_instances="always",
    serialize_by_alias=True,
    validate_by_name=True,
    validate_default=True,
)

packages instance-attribute source

packages: dict[PackageName, StrictStr]

Each installed package and its version, sorted by name.

python instance-attribute source

python: StrictStr

The interpreter version, for example 3.13.1.

capture classmethod source

capture(git: GitSourceMap) -> Self

Describe this interpreter, with git from TimeNetSnapshot.to_source_map().

FilePart source

Bases: TimeFModel

One data file of a dataset version. It has a path, a checksum, and a byte size in one record.

The path, checksum, and size stay together in one record. A reader does not need to join a file to its digest across two structures. A consumer can verify integrity and plan a download from the manifest alone.

checksum instance-attribute source

checksum: Checksum

The digest of the file, with a sha256: prefix.

model_config class-attribute instance-attribute source

model_config = ConfigDict(
    extra="forbid",
    frozen=True,
    revalidate_instances="always",
    serialize_by_alias=True,
    validate_by_name=True,
    validate_default=True,
)

path instance-attribute source

path: _ManifestPath

The version-relative POSIX path to the file.

size instance-attribute source

size: StrictNonNegativeInt

The size of the file, in bytes.

GitSource source

Bases: TimeFModel

The git commit that one TimeNet package was built from.

contains_local_changes instance-attribute source

contains_local_changes: StrictBool

Whether the package directory had uncommitted changes, untracked files included.

model_config class-attribute instance-attribute source

model_config = ConfigDict(
    extra="forbid",
    frozen=True,
    revalidate_instances="always",
    serialize_by_alias=True,
    validate_by_name=True,
    validate_default=True,
)

sha instance-attribute source

sha: GitSHA

The commit of the checkout, or of the checkout that built the wheel.

Manifest source

Bases: TimeFModel

The single source of truth a consumer reads to interpret a dataset version.

build_env instance-attribute source

build_env: BuildEnvironment

The Python version, the package set, and the TimeNet commits that produced this version.

counts class-attribute instance-attribute source

counts: ManifestCounts = Field(
    default_factory=ManifestCounts
)

Row and entity counts recorded for quick inspection.

dataset_id instance-attribute source

dataset_id: DatasetId

A denormalized copy of metadata.dataset_id.

dataset_schema class-attribute instance-attribute source

dataset_schema: Annotated[
    DatasetSchema, Field(alias=schema)
] = Field(default_factory=DatasetSchema)

Structural schema: time-series specs, annotations, and tasks.

files instance-attribute source

files: ManifestFiles

Descriptor for every data artifact, grouped by kind.

lineage class-attribute instance-attribute source

lineage: tuple[ParentDatasetCard, ...] = ()

Flattened provenance snapshots; never traversed to read data.

metadata instance-attribute source

metadata: DatasetMetadata

Descriptive identity of the dataset.

model_config class-attribute instance-attribute source

model_config = ConfigDict(
    extra="forbid",
    frozen=True,
    revalidate_instances="always",
    serialize_by_alias=True,
    validate_by_name=True,
    validate_default=True,
)

parent_cards class-attribute instance-attribute source

parent_cards: dict[ParentAlias, ParentDatasetCard] = Field(
    default_factory=dict
)

Copies of the cards of the declared parents, indexed by parent alias.

payload_sources class-attribute instance-attribute source

payload_sources: dict[SourceKeyField, PayloadSource] = (
    Field(default_factory=dict)
)

Every release that stores a payload file this release reads, by source key, pinned to those files. The writer flattens them, so a source's own sources are not listed unless this release reads their files too.

timef_format_version instance-attribute source

timef_format_version: Annotated[
    int,
    Field(
        strict=True,
        ge=OLDEST_READABLE_TIMEF_FORMAT_VERSION,
        le=TIMEF_FORMAT_VERSION,
    ),
]

The TimeF format version of the release: every file except control.duckdb, which has its own.

value_encoding class-attribute instance-attribute source

value_encoding: dict[StrictStr, ValueEncoding] = Field(
    default_factory=dict
)

spec_type to the values-column encoding used by its shards.

values_backend class-attribute instance-attribute source

values_backend: ValuesBackend = ValuesBackend.PARQUET

Storage backend for the time-series values plane.

difference_from source

difference_from(
    other: Manifest,
) -> ReleaseDifference | None

Compare this manifest with another one of the same release, ignoring their build environments.

Parameters:

Name Type Description Default
other Manifest

The other manifest.

 required

Returns:

Type Description
ReleaseDifference | None

FILES if the file records differ, else CARD if the rest differs, or None for the same
ReleaseDifference | None

release.

required_releases source

required_releases(
    *, with_parents: bool
) -> list[DatasetRef]

List the releases that this release needs: its payload sources, and its parents if asked.

Parameters:

Name Type Description Default
with_parents bool

Also list the declared parents.

 required

Returns:

Type Description
list[DatasetRef]

The releases of the payload sources, then the parents.

to_dict source

to_dict() -> dict[str, Any]

Serialize the manifest to a validated JSON-compatible dict with all keys present.

Returns:

Type Description
dict[str, Any]

The canonical dict form.

to_json source

to_json() -> str

Serialize the manifest to a pretty JSON string.

Returns:

Type Description
str

The JSON text.

ManifestCounts source

Bases: TimeFModel

TimeF entity counts recorded for inspection without opening DuckDB.

annotation_contents class-attribute instance-attribute source

annotation_contents: StrictNonNegativeInt = 0

Number of reusable annotation content items.

annotation_occurrences class-attribute instance-attribute source

annotation_occurrences: StrictNonNegativeInt = 0

Number of annotation attachments across all object types.

axes class-attribute instance-attribute source

axes: StrictNonNegativeInt = 0

Number of distinct shared TimeAxes.

model_config class-attribute instance-attribute source

model_config = ConfigDict(
    extra="forbid",
    frozen=True,
    revalidate_instances="always",
    serialize_by_alias=True,
    validate_by_name=True,
    validate_default=True,
)

records class-attribute instance-attribute source

records: StrictNonNegativeInt = 0

Total number of records in the dataset.

signal_chunks class-attribute instance-attribute source

signal_chunks: StrictNonNegativeInt = 0

Number of Signal chunk placements in the values plane.

signals class-attribute instance-attribute source

signals: StrictNonNegativeInt = 0

Total number of Signals.

signals_by_spec class-attribute instance-attribute source

signals_by_spec: dict[StrictStr, StrictNonNegativeInt] = (
    Field(default_factory=dict)
)

Signal count keyed by specification type.

sources class-attribute instance-attribute source

sources: StrictNonNegativeInt = 0

Total number of Sources at every recursion depth.

tasks class-attribute instance-attribute source

tasks: dict[StrictStr, StrictNonNegativeInt] = Field(
    default_factory=dict
)

Count of tasks keyed by task type.

ManifestFiles source

Bases: TimeFModel

Descriptors for every TimeF artifact. Readers use this data rather than a file glob.

control instance-attribute source

control: ControlParts

The single immutable control.duckdb file.

model_config class-attribute instance-attribute source

model_config = ConfigDict(
    extra="forbid",
    frozen=True,
    revalidate_instances="always",
    serialize_by_alias=True,
    validate_by_name=True,
    validate_default=True,
)

payloads class-attribute instance-attribute source

payloads: tuple[FilePart, ...] = ()

Packed large text, string lists, and irregular time offsets.

time_series class-attribute instance-attribute source

time_series: tuple[FilePart, ...] = ()

Parquet shards or files inside the Zarr values store.

all_files source

all_files() -> tuple[FilePart, ...]

Return every file descriptor across all artifacts, in a stable order.

Returns:

Type Description
tuple[FilePart, ...]

The control file followed by every values-plane artifact.

all_parts source

all_parts() -> tuple[str, ...]

Return the version-relative path of every file, in the same order as :meth:all_files.

Returns:

Type Description
str

The path of every file. Use this when you only need to find the files, for example to
...

download them.

PayloadSource source

Bases: TimeFModel

A release whose payload files this release reads, pinned to the files it reuses.

The pins copy the source's own manifest records, so checking that the source is still the release this one was written against needs only the two manifests, no hashing.

files instance-attribute source

files: Annotated[tuple[FilePart, ...], Field(min_length=1)]

The source's manifest records for the files this release reads, in path order.

model_config class-attribute instance-attribute source

model_config = ConfigDict(
    extra="forbid",
    frozen=True,
    revalidate_instances="always",
    serialize_by_alias=True,
    validate_by_name=True,
    validate_default=True,
)

release instance-attribute source

release: DatasetRef

The exact release that stores the files.

parse_manifest source

parse_manifest(
    text: str | bytes,
    where: str,
    *,
    path: Path | None = None,
) -> Manifest

Parse the JSON text of a stored manifest.

Parameters:

Name Type Description Default
text str | bytes

The JSON text.

 required
where str

The source location to include in errors.

 required
path Path | None

The file of the manifest, or None for a manifest that a registry served.

 None

Returns:

Type Description
Manifest

The manifest.

Raises:

Type Description
TimeFFormatVersionError

If the manifest uses an older or a newer format than this TimeNet reads.
TimeNetManifestError

If the text is not a valid manifest.

read_manifest source

read_manifest(path: Path) -> Manifest

Read a manifest.json file.

Parameters:

Name Type Description Default
path Path

The file.

 required

Returns:

Type Description
Manifest

The manifest.

Raises:

Type Description
TimeFFormatError

If the file cannot be read.
TimeFFormatVersionError

If the manifest uses an older or a newer format than this TimeNet reads.
TimeNetManifestError

If the file is not a valid manifest.