Connectors¶

The unit of dataset integration: one connector class per dataset. A connector fetches raw data and converts it into a TimeFDataset. It has no knowledge of the registry, the engine, or any other connector, and the consumer SDK never runs it.

The contract, BaseConnector, lives in the timenet package (timenet.connectors). Concrete connectors live in the timenet-connectors repo alongside their dataset card.

BaseConnector¶

from pathlib import Path
from timenet.connectors import BaseConnector
from timenet.dataset import TimeFDataset

class MyConnector(BaseConnector[MyRawRef]):
    def download(self, cache_dir: Path) -> list[MyRawRef]: ...
    def convert(self, raw_refs: list[MyRawRef]) -> TimeFDataset: ...

convert is the one abstract stage; download is concrete with a default return [], so a synthetic connector implements convert alone. The engine drives download -> convert -> derive_schema -> store:

metadata() and store() are concrete methods you inherit, not stages you implement:

• metadata() reads and validates the dataset's dataset.yaml card from disk (DatasetMetadata.from_yaml), so it does file I/O; override it only to point at a different card. metadata().dataset_id must match the id the connector is curated under.
• store() writes the dataset through a TimeFWriter, deriving the schema first if absent, and returns the committed version directory; most connectors never override it.

Connectors take no constructor arguments. Configuration comes from environment variables read in __init__. TRaw is whatever reference type the connector defines (a path, a small dataclass, an S3 key). Generic via PEP 695: class MyConnector(BaseConnector[MyRawRef]).

Sharing data¶

To share time-series data across samples, attach the same TimeSeries instance (or two instances with the same explicit time_series_id) to each sample. The writer dedupes by time_series_id, so the bytes are stored once. The same applies to annotations, which the writer dedupes by id.

Discovery and layout¶

Connectors are found lazily by dataset id: there is no central registry to maintain. A concrete connector lives in its own folder at datasets/<org>/<name>/ (lowercase Python package names): the package's __init__.py exposes a module-level CONNECTOR and a dataset.yaml card sits beside it, so timenet-curate build <org>/<name> imports just that package. Reusable bases live under bases/. Each connector declares its own id in metadata(); ids are lowercase org/name.

Optional dependencies and credentials¶

A connector may need libraries or credentials its source requires. Declare heavy libraries as an optional extra and import them lazily inside the connector so base users don't have to install them; a missing library should raise a clear error. Credentials come from the environment. For the HuggingFace Hub, a token is read from HF_TOKEN automatically (needed only for gated/private sources). Downloaded source files cache under <TIMENET_CACHE> (see client config).

Example connectors¶

• timenet/hello-world is a synthetic, offline reference connector. It needs no network and produces a fully deterministic dataset, so it doubles as the round-trip fixture: two modalities over a shared data source, a series shared across samples, a windowed sample, a chunk-split-sized series, all three annotation shapes (one shared), and a ClassificationTask -> QATask chain plus a LabelingTask. Its dataset card, dataset.yaml, sits beside it in datasets/timenet/hello_world/.
• chengsenwang/tsqa is a time-series QA dataset: each row's series becomes a TimeSeries and its question/answer a QATask.

timenet-curate build chengsenwang/tsqa                 # live download from the Hub into the local registry
timenet-curate build chengsenwang/tsqa --keep-cache    # keep the raw sources for a faster rebuild

A successful build removes the dataset's raw download cache (<TIMENET_CACHE>/<dataset_id>), since the sources are only needed during conversion; pass --keep-cache to retain them.

Keeping download and convert apart is what makes a connector testable offline: convert takes raw references and touches no network, so a test hands it a checked-in fixture and skips download entirely. See packages/timenet-connectors/tests/fixtures/ and the _convert() helpers beside them.

Once built, load and inspect a dataset with the SDK. See examples/load_tsqa.py, which loads a dataset and calls describe() to print its identity, counts, per-spec columns, and a sample preview.

See the API reference for timenet.connectors for the full symbol listing.