Connectors
timenet.connectors ¶source
The connector contract: :class:BaseConnector.
BaseConnector ¶source
Abstract base for dataset connectors. One concrete subclass per dataset.
A connector lives in its own folder and declares its descriptive identity in a dataset.yaml
card beside it (read by :meth:metadata). Set :attr:CARD to point elsewhere. Subclasses
implement the two abstract stages. download is I/O-only and convert is CPU-only.
Connectors take no constructor arguments.
CARD
class-attribute
¶source
Optional explicit path to the dataset card YAML. When None (the default), the connector
reads the card from dataset.yaml in its own folder.
values_backend
class-attribute
instance-attribute
¶source
values_backend: str = 'parquet'
Default storage backend for this connector's values plane.
convert
abstractmethod
¶source
convert(raw_refs: list[TRaw]) -> TimeFDataset
Parse raw references and populate a :class:~timenet.dataset.TimeFDataset.
CPU-bound: no network I/O. Attach time-series values as lazy loaders instead of materializing them.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
raw_refs
|
list[TRaw]
|
The references returned by :meth: |
required |
Returns:
| Type | Description |
|---|---|
TimeFDataset
|
The populated dataset. |
download ¶source
Fetch or discover raw source files and return lightweight references to them.
I/O only: no parsing, no array work. Must be idempotent for a given cache_dir. Override
this for a synchronous connector. For an I/O-bound one, override :meth:download_async
instead and leave this default, which drives it to completion, since the engine calls
connectors synchronously.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
cache_dir
|
Path
|
Directory to write downloaded files into (created by the engine). |
required |
Returns:
| Type | Description |
|---|---|
list[TRaw]
|
Raw references passed directly to :meth: |
download_async
async
¶source
Async variant of :meth:download for connectors whose downloads are I/O-bound.
Override this to fetch artifacts concurrently, for example with the connector HTTP download
helpers. The default :meth:download runs it for you. Implement exactly one of the two.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
cache_dir
|
Path
|
Directory to write downloaded files into (created by the engine). |
required |
Returns:
| Type | Description |
|---|---|
list[TRaw]
|
Raw references passed directly to :meth: |
Raises:
| Type | Description |
|---|---|
NotImplementedError
|
If a subclass overrides neither :meth: |
metadata ¶source
metadata() -> DatasetMetadata
Return the dataset's descriptive identity, loaded and validated from its card YAML.
Reads the card by convention (dataset.yaml beside the connector, unless :attr:CARD
overrides it). Its dataset_id must match the id used to register or build the connector.
Returns:
| Type | Description |
|---|---|
DatasetMetadata
|
The dataset's :class: |