# Timenet.connectors

## timenet.connectors

The connector contract: :class:`BaseConnector`.

### BaseConnector

Bases: `ABC`, `Generic[TRaw]`

Abstract base for dataset connectors. One concrete subclass per dataset.

A connector lives in its own folder and declares its descriptive identity in a `dataset.yaml`
card beside it (read by :meth:`metadata`). Set :attr:`CARD` to point elsewhere. Subclasses
implement a download stage and either `convert` or `compose`. Connectors take no constructor
arguments.

#### CARD `class-attribute`

```
CARD: str | Path | None = None
```

Optional explicit path to the dataset card YAML. When `None` (the default), the connector
reads the card from `dataset.yaml` in its own folder.

#### values\_backend `class-attribute` `instance-attribute`

```
values_backend: ValuesBackend = ValuesBackend.PARQUET
```

Storage backend for the dataset's time-series values.

#### compose

```
compose(
    raw_refs: list[TRaw], context: BuildContext
) -> TimeFDataset
```

Build the dataset from raw references and the opened parent releases.

A connector whose card declares `parents` must override this hook. The connector reads
each parent through `context.parents[alias]` and copies objects with
`dataset.import_from(...)`. After this hook returns, the engine records each declared
parent in the dataset: its card and its lineage. The default ignores `context` and calls
:meth:`convert`.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `raw_refs` | `list[TRaw]` | The references returned by :meth:`download`. | *required* |
| `context` | `BuildContext` | The opened parents, keyed by the parent aliases on the card. | *required* |

Returns:

| Type | Description |
| --- | --- |
| `TimeFDataset` | The populated dataset. |

#### convert

```
convert(raw_refs: list[TRaw]) -> TimeFDataset
```

Parse raw references and populate a :class:`~timenet.dataset.TimeFDataset`.

CPU-bound: no network I/O. Attach time-series values as lazy loaders instead of materializing
them.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `raw_refs` | `list[TRaw]` | The references returned by :meth:`download`. | *required* |

Returns:

| Type | Description |
| --- | --- |
| `TimeFDataset` | The populated dataset. |

Raises:

| Type | Description |
| --- | --- |
| `NotImplementedError` | If called on a connector that only implements `compose`. |

#### download

```
download(cache_dir: Path) -> list[TRaw]
```

Fetch or discover raw source files and return lightweight references to them.

I/O only: no parsing, no array work. Must be idempotent for a given `cache_dir`. Override
this for a synchronous connector. For an I/O-bound one, override :meth:`download_async`
instead and leave this default, which drives it to completion, since the engine calls
connectors synchronously.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `cache_dir` | `Path` | Directory to write downloaded files into (created by the engine). | *required* |

Returns:

| Type | Description |
| --- | --- |
| `list[TRaw]` | Raw references passed directly to :meth:`convert`. |

#### download\_async `async`

```
download_async(cache_dir: Path) -> list[TRaw]
```

Async variant of :meth:`download` for connectors whose downloads are I/O-bound.

Override this to fetch artifacts concurrently, for example with the connector HTTP download
helpers. The default :meth:`download` runs it for you. Implement exactly one of the two.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `cache_dir` | `Path` | Directory to write downloaded files into (created by the engine). | *required* |

Returns:

| Type | Description |
| --- | --- |
| `list[TRaw]` | Raw references passed directly to :meth:`convert`. |

Raises:

| Type | Description |
| --- | --- |
| `NotImplementedError` | If a subclass overrides neither :meth:`download` nor :meth:`download_async`. |

#### metadata

```
metadata() -> DatasetMetadata
```

Return the dataset's descriptive identity, loaded and validated from its card YAML.

Reads the card by convention (`dataset.yaml` beside the connector, unless :attr:`CARD`
overrides it). Its `dataset_id` must match the id used to register or build the connector.

Returns:

| Type | Description |
| --- | --- |
| `DatasetMetadata` | The dataset's :class:`~timenet.types.DatasetMetadata`. |

### BuildContext `dataclass`

The declared parent releases of one connector build, opened by parent alias.

The engine opens the parents from the output registry before the download starts, and records
each of them in the composed dataset after :meth:`~timenet.connectors.BaseConnector.compose`
returns. The context only holds the readers and closes them.

#### parents `class-attribute` `instance-attribute`

```
parents: dict[ParentAlias, TimeFReader] = field(
    default_factory=dict
)
```

#### references `class-attribute` `instance-attribute`

```
references: dict[ParentAlias, DatasetRef] = field(
    default_factory=dict
)
```

#### close

```
close() -> None
```

Release the parent readers once the writer has consumed them.

The reader opened last closes first. A reader that fails to close does not keep the others
open: each of them still closes, and this method then raises the error.