# Timenet connectors.sources

## timenet\_connectors.sources

Shared source access and media conversion helpers for dataset connectors.

### huggingface\_hub

Helpers for downloading files from pinned Hugging Face dataset revisions.

:func:`hub_snapshot` calls `huggingface_hub.snapshot_download` for the files that match the given
patterns, such as a filename or `data/*.parquet`. The Hub client runs the downloads in its own thread
pool. Its progress bars are hidden; the file bytes go to the
:mod:`~timenet_connectors.download.progress` sink, so the build CLI shows them like any other download.
`huggingface_hub` is listed in the connector's `requirements.txt` and imported lazily, so
`--no-isolation` still works. The Hub client reads `HF_TOKEN` from the environment for gated datasets.

#### hub\_snapshot `async`

```
hub_snapshot(
    repo: str,
    revision: str,
    cache_dir: Path,
    patterns: Sequence[str],
) -> Path
```

Download the files of a dataset revision that match `patterns`.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `repo` | `str` | The Hub dataset id, such as `"BASH-Lab/SensorCaps"`. | *required* |
| `revision` | `str` | The revision to read, normally a pinned commit. | *required* |
| `cache_dir` | `Path` | Directory where the connector caches Hub files. | *required* |
| `patterns` | `Sequence[str]` | Repo-relative filenames or globs, such as `"data/*.parquet"`. | *required* |

Returns:

| Type | Description |
| --- | --- |
| `Path` | The local snapshot directory, with the files at their repo paths. The caller checks |
| `Path` | that the files it needs exist. |

Raises:

| Type | Description |
| --- | --- |
| `ImportError` | If `huggingface_hub`, declared in the connector's requirements, is missing. |
| `TimeNetDownloadError` | If the download fails. |

### images

Represent source images as RGB or RGBA tensor Signals for Zarr.

#### image\_signal

```
image_signal(source: Path | bytes, *, name: str) -> Signal
```

Create a lazy one-frame RGB or RGBA Signal with a shape-specific spec of the image modality.

Returns:

| Type | Description |
| --- | --- |
| `Signal` | A Signal whose single value is the original-size image tensor. |

### readers

Validated readers for common source file formats.

#### CsvDataReader

Bases: `DataReader[RowT]`

Read a UTF-8 CSV file with a header.

##### expected\_columns `instance-attribute`

```
expected_columns = expected_columns
```

##### path `instance-attribute`

```
path = path
```

##### row\_type `instance-attribute`

```
row_type = row_type
```

##### iter\_rows

```
iter_rows() -> Generator[RowT, None, None]
```

Yield validated rows parsed from CSV cells.

Yields:

| Type | Description |
| --- | --- |
| `RowT` | Validated source rows. |

Raises:

| Type | Description |
| --- | --- |
| `TimeNetConnectorError` | If the file or a row is invalid. |

#### DataReader

Bases: `ABC`, `Generic[RowT]`

Read and validate rows from one local source file.

The row schema determines validation strictness. CSV parses its string cells.

##### path `instance-attribute`

```
path = path
```

##### row\_type `instance-attribute`

```
row_type = row_type
```

##### iter\_rows `abstractmethod`

```
iter_rows() -> Generator[RowT, None, None]
```

Yield validated rows in source order.

Close the iterator explicitly if stopping early to release its file.

Yields:

| Type | Description |
| --- | --- |
| `RowT` | Validated source rows. |

#### DataRow

Bases: `BaseModel`

A connector-owned schema for a source row.

##### model\_config `class-attribute` `instance-attribute`

```
model_config = ConfigDict(extra='forbid')
```

#### JsonArrayDataReader

Bases: `DataReader[RowT]`

Read rows from a UTF-8 JSON array, loaded into memory.

##### path `instance-attribute`

```
path = path
```

##### row\_type `instance-attribute`

```
row_type = row_type
```

##### iter\_rows

```
iter_rows() -> Generator[RowT, None, None]
```

Yield validated elements of the array.

Yields:

| Type | Description |
| --- | --- |
| `RowT` | Validated source rows. |

Raises:

| Type | Description |
| --- | --- |
| `TimeNetConnectorError` | If the file or an element is invalid. |

#### JsonlDataReader

Bases: `DataReader[RowT]`

Read one JSON value per line of a UTF-8 file.

##### path `instance-attribute`

```
path = path
```

##### row\_type `instance-attribute`

```
row_type = row_type
```

##### iter\_rows

```
iter_rows() -> Generator[RowT, None, None]
```

Yield validated lines.

Yields:

| Type | Description |
| --- | --- |
| `RowT` | Validated source rows. |

Raises:

| Type | Description |
| --- | --- |
| `TimeNetConnectorError` | If the file or a line is invalid. |

#### ParquetDataReader

Bases: `DataReader[RowT]`

Read a Parquet file in batches, yielding individual rows.

##### path `instance-attribute`

```
path = path
```

##### row\_type `instance-attribute`

```
row_type = row_type
```

##### selected\_columns `instance-attribute`

```
selected_columns = (
    None
    if selected_columns is None
    else list(selected_columns)
)
```

##### iter\_rows

```
iter_rows() -> Generator[RowT, None, None]
```

Yield validated rows in file order.

Yields:

| Type | Description |
| --- | --- |
| `RowT` | Validated source rows. |

Raises:

| Type | Description |
| --- | --- |
| `TimeNetConnectorError` | If the file or a row is invalid. |

#### base

Common contract for validated source rows.

##### RowT `module-attribute`

```
RowT = TypeVar('RowT', bound=DataRow)
```

##### DataReader

Bases: `ABC`, `Generic[RowT]`

Read and validate rows from one local source file.

The row schema determines validation strictness. CSV parses its string cells.

###### path `instance-attribute`

```
path = path
```

###### row\_type `instance-attribute`

```
row_type = row_type
```

###### iter\_rows `abstractmethod`

```
iter_rows() -> Generator[RowT, None, None]
```

Yield validated rows in source order.

Close the iterator explicitly if stopping early to release its file.

Yields:

| Type | Description |
| --- | --- |
| `RowT` | Validated source rows. |

##### DataRow

Bases: `BaseModel`

A connector-owned schema for a source row.

###### model\_config `class-attribute` `instance-attribute`

```
model_config = ConfigDict(extra='forbid')
```

#### csv

Read CSV source rows.

##### CsvDataReader

Bases: `DataReader[RowT]`

Read a UTF-8 CSV file with a header.

###### expected\_columns `instance-attribute`

```
expected_columns = expected_columns
```

###### path `instance-attribute`

```
path = path
```

###### row\_type `instance-attribute`

```
row_type = row_type
```

###### iter\_rows

```
iter_rows() -> Generator[RowT, None, None]
```

Yield validated rows parsed from CSV cells.

Yields:

| Type | Description |
| --- | --- |
| `RowT` | Validated source rows. |

Raises:

| Type | Description |
| --- | --- |
| `TimeNetConnectorError` | If the file or a row is invalid. |

#### json

Read JSON and JSON Lines source rows.

##### JsonArrayDataReader

Bases: `DataReader[RowT]`

Read rows from a UTF-8 JSON array, loaded into memory.

###### path `instance-attribute`

```
path = path
```

###### row\_type `instance-attribute`

```
row_type = row_type
```

###### iter\_rows

```
iter_rows() -> Generator[RowT, None, None]
```

Yield validated elements of the array.

Yields:

| Type | Description |
| --- | --- |
| `RowT` | Validated source rows. |

Raises:

| Type | Description |
| --- | --- |
| `TimeNetConnectorError` | If the file or an element is invalid. |

##### JsonlDataReader

Bases: `DataReader[RowT]`

Read one JSON value per line of a UTF-8 file.

###### path `instance-attribute`

```
path = path
```

###### row\_type `instance-attribute`

```
row_type = row_type
```

###### iter\_rows

```
iter_rows() -> Generator[RowT, None, None]
```

Yield validated lines.

Yields:

| Type | Description |
| --- | --- |
| `RowT` | Validated source rows. |

Raises:

| Type | Description |
| --- | --- |
| `TimeNetConnectorError` | If the file or a line is invalid. |

#### parquet

Read Parquet source rows.

##### ParquetDataReader

Bases: `DataReader[RowT]`

Read a Parquet file in batches, yielding individual rows.

###### path `instance-attribute`

```
path = path
```

###### row\_type `instance-attribute`

```
row_type = row_type
```

###### selected\_columns `instance-attribute`

```
selected_columns = (
    None
    if selected_columns is None
    else list(selected_columns)
)
```

###### iter\_rows

```
iter_rows() -> Generator[RowT, None, None]
```

Yield validated rows in file order.

Yields:

| Type | Description |
| --- | --- |
| `RowT` | Validated source rows. |

Raises:

| Type | Description |
| --- | --- |
| `TimeNetConnectorError` | If the file or a row is invalid. |