# Timenet connectors.bases

## timenet\_connectors.bases

Reusable connector base classes.

### edf

Read EDF and EDF+ files for the connectors that build datasets from them.

#### reader

Read the EDF container: the header of a file, and one data record from it.

EDF holds an ASCII header and then data records of 16-bit integers. One record holds the
samples of every signal for one stretch of time, one signal after the other. Signals of
different rates therefore hold a different count of samples in the same record.

This module parses no bytes. `edfio` does that. A connector that reads EDF declares
`edfio` in its `requirements.txt`.

:class:`EdfHeader` and :class:`EdfFile` are the boundary: `edfio` types stay inside this
file, thus a later change of library does not reach the modules that call it.

##### EdfAnnotation

Bases: `NamedTuple`

One annotation of an EDF+ file: a labelled stretch of the recording timeline.

###### duration\_microseconds `instance-attribute`

```
duration_microseconds: int
```

###### label `instance-attribute`

```
label: str
```

###### onset\_microseconds `instance-attribute`

```
onset_microseconds: int
```

##### EdfFile

Bases: `NamedTuple`

An open EDF file: its header, and the `edfio.Edf` handle behind it.

###### handle `instance-attribute`

```
handle: Any
```

###### header `instance-attribute`

```
header: EdfHeader
```

###### path `instance-attribute`

```
path: Path
```

##### EdfHeader

Bases: `NamedTuple`

The header of one EDF file. Each tuple below holds one entry for each signal.

###### num\_records `instance-attribute`

```
num_records: int
```

###### patient\_id `instance-attribute`

```
patient_id: str
```

###### record\_duration `instance-attribute`

```
record_duration: Fraction
```

###### samples\_per\_record `instance-attribute`

```
samples_per_record: tuple[int, ...]
```

###### signals `instance-attribute`

```
signals: tuple[str, ...]
```

###### start\_time `instance-attribute`

```
start_time: datetime
```

###### units `instance-attribute`

```
units: tuple[str, ...]
```

##### build\_signal\_loader

```
build_signal_loader(
    file: EdfFile, index: int
) -> Callable[[], Array]
```

Build the lazy loader of one signal.

The loader holds the open file, thus the signals of one recording share one open file and
one memory map. A build keeps every file it opened open until the writer has called the
loaders.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `file` | `EdfFile` | The open file, from :func:`open_edf`. | *required* |
| `index` | `int` | The signal, in the order of the header. | *required* |

Returns:

| Type | Description |
| --- | --- |
| `Callable[[], Array]` | A loader that takes no argument and gives the signal in physical units. |

##### compute\_signal\_end\_microseconds

```
compute_signal_end_microseconds(header: EdfHeader) -> int
```

Give where the recorded signals stop, in microseconds from the first sample.

The timeline starts at zero, so this is both the length of the signals and the exclusive
end of the timeline. An annotation is measured against this bound.

Every record of the file follows the one before it with no gap, because :func:`open_edf`
refuses a discontinuous file. The records therefore multiply out to the whole timeline.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `header` | `EdfHeader` | The header of a signal file. | *required* |

Returns:

| Type | Description |
| --- | --- |
| `int` | `num_records * record_duration` as whole microseconds. |

Raises:

| Type | Description |
| --- | --- |
| `TimeFFormatError` | If that product is not a whole number of microseconds. |

##### convert\_digital\_to\_physical

```
convert_digital_to_physical(
    digital: ndarray,
    *,
    digital_min: float,
    digital_max: float,
    physical_min: float,
    physical_max: float,
) -> ndarray
```

Convert stored integers into the physical unit that the header names.

EDF stores no volts. It stores counts, and the header of each file states which range of
physical values those counts cover::

```
gain = (physical_max - physical_min) / (digital_max - digital_min)
physical = (digital - digital_min) * gain + physical_min
```

The four values must come from the header of the file that holds these counts.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `digital` | `ndarray` | The stored counts. | *required* |
| `digital_min` | `float` | The smallest count that the signal writes. | *required* |
| `digital_max` | `float` | The largest count that the signal writes. | *required* |
| `physical_min` | `float` | The value that `digital_min` means. | *required* |
| `physical_max` | `float` | The value that `digital_max` means. | *required* |

Returns:

| Type | Description |
| --- | --- |
| `ndarray` | The values in the physical unit of the signal, as float32. |

Raises:

| Type | Description |
| --- | --- |
| `TimeFFormatError` | If the digital range is empty, because it then gives no scale. |

##### open\_edf

```
open_edf(path: Path) -> EdfFile
```

Open an EDF file, read its header, and map its records.

Reading is lazy: a caller that wants the header alone reads no signal bytes. Open a file
one time and pass the result to :func:`read_signal`, so its header is parsed once.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `path` | `Path` | The `.edf` file. | *required* |

Returns:

| Type | Description |
| --- | --- |
| `EdfFile` | The open file, with its header. |

Raises:

| Type | Description |
| --- | --- |
| `TimeFFormatError` | If the file is not readable as EDF, if it is shorter than the records that its own header states, if it is discontinuous, or if it states no start date. |

##### read\_annotations

```
read_annotations(
    file: EdfFile,
) -> tuple[EdfAnnotation, ...]
```

Read every annotation of an EDF+ file, in file order.

An EDF+ file states an onset in seconds as text. Parsing that text with
:class:`~decimal.Decimal` keeps the onset the file states; a float would move it.

Each data record opens with an empty-text annotation that only timestamps the record.
`if annotation.text` drops those.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `file` | `EdfFile` | The open file that holds the annotations, from :func:`open_edf`. An EDF+ file carries its own annotations, thus a caller that already holds the recording reads them from it. A scoring that lives in a file of its own is opened first. | *required* |

Returns:

| Name | Type | Description |
| --- | --- | --- |
| `One` | `tuple[EdfAnnotation, ...]` | class:`EdfAnnotation` for each labelled entry, in microseconds. |

Raises:

| Type | Description |
| --- | --- |
| `TimeFFormatError` | If an onset or a duration is not a whole number of microseconds. |

##### read\_record

```
read_record(
    file: EdfFile, index: int
) -> tuple[ndarray, ...]
```

Read one data record, and give one array for each signal.

The arrays have different lengths when the signals have different rates.

Reading one record reads one record. It does not read the signals it slices, thus a
caller that walks a recording record by record never holds the whole recording.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `file` | `EdfFile` | The open file, from :func:`open_edf`. | *required* |
| `index` | `int` | The record to read, counted from zero. | *required* |

Returns:

| Type | Description |
| --- | --- |
| `tuple[ndarray, ...]` | One array for each signal, in the order of the header, in physical units. |

Raises:

| Type | Description |
| --- | --- |
| `TimeFFormatError` | If the file holds no record with this index. |

##### read\_signal

```
read_signal(file: EdfFile, index: int) -> ndarray
```

Read one whole signal, from its first sample to its last.

Reading one signal does not read the others.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `file` | `EdfFile` | The open file, from :func:`open_edf`. | *required* |
| `index` | `int` | The signal, counted from zero, in the order of the header. | *required* |

Returns:

| Type | Description |
| --- | --- |
| `ndarray` | Every sample of that signal, in physical units. |

Raises:

| Type | Description |
| --- | --- |
| `TimeFFormatError` | If the file holds no signal with this index. |

#### timeseries

Turn the signals of one recording into :class:`~timenet.dataset.Signal`.

The caller passes the signal-to-spec table and the loader factory, thus this module holds
nothing of this dataset and reads no sample. Every other attribute comes from the header.

##### build

```
build(
    file: EdfFile,
    specs: Mapping[str, TimeSeriesSpec],
    *,
    loader: Callable[[EdfFile, int], Callable[[], Array]],
) -> tuple[Signal, ...]
```

Give one time series for each signal that the header of a recording names.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `file` | `EdfFile` | Its open PSG file, from :func:`reader.open_edf`. | *required* |
| `specs` | `Mapping[str, TimeSeriesSpec]` | The table of the release, from a signal name to its spec. | *required* |
| `loader` | `Callable[[EdfFile, int], Callable[[], Array]]` | Builds the lazy loader of one signal, from the open file and a signal index. | *required* |

Returns:

| Type | Description |
| --- | --- |
| `tuple[Signal, ...]` | One time series for each signal, in the order of the header. |

Raises:

| Type | Description |
| --- | --- |
| `TimeFFormatError` | If the header names a signal that the spec table does not hold. |

### excel

Read a legacy `.xls` workbook, and decode the cell values it gives back.

A connector whose source ships a spreadsheet opens it with :func:`read_table_rows` and gets the
rows of the first sheet. The cells come back as the workbook states them, so nothing here knows
what a column means. The connector states that.

The decoders cover what Excel itself states about a cell and nothing more. A whole number is a
whole number in every workbook, and Excel writes a bare time as the fraction of a day past
midnight in every workbook. A code whose meaning changes from one sheet to the next belongs to
the connector that reads that sheet.

Each decoder takes the workbook name and the row number so that its error names the cell a
reader has to go and look at.

#### SECONDS\_PER\_DAY `module-attribute`

```
SECONDS_PER_DAY = 24 * 60 * 60
```

#### decode\_day\_fraction\_as\_time

```
decode_day_fraction_as_time(
    cell: object, workbook: str, row_number: int
) -> time
```

Convert an Excel day fraction into a time of day.

Excel stores a bare time as the fraction of a day past midnight, so 0.5 is 12:00:00 and a
fraction below 0.5 is a time after midnight and not an error.

A string reaches this function only by accident. A column that has become text is a workbook
the caller does not know.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `cell` | `object` | The value of the cell, as the workbook states it. | *required* |
| `workbook` | `str` | The name of the workbook, for the error message. | *required* |
| `row_number` | `int` | The row of the sheet, counted from 1, for the error message. | *required* |

Returns:

| Type | Description |
| --- | --- |
| `time` | The clock time that the fraction states, to the second. |

Raises:

| Type | Description |
| --- | --- |
| `TimeFFormatError` | If the cell holds no number, or a number outside 0 up to 1. |

#### decode\_whole\_number

```
decode_whole_number(
    cell: object, field: str, workbook: str, row_number: int
) -> int
```

Decode the whole number that a cell holds.

Excel writes every number as a float, so a whole number reaches this as `44.0`. A cell that
holds a fraction, a bool or text states no whole number and raises.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `cell` | `object` | The value of the cell, as the workbook states it. | *required* |
| `field` | `str` | The name of the column, for the error message. | *required* |
| `workbook` | `str` | The name of the workbook, for the error message. | *required* |
| `row_number` | `int` | The row of the sheet, counted from 1, for the error message. | *required* |

Returns:

| Type | Description |
| --- | --- |
| `int` | The value as a whole number. |

Raises:

| Type | Description |
| --- | --- |
| `TimeFFormatError` | If the cell holds no whole number. |

#### read\_table\_rows

```
read_table_rows(path: Path) -> list[tuple[object, ...]]
```

Open a workbook and give every row of its first sheet, header rows included.

This gives the cells as the workbook states them and decodes nothing.

A workbook can carry more than one sheet. This takes the first and searches for no sheet by
name, because the first sheet is where a source that ships one table puts it.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `path` | `Path` | The `.xls` workbook. | *required* |

Returns:

| Type | Description |
| --- | --- |
| `list[tuple[object, ...]]` | One tuple of cell values for each row of the first sheet, in sheet order. |

Raises:

| Type | Description |
| --- | --- |
| `TimeFFormatError` | If the file does not open as a workbook, or holds no sheet. |

### huggingface

A reusable base for connectors that read files from a pinned HuggingFace Hub dataset revision.

Subclasses set `HF_REPO`, `SOURCE_REVISION`, and `SOURCE_FILES`, ship a `dataset.yaml` card
beside the connector, and implement `convert()` over the :class:`HubFileRef` list `download()`
returns. The base checks that list before the subclass's `convert()` runs.
:mod:`timenet_connectors.sources.huggingface_hub` describes how the files download. For the Hub's
parquet conversion, pin the commit `refs/convert/parquet` points to.

#### BaseHuggingFaceConnector

Bases: `BaseConnector[HubFileRef]`, `ABC`

Base class for connectors that read files from a pinned Hub dataset revision.

Subclasses set :attr:`HF_REPO`, :attr:`SOURCE_REVISION`, and :attr:`SOURCE_FILES`, and implement
`convert()` over the returned :class:`HubFileRef` list. The base wraps each subclass's `convert()`
to run :meth:`check_refs` first.

##### CARD `class-attribute`

```
CARD: str | Path | None = None
```

Optional explicit path to the dataset card YAML. When `None` (the default), the connector
reads the card from `dataset.yaml` in its own folder.

##### HF\_REPO `class-attribute`

```
HF_REPO: str
```

##### SOURCE\_FILES `class-attribute`

```
SOURCE_FILES: Mapping[str, str]
```

The repo-relative filename of each part, in download-order. A filename can end
in a glob like `data/*.parquet`; the part's path is then the directory of the matching files.

##### SOURCE\_REVISION `class-attribute`

```
SOURCE_REVISION: str
```

An immutable commit: a mutable branch can silently change rebuilt datasets.

##### values\_backend `class-attribute` `instance-attribute`

```
values_backend: ValuesBackend = ValuesBackend.PARQUET
```

Storage backend for the dataset's time-series values.

##### check\_refs

```
check_refs(raw_refs: Sequence[HubFileRef]) -> None
```

Check that `raw_refs` is nonempty and names each part once, only parts :attr:`SOURCE_FILES` declares.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `raw_refs` | `Sequence[HubFileRef]` | The references `convert()` received. | *required* |

Raises:

| Type | Description |
| --- | --- |
| `TimeFValidationError` | If `raw_refs` is empty, repeats a part, or names an undeclared part. |

##### compose

```
compose(
    raw_refs: list[TRaw], context: BuildContext
) -> TimeFDataset
```

Build the dataset from raw references and the opened parent releases.

A connector whose card declares `parents` must override this hook. The connector reads
each parent through `context.parents[alias]` and copies objects with
`dataset.import_from(...)`. After this hook returns, the engine records each declared
parent in the dataset: its card and its lineage. The default ignores `context` and calls
:meth:`convert`.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `raw_refs` | `list[TRaw]` | The references returned by :meth:`download`. | *required* |
| `context` | `BuildContext` | The opened parents, keyed by the parent aliases on the card. | *required* |

Returns:

| Type | Description |
| --- | --- |
| `TimeFDataset` | The populated dataset. |

##### convert

```
convert(raw_refs: list[TRaw]) -> TimeFDataset
```

Parse raw references and populate a :class:`~timenet.dataset.TimeFDataset`.

CPU-bound: no network I/O. Attach time-series values as lazy loaders instead of materializing
them.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `raw_refs` | `list[TRaw]` | The references returned by :meth:`download`. | *required* |

Returns:

| Type | Description |
| --- | --- |
| `TimeFDataset` | The populated dataset. |

Raises:

| Type | Description |
| --- | --- |
| `NotImplementedError` | If called on a connector that only implements `compose`. |

##### download

```
download(cache_dir: Path) -> list[TRaw]
```

Fetch or discover raw source files and return lightweight references to them.

I/O only: no parsing, no array work. Must be idempotent for a given `cache_dir`. Override
this for a synchronous connector. For an I/O-bound one, override :meth:`download_async`
instead and leave this default, which drives it to completion, since the engine calls
connectors synchronously.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `cache_dir` | `Path` | Directory to write downloaded files into (created by the engine). | *required* |

Returns:

| Type | Description |
| --- | --- |
| `list[TRaw]` | Raw references passed directly to :meth:`convert`. |

##### download\_async `async`

```
download_async(cache_dir: Path) -> list[HubFileRef]
```

Download :attr:`SOURCE_FILES` at :attr:`SOURCE_REVISION` concurrently.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `cache_dir` | `Path` | Directory where the connector caches Hub files. | *required* |

Returns:

| Type | Description |
| --- | --- |
| `list[HubFileRef]` | One reference per part, in :attr:`SOURCE_FILES` order. |

Raises:

| Type | Description |
| --- | --- |
| `ImportError` | If `huggingface_hub`, declared in the connector's requirements, is missing. |
| `TimeNetDownloadError` | If a declared file is absent from the revision or cannot be downloaded. |

##### metadata

```
metadata() -> DatasetMetadata
```

Return the dataset's descriptive identity, loaded and validated from its card YAML.

Reads the card by convention (`dataset.yaml` beside the connector, unless :attr:`CARD`
overrides it). Its `dataset_id` must match the id used to register or build the connector.

Returns:

| Type | Description |
| --- | --- |
| `DatasetMetadata` | The dataset's :class:`~timenet.types.DatasetMetadata`. |

##### ref\_paths

```
ref_paths(
    raw_refs: Sequence[HubFileRef], *parts: str
) -> dict[str, Path]
```

Map each part in `raw_refs` to its local path.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `raw_refs` | `Sequence[HubFileRef]` | The references `convert()` received. | *required* |
| `*parts` | `str` | The parts the caller needs. | `()` |

Returns:

| Type | Description |
| --- | --- |
| `dict[str, Path]` | The path of each part: a file, or the directory of a pattern part. |

Raises:

| Type | Description |
| --- | --- |
| `TimeNetConnectorError` | If one of `parts` is not in `raw_refs`. |

#### HubFileRef `dataclass`

A downloaded file (or a pattern's directory) and the part it belongs to.

##### part `instance-attribute`

```
part: str
```

The connector's own name for this file, such as `"hhar_v2"`: the key in `SOURCE_FILES`, which
`convert()` branches on and builds ids from.

##### path `instance-attribute`

```
path: Path
```

### physionet

A reusable base class for connectors that read WFDB records from PhysioNet.

A subclass downloads a PhysioNet database archive with :func:`~timenet_connectors.download.ensure_archive`
and reads the records. Headers are parsed straight from the `.hea` text, and format-16 signals are read
directly from the `.dat` with NumPy; `wfdb` is used only as a fallback for signal formats the direct
reader does not handle. `wfdb` is declared in the connector's `requirements.txt` and installed into the
environment the build runs in, and the base class imports it lazily, so `--no-isolation` stays usable and
a build that never reaches the fallback needs nothing extra.

#### TRaw `module-attribute`

```
TRaw = TypeVar('TRaw')
```

#### BasePhysioNetConnector

Bases: `BaseConnector[TRaw]`, `ABC`

The base class for PhysioNet-backed connectors that read WFDB records.

##### CARD `class-attribute`

```
CARD: str | Path | None = None
```

Optional explicit path to the dataset card YAML. When `None` (the default), the connector
reads the card from `dataset.yaml` in its own folder.

##### values\_backend `class-attribute` `instance-attribute`

```
values_backend: ValuesBackend = ValuesBackend.PARQUET
```

Storage backend for the dataset's time-series values.

##### compose

```
compose(
    raw_refs: list[TRaw], context: BuildContext
) -> TimeFDataset
```

Build the dataset from raw references and the opened parent releases.

A connector whose card declares `parents` must override this hook. The connector reads
each parent through `context.parents[alias]` and copies objects with
`dataset.import_from(...)`. After this hook returns, the engine records each declared
parent in the dataset: its card and its lineage. The default ignores `context` and calls
:meth:`convert`.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `raw_refs` | `list[TRaw]` | The references returned by :meth:`download`. | *required* |
| `context` | `BuildContext` | The opened parents, keyed by the parent aliases on the card. | *required* |

Returns:

| Type | Description |
| --- | --- |
| `TimeFDataset` | The populated dataset. |

##### convert

```
convert(raw_refs: list[TRaw]) -> TimeFDataset
```

Parse raw references and populate a :class:`~timenet.dataset.TimeFDataset`.

CPU-bound: no network I/O. Attach time-series values as lazy loaders instead of materializing
them.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `raw_refs` | `list[TRaw]` | The references returned by :meth:`download`. | *required* |

Returns:

| Type | Description |
| --- | --- |
| `TimeFDataset` | The populated dataset. |

Raises:

| Type | Description |
| --- | --- |
| `NotImplementedError` | If called on a connector that only implements `compose`. |

##### download

```
download(cache_dir: Path) -> list[TRaw]
```

Fetch or discover raw source files and return lightweight references to them.

I/O only: no parsing, no array work. Must be idempotent for a given `cache_dir`. Override
this for a synchronous connector. For an I/O-bound one, override :meth:`download_async`
instead and leave this default, which drives it to completion, since the engine calls
connectors synchronously.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `cache_dir` | `Path` | Directory to write downloaded files into (created by the engine). | *required* |

Returns:

| Type | Description |
| --- | --- |
| `list[TRaw]` | Raw references passed directly to :meth:`convert`. |

##### download\_async `async`

```
download_async(cache_dir: Path) -> list[TRaw]
```

Async variant of :meth:`download` for connectors whose downloads are I/O-bound.

Override this to fetch artifacts concurrently, for example with the connector HTTP download
helpers. The default :meth:`download` runs it for you. Implement exactly one of the two.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `cache_dir` | `Path` | Directory to write downloaded files into (created by the engine). | *required* |

Returns:

| Type | Description |
| --- | --- |
| `list[TRaw]` | Raw references passed directly to :meth:`convert`. |

Raises:

| Type | Description |
| --- | --- |
| `NotImplementedError` | If a subclass overrides neither :meth:`download` nor :meth:`download_async`. |

##### metadata

```
metadata() -> DatasetMetadata
```

Return the dataset's descriptive identity, loaded and validated from its card YAML.

Reads the card by convention (`dataset.yaml` beside the connector, unless :attr:`CARD`
overrides it). Its `dataset_id` must match the id used to register or build the connector.

Returns:

| Type | Description |
| --- | --- |
| `DatasetMetadata` | The dataset's :class:`~timenet.types.DatasetMetadata`. |

#### WfdbHeader

Bases: `NamedTuple`

The header fields a PhysioNet connector reads, parsed straight from the `.hea` text.

`dat_names`/`formats`/`gains`/`baselines` are per signal, in signal order; the direct
`.dat` reader uses them to turn raw ADC samples into physical units.

##### baselines `instance-attribute`

```
baselines: tuple[int, ...]
```

##### dat\_names `instance-attribute`

```
dat_names: tuple[str, ...]
```

##### direct\_read `instance-attribute`

```
direct_read: bool
```

Whether the whole record can be read by the direct format-16 path (see :func:`_read_dat16`).
False routes to `wfdb`, which handles the parts the direct reader does not.

##### formats `instance-attribute`

```
formats: tuple[str, ...]
```

##### fs `instance-attribute`

```
fs: float
```

##### gains `instance-attribute`

```
gains: tuple[float, ...]
```

##### sig\_len `instance-attribute`

```
sig_len: int
```

##### sig\_name `instance-attribute`

```
sig_name: list[str]
```