Skip to content

Bases

timenet_connectors.bases source

Reusable connector base classes.

edf source

Read EDF and EDF+ files for the connectors that build datasets from them.

reader source

Read the EDF container: the header of a file, and one data record from it.

EDF holds an ASCII header and then data records of 16-bit integers. One record holds the samples of every signal for one stretch of time, one signal after the other. Signals of different rates therefore hold a different count of samples in the same record.

This module parses no bytes. edfio does that. A connector that reads EDF declares edfio in its requirements.txt.

:class:EdfHeader and :class:EdfFile are the boundary: edfio types stay inside this file, thus a later change of library does not reach the modules that call it.

EdfAnnotation source

Bases: NamedTuple

One annotation of an EDF+ file: a labelled stretch of the recording timeline.

duration_microseconds instance-attribute source
duration_microseconds: int
label instance-attribute source
label: str
onset_microseconds instance-attribute source
onset_microseconds: int
EdfFile source

Bases: NamedTuple

An open EDF file: its header, and the edfio.Edf handle behind it.

handle instance-attribute source
handle: Any
header instance-attribute source
header: EdfHeader
path instance-attribute source
path: Path
EdfHeader source

Bases: NamedTuple

The header of one EDF file. Each tuple below holds one entry for each signal.

num_records instance-attribute source
num_records: int
patient_id instance-attribute source
patient_id: str
record_duration instance-attribute source
record_duration: Fraction
samples_per_record instance-attribute source
samples_per_record: tuple[int, ...]
signals instance-attribute source
signals: tuple[str, ...]
start_time instance-attribute source
start_time: datetime
units instance-attribute source
units: tuple[str, ...]
build_signal_loader source
build_signal_loader(
    file: EdfFile, index: int
) -> Callable[[], Array]

Build the lazy loader of one signal.

The loader holds the open file, thus the signals of one recording share one open file and one memory map. A build keeps every file it opened open until the writer has called the loaders.

Parameters:

Name Type Description Default
file EdfFile

The open file, from :func:open_edf.

 required
index int

The signal, in the order of the header.

 required

Returns:

Type Description
Callable[[], Array]

A loader that takes no argument and gives the signal in physical units.
compute_signal_end_microseconds source
compute_signal_end_microseconds(header: EdfHeader) -> int

Give where the recorded signals stop, in microseconds from the first sample.

The timeline starts at zero, so this is both the length of the signals and the exclusive end of the timeline. An annotation is measured against this bound.

Every record of the file follows the one before it with no gap, because :func:open_edf refuses a discontinuous file. The records therefore multiply out to the whole timeline.

Parameters:

Name Type Description Default
header EdfHeader

The header of a signal file.

 required

Returns:

Type Description
int

num_records * record_duration as whole microseconds.

Raises:

Type Description
TimeFFormatError

If that product is not a whole number of microseconds.
convert_digital_to_physical source
convert_digital_to_physical(
    digital: ndarray,
    *,
    digital_min: float,
    digital_max: float,
    physical_min: float,
    physical_max: float,
) -> ndarray

Convert stored integers into the physical unit that the header names.

EDF stores no volts. It stores counts, and the header of each file states which range of physical values those counts cover::

gain = (physical_max - physical_min) / (digital_max - digital_min)
physical = (digital - digital_min) * gain + physical_min

The four values must come from the header of the file that holds these counts.

Parameters:

Name Type Description Default
digital ndarray

The stored counts.

 required
digital_min float

The smallest count that the signal writes.

 required
digital_max float

The largest count that the signal writes.

 required
physical_min float

The value that digital_min means.

 required
physical_max float

The value that digital_max means.

 required

Returns:

Type Description
ndarray

The values in the physical unit of the signal, as float32.

Raises:

Type Description
TimeFFormatError

If the digital range is empty, because it then gives no scale.
open_edf source
open_edf(path: Path) -> EdfFile

Open an EDF file, read its header, and map its records.

Reading is lazy: a caller that wants the header alone reads no signal bytes. Open a file one time and pass the result to :func:read_signal, so its header is parsed once.

Parameters:

Name Type Description Default
path Path

The .edf file.

 required

Returns:

Type Description
EdfFile

The open file, with its header.

Raises:

Type Description
TimeFFormatError

If the file is not readable as EDF, if it is shorter than the records that its own header states, if it is discontinuous, or if it states no start date.
read_annotations source
read_annotations(
    file: EdfFile,
) -> tuple[EdfAnnotation, ...]

Read every annotation of an EDF+ file, in file order.

An EDF+ file states an onset in seconds as text. Parsing that text with :class:~decimal.Decimal keeps the onset the file states; a float would move it.

Each data record opens with an empty-text annotation that only timestamps the record. if annotation.text drops those.

Parameters:

Name Type Description Default
file EdfFile

The open file that holds the annotations, from :func:open_edf. An EDF+ file carries its own annotations, thus a caller that already holds the recording reads them from it. A scoring that lives in a file of its own is opened first.

 required

Returns:

Name Type Description
One tuple[EdfAnnotation, ...]

class:EdfAnnotation for each labelled entry, in microseconds.

Raises:

Type Description
TimeFFormatError

If an onset or a duration is not a whole number of microseconds.
read_record source
read_record(
    file: EdfFile, index: int
) -> tuple[ndarray, ...]

Read one data record, and give one array for each signal.

The arrays have different lengths when the signals have different rates.

Reading one record reads one record. It does not read the signals it slices, thus a caller that walks a recording record by record never holds the whole recording.

Parameters:

Name Type Description Default
file EdfFile

The open file, from :func:open_edf.

 required
index int

The record to read, counted from zero.

 required

Returns:

Type Description
tuple[ndarray, ...]

One array for each signal, in the order of the header, in physical units.

Raises:

Type Description
TimeFFormatError

If the file holds no record with this index.
read_signal source
read_signal(file: EdfFile, index: int) -> ndarray

Read one whole signal, from its first sample to its last.

Reading one signal does not read the others.

Parameters:

Name Type Description Default
file EdfFile

The open file, from :func:open_edf.

 required
index int

The signal, counted from zero, in the order of the header.

 required

Returns:

Type Description
ndarray

Every sample of that signal, in physical units.

Raises:

Type Description
TimeFFormatError

If the file holds no signal with this index.

timeseries source

Turn the signals of one recording into :class:~timenet.dataset.TimeSeries.

The caller passes the signal-to-spec table and the loader factory, thus this module holds nothing of this dataset and reads no sample. Every other attribute comes from the header.

build source
build(
    record_id: str,
    file: EdfFile,
    specs: Mapping[str, TimeSeriesSpec],
    *,
    loader: Callable[[EdfFile, int], Callable[[], Array]],
) -> tuple[TimeSeries, ...]

Give one time series for each signal that the header of a recording names.

Parameters:

Name Type Description Default
record_id str

The id of the record these signals belong to, from connector.py. source_id and each time_series_id are built from it.

 required
file EdfFile

Its open PSG file, from :func:reader.open_edf.

 required
specs Mapping[str, TimeSeriesSpec]

The table of the release, from a signal name to its spec.

 required
loader Callable[[EdfFile, int], Callable[[], Array]]

Builds the lazy loader of one signal, from the open file and a signal index.

 required

Returns:

Type Description
tuple[TimeSeries, ...]

One time series for each signal, in the order of the header.

Raises:

Type Description
TimeFFormatError

If the header names a signal that the spec table does not hold.

excel source

Read a legacy .xls workbook, and decode the cell values it gives back.

A connector whose source ships a spreadsheet opens it with :func:read_table_rows and gets the rows of the first sheet. The cells come back as the workbook states them, so nothing here knows what a column means. The connector states that.

The decoders cover what Excel itself states about a cell and nothing more. A whole number is a whole number in every workbook, and Excel writes a bare time as the fraction of a day past midnight in every workbook. A code whose meaning changes from one sheet to the next belongs to the connector that reads that sheet.

Each decoder takes the workbook name and the row number so that its error names the cell a reader has to go and look at.

SECONDS_PER_DAY module-attribute source

SECONDS_PER_DAY = 24 * 60 * 60

decode_day_fraction_as_time source

decode_day_fraction_as_time(
    cell: object, workbook: str, row_number: int
) -> time

Convert an Excel day fraction into a time of day.

Excel stores a bare time as the fraction of a day past midnight, so 0.5 is 12:00:00 and a fraction below 0.5 is a time after midnight and not an error.

A string reaches this function only by accident. A column that has become text is a workbook the caller does not know.

Parameters:

Name Type Description Default
cell object

The value of the cell, as the workbook states it.

 required
workbook str

The name of the workbook, for the error message.

 required
row_number int

The row of the sheet, counted from 1, for the error message.

 required

Returns:

Type Description
time

The clock time that the fraction states, to the second.

Raises:

Type Description
TimeFFormatError

If the cell holds no number, or a number outside 0 up to 1.

decode_whole_number source

decode_whole_number(
    cell: object, field: str, workbook: str, row_number: int
) -> int

Decode the whole number that a cell holds.

Excel writes every number as a float, so a whole number reaches this as 44.0. A cell that holds a fraction, a bool or text states no whole number and raises.

Parameters:

Name Type Description Default
cell object

The value of the cell, as the workbook states it.

 required
field str

The name of the column, for the error message.

 required
workbook str

The name of the workbook, for the error message.

 required
row_number int

The row of the sheet, counted from 1, for the error message.

 required

Returns:

Type Description
int

The value as a whole number.

Raises:

Type Description
TimeFFormatError

If the cell holds no whole number.

read_table_rows source

read_table_rows(path: Path) -> list[tuple[object, ...]]

Open a workbook and give every row of its first sheet, header rows included.

This gives the cells as the workbook states them and decodes nothing.

A workbook can carry more than one sheet. This takes the first and searches for no sheet by name, because the first sheet is where a source that ships one table puts it.

Parameters:

Name Type Description Default
path Path

The .xls workbook.

 required

Returns:

Type Description
list[tuple[object, ...]]

One tuple of cell values for each row of the first sheet, in sheet order.

Raises:

Type Description
TimeFFormatError

If the file does not open as a workbook, or holds no sheet.

huggingface source

A reusable base for connectors that pull rows from a HuggingFace Hub dataset.

Subclasses set HF_REPO, ship a dataset.yaml card beside the connector, and implement convert(). Downloads read the Hub's auto-generated parquet ref. The Hub produces that ref for public and gated datasets. huggingface_hub is declared in the connector's requirements.txt and installed into the environment the build runs in; the base imports it lazily, which is what keeps --no-isolation usable. The base reads HF_TOKEN from the environment, so gated datasets work with no extra wiring. Fully private datasets have no auto-parquet ref, and this base does not support them.

BaseHuggingFaceConnector source

Bases: BaseConnector[dict[str, Any]], ABC

Base class for HuggingFace-backed connectors. Rows are plain dicts (one per dataset row).

CARD class-attribute 
CARD: str | Path | None = None

Optional explicit path to the dataset card YAML. When None (the default), the connector reads the card from dataset.yaml in its own folder.

HF_REPO class-attribute source
HF_REPO: str
PARQUET_REVISION class-attribute source
PARQUET_REVISION: str = 'refs/convert/parquet'
values_backend class-attribute instance-attribute 
values_backend: str = 'parquet'

Default storage backend for this connector's values plane.

convert abstractmethod 
convert(raw_refs: list[TRaw]) -> TimeFDataset

Parse raw references and populate a :class:~timenet.dataset.TimeFDataset.

CPU-bound: no network I/O. Attach time-series values as lazy loaders instead of materializing them.

Parameters:

Name Type Description Default
raw_refs list[TRaw]

The references returned by :meth:download.

 required

Returns:

Type Description
TimeFDataset

The populated dataset.
download source
download(cache_dir: Path) -> list[dict[str, Any]]

Download the repo's auto-converted parquet file(s) and return their rows.

Reads the Hub's refs/convert/parquet branch (see :attr:PARQUET_REVISION), so it handles any source format the same way. For very large datasets the Hub conversion can be partial.

Parameters:

Name Type Description Default
cache_dir Path

Directory where the connector caches Hub files.

 required

Returns:

Type Description
list[dict[str, Any]]

One dict per row across all parquet files.

Raises:

Type Description
ImportError

If huggingface_hub, declared in this connector's requirements, is missing.
TimeNetDatasetNotFoundError

If the revision holds no parquet files, or they hold no rows.
download_async async 
download_async(cache_dir: Path) -> list[TRaw]

Async variant of :meth:download for connectors whose downloads are I/O-bound.

Override this to fetch artifacts concurrently, for example with the connector HTTP download helpers. The default :meth:download runs it for you. Implement exactly one of the two.

Parameters:

Name Type Description Default
cache_dir Path

Directory to write downloaded files into (created by the engine).

 required

Returns:

Type Description
list[TRaw]

Raw references passed directly to :meth:convert.

Raises:

Type Description
NotImplementedError

If a subclass overrides neither :meth:download nor :meth:download_async.
metadata 
metadata() -> DatasetMetadata

Return the dataset's descriptive identity, loaded and validated from its card YAML.

Reads the card by convention (dataset.yaml beside the connector, unless :attr:CARD overrides it). Its dataset_id must match the id used to register or build the connector.

Returns:

Type Description
DatasetMetadata

The dataset's :class:~timenet.types.DatasetMetadata.

physionet source

A reusable base class for connectors that read WFDB records from PhysioNet.

A subclass downloads a PhysioNet database archive with :func:~timenet_connectors.download.ensure_archive and reads the records. Headers are parsed straight from the .hea text, and format-16 signals are read directly from the .dat with NumPy; wfdb is used only as a fallback for signal formats the direct reader does not handle. wfdb is declared in the connector's requirements.txt and installed into the environment the build runs in, and the base class imports it lazily, so --no-isolation stays usable and a build that never reaches the fallback needs nothing extra.

TRaw module-attribute source

TRaw = TypeVar('TRaw')

BasePhysioNetConnector source

Bases: BaseConnector[TRaw], ABC

The base class for PhysioNet-backed connectors that read WFDB records.

CARD class-attribute 
CARD: str | Path | None = None

Optional explicit path to the dataset card YAML. When None (the default), the connector reads the card from dataset.yaml in its own folder.

values_backend class-attribute instance-attribute 
values_backend: str = 'parquet'

Default storage backend for this connector's values plane.

convert abstractmethod 
convert(raw_refs: list[TRaw]) -> TimeFDataset

Parse raw references and populate a :class:~timenet.dataset.TimeFDataset.

CPU-bound: no network I/O. Attach time-series values as lazy loaders instead of materializing them.

Parameters:

Name Type Description Default
raw_refs list[TRaw]

The references returned by :meth:download.

 required

Returns:

Type Description
TimeFDataset

The populated dataset.
download 
download(cache_dir: Path) -> list[TRaw]

Fetch or discover raw source files and return lightweight references to them.

I/O only: no parsing, no array work. Must be idempotent for a given cache_dir. Override this for a synchronous connector. For an I/O-bound one, override :meth:download_async instead and leave this default, which drives it to completion, since the engine calls connectors synchronously.

Parameters:

Name Type Description Default
cache_dir Path

Directory to write downloaded files into (created by the engine).

 required

Returns:

Type Description
list[TRaw]

Raw references passed directly to :meth:convert.
download_async async 
download_async(cache_dir: Path) -> list[TRaw]

Async variant of :meth:download for connectors whose downloads are I/O-bound.

Override this to fetch artifacts concurrently, for example with the connector HTTP download helpers. The default :meth:download runs it for you. Implement exactly one of the two.

Parameters:

Name Type Description Default
cache_dir Path

Directory to write downloaded files into (created by the engine).

 required

Returns:

Type Description
list[TRaw]

Raw references passed directly to :meth:convert.

Raises:

Type Description
NotImplementedError

If a subclass overrides neither :meth:download nor :meth:download_async.
metadata 
metadata() -> DatasetMetadata

Return the dataset's descriptive identity, loaded and validated from its card YAML.

Reads the card by convention (dataset.yaml beside the connector, unless :attr:CARD overrides it). Its dataset_id must match the id used to register or build the connector.

Returns:

Type Description
DatasetMetadata

The dataset's :class:~timenet.types.DatasetMetadata.

WfdbHeader source

Bases: NamedTuple

The header fields a PhysioNet connector reads, parsed straight from the .hea text.

dat_names/formats/gains/baselines are per signal, in signal order; the direct .dat reader uses them to turn raw ADC samples into physical units.

baselines instance-attribute source
baselines: tuple[int, ...]
dat_names instance-attribute source
dat_names: tuple[str, ...]
direct_read instance-attribute source
direct_read: bool

Whether the whole record can be read by the direct format-16 path (see :func:_read_dat16). False routes to wfdb, which handles the parts the direct reader does not.

formats instance-attribute source
formats: tuple[str, ...]
fs instance-attribute source
fs: float
gains instance-attribute source
gains: tuple[float, ...]
sig_len instance-attribute source
sig_len: int
sig_name instance-attribute source
sig_name: list[str]