Usage¶

Every dataset loads the same way. Then it hands off to your framework. There are two entry points:

• TimeNet().load("org/name") returns an in-memory TimeFDataset with lazy per-series values. Use it for single-node work (pandas, polars, torch).
• TimeNet().download("org/name") returns the local TimeF version directory. Its control tables are Parquet. Its values plane is Parquet or Zarr, as recorded in the manifest.

Example status:

• pandas: load a record's series into a DataFrame
• polars: pl.from_arrow over to_arrow()
• PyTorch: load_torch plus a DataLoader
• Spark: planned

Each series carries its own signal and time_axis. It reads its values lazily through to_arrow() / to_numpy(). The framework examples below all start from one loaded record.

import numpy as np
import pandas as pd
from timenet.client import TimeNet

# read in place through the registry (lazy per-series values)
dataset = TimeNet().load("chengsenwang/tsqa")
series = dataset.records[0].time_series[0]

values = series.to_numpy()  # shape: (n_steps, *series.spec.value_shape)
# tsqa is an ordinal series: it has an order and no timeline, so there is no time
# column to build. A regularly sampled series would use series.time_axis.time_offset_us(i).
frame = pd.DataFrame({series.signal: values})

import polars as pl
from timenet.client import TimeNet

dataset = TimeNet().load("chengsenwang/tsqa")
series = dataset.records[0].time_series[0]

# pl.from_arrow reads the Arrow array into a polars Series without a copy.
column = pl.from_arrow(series.to_arrow())
frame = pl.DataFrame({series.signal: column})

from torch.utils.data import DataLoader
from timenet.client import TimeNet

# needs: pip install 'timenet[torch]'
ds = TimeNet().load_torch("chengsenwang/tsqa")
item = ds[0]
series, prompt = item["series"][0], item["tasks"][0].prompt

# Series lengths vary between records, so batch with a collate_fn that picks
# out what the model needs.
loader = DataLoader(
    ds,
    batch_size=8,
    collate_fn=lambda batch: [
        (x["series"][0], x["tasks"][0].target) for x in batch
    ],
)

Example: train a classifier end-to-end¶

One script does the whole loop: build a dataset, load it, train a model. The timenet/test-mean demo is simple. Each record is one noisy signal. The label is above_zero or below_zero, by the sign of the mean. So a classifier only must recover that sign.

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

from timenet.client import TimeNet
import timenet_connectors

# Build the connector's dataset into the local registry (the producer
# side), then load it back.
timenet_connectors.build("timenet/test-mean")
dataset = TimeNet().load("timenet/test-mean")

# Pair each record's values with its target. Materialization is deferred by
# default (Arrow); ask for output="numpy" since scikit-learn needs it.
x, y = dataset.to_features_and_targets(output="numpy")

x_train, x_test, y_train, y_test = train_test_split(
    x, y, test_size=0.25, stratify=y, random_state=0
)
model = LogisticRegression(max_iter=1000).fit(x_train, y_train)
print(f"test accuracy: {model.score(x_test, y_test):.3f}")   # -> 1.000

to_features_and_targets defers materialization. output="arrow" (the default) hands back a FixedSizeListArray and a string array with no NumPy copy. The example asks for output="numpy" because scikit-learn needs it. It also takes features="series". This returns one variable-length sequence per record (a ListArray / object array) instead of the rectangular "timestep" matrix. test-mean has a single task type, so task is inferred here. When a dataset carries several tasks, pass task=....

The full runnable version is examples/test_mean_classifier.py.