# Engine

## timenet.engine

The curation engine: drive a connector through `download -> convert -> derive_schema -> store`.

### run\_pipeline

```
run_pipeline(
    connector: BaseConnector,
    root: Path,
    *,
    cache_dir: Path | None = None,
    clean_cache: bool = False,
    progress_cb: Callable[[WriteProgressEvent], None]
    | None = None,
    force: bool = False,
) -> Path
```

Run one connector through the full curation pipeline and return the version directory.

Idempotent: if the target version is already committed, the expensive `download` / `convert` /
`store` stages are skipped and the existing directory is returned. Pass `force` to rebuild it.
Otherwise the stages are: create the cache directory, `download` raw references into it,
`convert` them into a dataset, `derive_schema`, then `store`. The engine only writes local
files; publishing to a remote registry is a separate step.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `connector` | `BaseConnector` | The connector to curate. | *required* |
| `root` | `Path` | Output root; the dataset is written to `<root>/<dataset_id>/<version>/`. | *required* |
| `cache_dir` | `Path | None` | Directory for downloaded artifacts (defaults to `<TIMENET_CACHE>/<dataset_id>`). | `None` |
| `clean_cache` | `bool` | Remove the cache directory once the dataset is stored. The raw sources are only needed during conversion, so this reclaims disk after a successful build (they re-download on the next run). | `False` |
| `progress_cb` | `Callable[[WriteProgressEvent], None] | None` | Optional writer progress callback. | `None` |
| `force` | `bool` | Rebuild even if the version is already committed. | `False` |

Returns:

| Type | Description |
| --- | --- |
| `Path` | The committed version directory. |