# Download

## timenet\_connectors.download

Async, scheme-dispatching artifact downloads for connectors.

The high-level entry points are :func:`download_files` (a list of :class:`Artifact`, any mix of `s3://`
and `http(s)://`) and :func:`ensure_archive` (download a zip and extract it once). Downloads report
progress through the ambient :func:`progress_sink`. The per-transport backends live in :mod:`.http` and
:mod:`.s3`.

### ProgressCallback `module-attribute`

```
ProgressCallback = Callable[[DownloadProgress], None]
```

### Artifact `dataclass`

One download: a URL, its destination, and optional headers, cookies, and a SHA-256 to verify.

The `headers`, `cookies`, and `sha256` apply to an HTTP download. An `s3://` URL
ignores them.

#### cookies `class-attribute` `instance-attribute`

```
cookies: Mapping[str, str] | None = None
```

#### dest `instance-attribute`

```
dest: Path
```

#### headers `class-attribute` `instance-attribute`

```
headers: Mapping[str, str] | None = None
```

#### sha256 `class-attribute` `instance-attribute`

```
sha256: str | None = None
```

#### url `instance-attribute`

```
url: str
```

### DownloadProgress `dataclass`

Progress for one download: bytes transferred so far and the total, if known.

#### downloaded `instance-attribute`

```
downloaded: int
```

#### total `instance-attribute`

```
total: int | None
```

#### url `instance-attribute`

```
url: str
```

### download\_files `async`

```
download_files(
    artifacts: Iterable[Artifact],
    *,
    headers: Mapping[str, str] | None = None,
    cookies: Mapping[str, str] | None = None,
    max_concurrency: int = _DEFAULT_MAX_CONCURRENCY,
    skip_existing: bool = True,
) -> list[Path]
```

Download a list of artifacts, mixing `s3://` and `http(s)://` freely.

HTTP entries download concurrently (bounded by `max_concurrency`, sharing one connection pool). S3
entries download one at a time, since boto3 blocks the event loop but parallelizes each transfer
itself. Batch `headers` and `cookies` apply to every HTTP request, under each artifact's own
(ignored for S3). The function validates every scheme up front, so an unsupported one fails before
any download starts.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `artifacts` | `Iterable[Artifact]` | The artifacts to download. A single file is a one-element list. | *required* |
| `headers` | `Mapping[str, str] | None` | Headers applied to every HTTP request, under each artifact's own. | `None` |
| `cookies` | `Mapping[str, str] | None` | Cookies applied to every HTTP request, under each artifact's own. | `None` |
| `max_concurrency` | `int` | Maximum number of concurrent HTTP downloads. | `_DEFAULT_MAX_CONCURRENCY` |
| `skip_existing` | `bool` | Skip any artifact whose destination already exists. | `True` |

Returns:

| Type | Description |
| --- | --- |
| `list[Path]` | The destination paths, in input order. |

Raises:

| Type | Description |
| --- | --- |
| `TimeFValidationError` | If any URL is neither an `s3://` nor an `http(s)://` URL. |

### ensure\_archive `async`

```
ensure_archive(
    url: str,
    target: str | Path,
    *,
    filename: str | None = None,
    headers: Mapping[str, str] | None = None,
    cookies: Mapping[str, str] | None = None,
    sha256: str | None = None,
) -> Path
```

Download a zip archive (`s3://` or `http(s)://`) and extract it into `target` once.

Idempotent: the function writes a marker file under `target`, keyed by the archive URL, after a
successful extraction. A re-run then reuses the extracted contents and skips the download.

A successful extraction then deletes the archive, because the marker alone makes the next run
cheap. A failed one keeps it, so the next run extracts it again instead of downloading it again.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `url` | `str` | The archive URL, `s3://` or `http(s)://`. | *required* |
| `target` | `str | Path` | Directory into which the function downloads the archive and extracts its contents. | *required* |
| `filename` | `str | None` | Overrides the cached archive name, for URLs whose path has no usable filename (a trailing slash or a `/download` suffix). | `None` |
| `headers` | `Mapping[str, str] | None` | Request headers for an HTTP download (ignored for S3). | `None` |
| `cookies` | `Mapping[str, str] | None` | Request cookies for an HTTP download (ignored for S3). | `None` |
| `sha256` | `str | None` | Optional hex digest the downloaded archive must match (HTTP only). | `None` |

Returns:

| Type | Description |
| --- | --- |
| `Path` | `target` (the extraction root). |

### find\_dir\_containing

```
find_dir_containing(root: Path, relative: str) -> Path
```

Find the directory under `root` that contains `relative` (archives extract nested).

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `root` | `Path` | The extraction root to search. | *required* |
| `relative` | `str` | A file name expected inside the wanted directory. | *required* |

Returns:

| Type | Description |
| --- | --- |
| `Path` | The parent directory of the first match. |

Raises:

| Type | Description |
| --- | --- |
| `FileNotFoundError` | If nothing matches. |

### progress\_sink

```
progress_sink(
    callback: ProgressCallback | None,
) -> Iterator[None]
```

Install `callback` as the download-progress sink for the duration of the block.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `callback` | `ProgressCallback | None` | The sink to receive :class:`DownloadProgress` events, or `None` to report nothing. | *required* |

Yields:

| Type | Description |
| --- | --- |
| `None` | Nothing. The sink is active within the `with` block and restored on exit. |