# Timenet connectors.download

## timenet\_connectors.download

Async, scheme-dispatching artifact downloads for connectors.

The high-level entry points are :func:`download_files` (a list of :class:`Artifact`, any mix of `s3://`
and `http(s)://`) and :func:`ensure_archive` (download a zip and extract it once). Downloads report
progress through the ambient :func:`progress_sink`. The per-transport backends live in :mod:`.http` and
:mod:`.s3`.

### DownloadCallback `module-attribute`

```
DownloadCallback = Callable[[DownloadProgress], None]
```

A callback that receives download progress events.

### Artifact `dataclass`

A file to download, with its URL, destination, and optional request settings and SHA-256 hash.

The headers and cookies apply only to HTTP downloads.
The SHA-256 hash applies to both HTTP and S3 downloads.

#### cookies `class-attribute` `instance-attribute`

```
cookies: Mapping[str, str] | None = None
```

#### dest `instance-attribute`

```
dest: Path
```

#### headers `class-attribute` `instance-attribute`

```
headers: Mapping[str, str] | None = None
```

#### sha256 `class-attribute` `instance-attribute`

```
sha256: str | None = None
```

#### url `instance-attribute`

```
url: str
```

### DownloadProgress `dataclass`

Progress for one download: bytes transferred so far and the total, if known.

#### downloaded `instance-attribute`

```
downloaded: int
```

#### total `instance-attribute`

```
total: int | None
```

#### url `instance-attribute`

```
url: str
```

### download\_files `async`

```
download_files(
    artifacts: Iterable[Artifact],
    *,
    headers: Mapping[str, str] | None = None,
    cookies: Mapping[str, str] | None = None,
    max_concurrency: int = _DEFAULT_MAX_CONCURRENCY,
    skip_existing: bool = True,
) -> list[Path]
```

Download files from S3 and HTTP URLs.

S3 downloads block the event loop and run one at a time, while HTTP downloads run concurrently.
The function checks all URL schemes before starting any download.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `artifacts` | `Iterable[Artifact]` | The files to download, with their URLs and destination paths. | *required* |
| `headers` | `Mapping[str, str] | None` | Default HTTP headers that each artifact's headers override. | `None` |
| `cookies` | `Mapping[str, str] | None` | Default HTTP cookies that each artifact's cookies override. | `None` |
| `max_concurrency` | `int` | Maximum number of concurrent HTTP downloads. | `_DEFAULT_MAX_CONCURRENCY` |
| `skip_existing` | `bool` | Skip existing files if they match `sha256` or have no expected hash. | `True` |

Returns:

| Type | Description |
| --- | --- |
| `list[Path]` | The destination paths, in input order. |

Raises:

| Type | Description |
| --- | --- |
| `TimeFValidationError` | If a URL uses a scheme other than `s3://`, `http://`, or `https://`. |

### ensure\_archive `async`

```
ensure_archive(
    url: str,
    target: str | Path,
    *,
    filename: str | None = None,
    headers: Mapping[str, str] | None = None,
    cookies: Mapping[str, str] | None = None,
    sha256: str | None = None,
) -> Path
```

Download a zip archive and extract it into `target`.

A marker file lets later calls reuse an extraction with the same URL, filename, and SHA-256 hash.
The function deletes the archive after extraction succeeds and keeps it if extraction fails.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `url` | `str` | The archive URL, `s3://` or `http(s)://`. | *required* |
| `target` | `str | Path` | The directory for the archive and its extracted contents. | *required* |
| `filename` | `str | None` | An optional archive filename to use instead of the name derived from the URL. | `None` |
| `headers` | `Mapping[str, str] | None` | Request headers for HTTP downloads only. | `None` |
| `cookies` | `Mapping[str, str] | None` | Request cookies for HTTP downloads only. | `None` |
| `sha256` | `str | None` | An optional SHA-256 hash in hexadecimal that the archive must match. | `None` |

Returns:

| Type | Description |
| --- | --- |
| `Path` | The extraction directory, `target`. |

### find\_dir\_containing

```
find_dir_containing(root: Path, relative: str) -> Path
```

Find the directory under `root` that contains `relative` (archives extract nested).

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `root` | `Path` | The extraction root to search. | *required* |
| `relative` | `str` | A file name expected inside the wanted directory. | *required* |

Returns:

| Type | Description |
| --- | --- |
| `Path` | The parent directory of the first match. |

Raises:

| Type | Description |
| --- | --- |
| `FileNotFoundError` | If nothing matches. |

### progress\_sink

```
progress_sink(
    callback: DownloadCallback | None,
) -> Generator[None]
```

Install `callback` as the download-progress sink for the duration of the block.

Parameters:

| Name | Type | Description | Default |
| --- | --- | --- | --- |
| `callback` | `DownloadCallback | None` | The sink to receive :class:`DownloadProgress` events, or `None` to report nothing. | *required* |

Yields:

| Type | Description |
| --- | --- |
| `Generator[None]` | Nothing. The sink is active within the `with` block and restored on exit. |