# LUPMIS2 raster ETL

One-off batch jobs that prepare raster data for LUPMIS2 and publish it to the
LUSPA MinIO object store, from where the PWA streams it as a Cloud-Optimized
GeoTIFF (COG).

This mirrors the existing OpenTopography contour workflow: a **preparation
step run occasionally**, not a live connection from the app.

```
DE Africa (Sandbox)  →  write_cog()  →  MinIO raster-objects  →  LUPMIS2 map
```

## Target infrastructure

| | |
|---|---|
| S3 API endpoint | `https://minioapi.lupmis4luspa.org` (path-style) |
| Bucket | `raster-objects` (anonymous read enabled) |
| Console | `https://minio.lupmis4luspa.org` — admin UI only, **not** the API |

Verified working for COG streaming: anonymous unsigned `GET` returns 200,
range requests return `206 Partial Content`, and CORS allows the PWA origin
with `Range` on preflight and `Content-Range` exposed.

## Credentials

Supplied via environment variables — **never** hardcode them:

```bash
export MINIO_KEY=...
export MINIO_SECRET=...
```

The PHP integration under `FromKwesi/minio-uploads` embeds its keys in source,
and an earlier key pair was committed to Gitea. Keep these out of git.

## `deafrica_dem_to_minio.py`

Exports a Digital Earth Africa DEM for an area of interest and uploads it.

Run it in the **DE Africa Sandbox** — the Open Data Cube is configured there.
Locally, DE Africa's docs note that `dc.load` / `load_ard` need additional
configuration (a database), so the Sandbox is the lower-maintenance choice.

```bash
# 1. confirm which DEM products exist before the first real run
python deafrica_dem_to_minio.py --list-products

# 2. export + upload a pilot district
python deafrica_dem_to_minio.py --district koforidua

# preferred for production: clip to the authoritative boundary
python deafrica_dem_to_minio.py --geojson koforidua.geojson --name koforidua

# inspect the output without uploading
python deafrica_dem_to_minio.py --district tamale --dry-run
```

The script writes `<name>_dem.tif`, uploads it to `dem/<name>_dem.tif`, then
verifies the public URL really is anonymously readable and range-capable. It
also emits `<name>_dem.manifest.json` recording the URL, product, bbox, CRS and
resolution.

### Loading the result in LUPMIS2

**Add External Layer → COG →** paste the object URL, e.g.

```
https://minioapi.lupmis4luspa.org/raster-objects/dem/koforidua_dem.tif
```

### Notes and caveats

- **Preset district bounding boxes are approximate.** Use `--geojson` with the
  real district boundary from PostGIS for anything beyond a first test.
- **Output CRS defaults to EPSG:3857** so the browser does no reprojection.
  For Stage 3 hydrology and slope, re-export in a metric CRS (UTM 30N =
  `EPSG:32630` west, 31N = `EPSG:32631` east) — Web Mercator distorts distance
  with latitude and will bias slope and flow-accumulation results.
- **Verify the product name** with `--list-products` first. The default is
  `dem_srtm`; DE Africa also publishes derivatives (`dem_srtm_deriv`), which is
  the source of the slope WMS layer LUPMIS2 already consumes.