ekke ef12e4477b Offline tile cache, polygon Divide, topographic layer integrations
Major feature batch covering drawing-tool improvements, layer additions,
and offline-first capabilities. Largest changes in MapView.js (+1700),
main.js (+1500), public/sw.js (+367), and new modules under src/.

Drawing & editing toolkit
  * Polygon Divide tool — sub-button under Split, divides a polygon into
    N equal-area pieces via binary search; user picks the cutting edge
  * UPN pick phase after Split and Divide — non-picked pieces have their
    identifier fields cleared automatically
  * Improved Merge algorithm — vertex-to-edge proximity (5 m tol.) with
    hybrid lockstep extension; bold A/B labels on selected polygons
  * Persistent vertex highlights — all vertices of the selected polygon
    rendered as dots while edit mode is on, without subclassing ol-ext
  * Toast notifications for merge/split/divide outcomes
  * Shapefile import — addGeoJSONLayer now includes an image style so
    Point features render (previously invisible)

Background & overlay layers
  * DEAfrica Coastlines v0.4 (WMS) in Biophysical Environment
  * DEAfrica Slope (SRTM 30m, style_slope) — semi-transparent background
  * Contours hillshade — get_contours_hillshade.php → local SQLite cache
  * OSM_roads — get_osm_roads.php → local SQLite cache, casing-stroke
    style (black 3.5 px outer, #F0F1F0 1.5 px inner)
  * External Source dialog — green + button in LayerSwitcher lets users
    add WMS / WFS / XYZ layers at runtime
  * Generic addWMSLayer / addXYZLayer with style, opacity, zIndex,
    legendUrl, onlineOnly options
  * TileWMS replaces ImageWMS (fixes 'Width exceeds 512' WMS errors)
  * Legend panel — bottom-right, auto-shown for visible layers that
    register a legendUrl
  * Default base map setting in Settings, persisted in localStorage;
    setBaseMap() on MapView

Offline tile cache (Phase 1 + 2)
  * Service worker: per-host tile caches (osm / topo / satellite /
    carto-light / carto-dark), counter-based eviction to prevent
    iOS Safari memory-pressure reloads, GET_TILE_STATS /
    CLEAR_TILE_CACHES message API
  * pwa.js helpers: getActiveServiceWorker, onServiceWorkerControllerChange,
    getTileCacheStats, clearTileCaches, getStorageEstimate
  * Settings: Offline Map Tiles card with per-provider stats + clear
  * Phase 2 download dialog: form to pick base map, area (current view /
    district / Ghana), zoom range; live tile-count + size estimate;
    progress bar with cancel; OfflineTileDownloader class with
    concurrency + throttling

Local database management
  * osm_roads table + saveOSMRoads / getLocalOSMRoads helpers
  * CACHED_LAYER_TABLES allow-list with clearTable / clearAllCachedLayers
  * Local Database Tables card: per-row Clear button (cached layers
    only) + 'Refresh cached layers' header button with reload prompt

Build & infrastructure
  * Shpjs lazy-loaded via dynamic import (saves ~140 kB from initial JS)
  * chunkSizeWarningLimit raised to 900 kB (openlayers + sqlite3.wasm
    can't be split further)
  * Toast notification module (src/toast.js)
  * Units module (src/units.js) for metric / imperial conversions
  * PDF export module (src/pdf-export.js)

Documentation & SQL
  * Topographic_Background_Layers_for_LUPMIS2.docx — research report
  * OpenTopography_Workflow.svg/.png — ETL pipeline diagram
  * LUPMIS2_Development_Status_Report.docx — April update section
  * sql/create_landuse_parcels.sql — PostgreSQL schema for the LUSPA
    land-use parcel specification (Feb 2026, revised), with PostGIS
    geometry column and standard indices

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 10:55:30 +02:00

537 lines
17 KiB
JavaScript
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

/**
* Service Worker
*
* Handles caching of:
* - App shell (HTML, CSS, JS)
* - Map tiles (passive runtime caching, per-host buckets)
* - API responses (network-first)
*
* Note: Database operations are handled by the SharedWorker (shared-db-worker.js),
* NOT by this service worker. They serve different purposes:
* - Service Worker: Caching, offline asset serving, push notifications
* - SharedWorker: Shared database connection across tabs
*/
// v3: lower per-cache limits (5000 → 1500) and counter-based eviction to
// prevent Safari memory-pressure reloads.
// v4: raise OSM and Topographic limits to 8000 to support active offline
// downloads (Phase 2). Other providers stay at 1500.
const CACHE_VERSION = 'v4';
const SHELL_CACHE = `shell-${CACHE_VERSION}`;
const MODULES_CACHE = `modules-${CACHE_VERSION}`;
const API_CACHE = `api-${CACHE_VERSION}`;
// ----------------------------------------------------------------------------
// Tile caches — one per provider so users can clear them independently.
// Limits are per-cache (not global). 5 000 tiles ≈ ~150 MB at ~30 KB/tile,
// which covers a Ghana district at zoom 1015 (typical field-work range).
// ----------------------------------------------------------------------------
const TILES_OSM = `tiles-osm-${CACHE_VERSION}`;
const TILES_TOPO = `tiles-topo-${CACHE_VERSION}`;
const TILES_SATELLITE = `tiles-satellite-${CACHE_VERSION}`;
const TILES_CARTO_LIGHT = `tiles-carto-light-${CACHE_VERSION}`;
const TILES_CARTO_DARK = `tiles-carto-dark-${CACHE_VERSION}`;
// Per-provider tile limits.
// • OSM and Topographic are the providers offered for active offline
// download (Phase 2 dialog), so they get a higher cap (~240 MB each at
// ~30 KB/tile) — enough for a typical Ghana district at zoom 1015.
// • The other providers serve passive caching only (whatever the user has
// already viewed), so 1 500 tiles ≈ 45 MB is plenty.
//
// Total max ≈ 5 × ~150 MB = ~750 MB on disk in the worst case, but only the
// two downloadable buckets are likely to fill. Eviction sweeps run every 100
// inserts (see EVICTION_CHECK_INTERVAL) so memory pressure stays bounded.
const TILE_LIMITS = {
[TILES_OSM]: 8000,
[TILES_TOPO]: 8000,
[TILES_SATELLITE]: 1500,
[TILES_CARTO_LIGHT]: 1500,
[TILES_CARTO_DARK]: 1500,
};
// Per-cache running insert counter, in memory. Avoids calling cache.keys()
// (which materialises every Request object in the cache) on every put — that
// was the cause of the Safari "reloaded due to memory pressure" failures.
//
// We only run a real eviction sweep every EVICTION_CHECK_INTERVAL inserts.
const _tileInsertCounters = new Map(); // cacheName → number of inserts since last eviction
const EVICTION_CHECK_INTERVAL = 100;
// Friendly name shown in the UI (matches Settings card labels)
const TILE_CACHE_LABELS = {
[TILES_OSM]: 'OpenStreetMap',
[TILES_TOPO]: 'Topographic',
[TILES_SATELLITE]: 'Satellite',
[TILES_CARTO_LIGHT]: 'Carto Light',
[TILES_CARTO_DARK]: 'Carto Dark',
};
const ALL_TILE_CACHES = Object.keys(TILE_LIMITS);
// Approximate average tile size — used for storage estimation.
// Real measurements: PNG tiles range 580 KB; 30 KB is a good middle ground.
const AVG_TILE_BYTES = 30 * 1024;
// ----------------------------------------------------------------------------
// App shell assets — precached on install.
// ----------------------------------------------------------------------------
const SHELL_ASSETS = [
'/',
'/index.html',
'/offline.html',
'/manifest.json'
];
// ============================================================================
// INSTALL EVENT
// ============================================================================
self.addEventListener('install', (event) => {
console.log('[SW] Installing...');
event.waitUntil(
caches.open(SHELL_CACHE)
.then((cache) => {
console.log('[SW] Precaching app shell');
return cache.addAll(SHELL_ASSETS);
})
.then(() => self.skipWaiting())
);
});
// ============================================================================
// ACTIVATE EVENT
// ============================================================================
self.addEventListener('activate', (event) => {
console.log('[SW] Activating...');
event.waitUntil(
caches.keys()
.then((cacheNames) => {
// Build the set of caches that should remain
const keep = new Set([SHELL_CACHE, MODULES_CACHE, API_CACHE, ...ALL_TILE_CACHES]);
return Promise.all(
cacheNames
// Delete anything that:
// • belongs to one of our managed cache prefixes (shell-, tiles-, modules-, api-)
// • but is NOT in the current keep set
// This includes the legacy "tiles-v1" single bucket.
.filter((name) => {
const isOurs =
name.startsWith('shell-') ||
name.startsWith('tiles-') ||
name.startsWith('modules-') ||
name.startsWith('api-');
return isOurs && !keep.has(name);
})
.map((name) => {
console.log('[SW] Deleting old cache:', name);
return caches.delete(name);
})
);
})
.then(() => self.clients.claim())
);
});
// ============================================================================
// FETCH EVENT
// ============================================================================
self.addEventListener('fetch', (event) => {
const request = event.request;
const url = new URL(request.url);
// Only handle GET requests
if (request.method !== 'GET') return;
// Skip chrome-extension and other non-http(s) requests
if (!url.protocol.startsWith('http')) return;
// Skip worker files and Vite dev-server node_modules requests —
// intercepting these breaks module workers (e.g. SQLocal/SQLite).
if (url.pathname.includes('node_modules') ||
url.search.includes('worker_file') ||
request.destination === 'worker') return;
// ----- TILE REQUESTS — passive cache-then-network (per-host bucket) -----
const tileCache = getTileCacheName(url);
if (tileCache) {
event.respondWith(tileCacheThenNetwork(request, tileCache));
return;
}
// ----- OTHER ROUTES (unchanged) -----
if (isApiRequest(url)) {
event.respondWith(networkFirst(request, API_CACHE));
} else if (isModuleAsset(url)) {
event.respondWith(staleWhileRevalidate(request, MODULES_CACHE));
} else if (isAppAsset(url)) {
event.respondWith(cacheFirst(request, SHELL_CACHE));
}
// Let other requests pass through to network
});
// ============================================================================
// URL CLASSIFICATION
// ============================================================================
/**
* Classify a URL into the appropriate tile cache.
* Returns `null` for non-tile requests, or for tile providers we deliberately
* do NOT cache (e.g. Google — caching is forbidden by their ToS).
*/
function getTileCacheName(url) {
const host = url.hostname;
// OpenStreetMap — tile.openstreetmap.org and a/b/c subdomains
if (host.endsWith('tile.openstreetmap.org')) return TILES_OSM;
// OpenTopoMap — a/b/c.tile.opentopomap.org
if (host.endsWith('tile.opentopomap.org') || host.endsWith('opentopomap.org')) return TILES_TOPO;
// Carto Basemaps — light_all / dark_all distinguished by path
if (host.endsWith('basemaps.cartocdn.com')) {
if (url.pathname.includes('/light_all/')) return TILES_CARTO_LIGHT;
if (url.pathname.includes('/dark_all/')) return TILES_CARTO_DARK;
return null; // unknown Carto style
}
// Esri — server.arcgisonline.com
if (host.endsWith('arcgisonline.com')) return TILES_SATELLITE;
// Google — caching forbidden by ToS, do not store
if (host.endsWith('google.com') || host.endsWith('googleapis.com')) return null;
// Other tile providers (WMS endpoints, OWS, custom) — not cached at this layer
// (the user's "online only" toast handles those).
return null;
}
function isApiRequest(url) {
return url.pathname.startsWith('/api/') ||
url.pathname.endsWith('.php');
}
function isModuleAsset(url) {
return url.pathname.startsWith('/modules/');
}
function isAppAsset(url) {
return url.origin === self.location.origin &&
(url.pathname.endsWith('.html') ||
url.pathname.endsWith('.css') ||
url.pathname.endsWith('.js') ||
url.pathname.endsWith('.wasm') ||
url.pathname.endsWith('.json') ||
url.pathname.match(/\.(png|jpg|jpeg|gif|svg|ico|webp)$/));
}
// ============================================================================
// CACHING STRATEGIES
// ============================================================================
/**
* Cache First — Use cache, fallback to network.
* Best for: App shell, static assets.
*/
async function cacheFirst(request, cacheName) {
const cached = await caches.match(request);
if (cached) return cached;
try {
const response = await fetch(request);
if (response.ok) {
const cache = await caches.open(cacheName);
cache.put(request, response.clone());
}
return response;
} catch (error) {
if (request.mode === 'navigate') {
return caches.match('/offline.html');
}
throw error;
}
}
/**
* Network First — Try network, fallback to cache.
* Best for: API requests, dynamic content.
*/
async function networkFirst(request, cacheName) {
try {
const response = await fetch(request);
if (response.ok) {
const cache = await caches.open(cacheName);
cache.put(request, response.clone());
}
return response;
} catch (error) {
const cached = await caches.match(request);
if (cached) return cached;
throw error;
}
}
/**
* Stale While Revalidate — Return cache immediately, update in background.
* Best for: Module assets, frequently updated content.
*/
async function staleWhileRevalidate(request, cacheName) {
const cache = await caches.open(cacheName);
const cached = await cache.match(request);
const fetchPromise = fetch(request).then((response) => {
if (response.ok) {
cache.put(request, response.clone());
}
return response;
}).catch(() => cached);
return cached || fetchPromise;
}
/**
* Tile Cache then Network — Per-host bucket with size limit.
* Cache first; on miss, fetch from network and store.
*
* Memory-conservative eviction:
* • Increments an in-memory counter on every successful insert
* • Only calls cache.keys() (which materialises all Request objects) every
* EVICTION_CHECK_INTERVAL inserts — so the cost is amortised
* • Eviction drops the oldest 10 % when over the per-host limit
*
* On network failure (offline), serves a 408 so the map renders a blank tile
* rather than throwing.
*/
async function tileCacheThenNetwork(request, cacheName) {
const cache = await caches.open(cacheName);
const cached = await cache.match(request);
if (cached) return cached;
try {
const response = await fetch(request);
if (response.ok) {
// Bump the counter; periodically run a real eviction sweep
const count = (_tileInsertCounters.get(cacheName) || 0) + 1;
_tileInsertCounters.set(cacheName, count);
if (count % EVICTION_CHECK_INTERVAL === 0) {
// Reset the counter — next sweep is another EVICTION_CHECK_INTERVAL away
_tileInsertCounters.set(cacheName, 0);
await maybeEvict(cache, cacheName);
}
// Don't await put() — it can run after we return the response, keeping
// the fetch hot path lightweight.
cache.put(request, response.clone()).catch((err) => {
// QuotaExceededError → run an immediate eviction sweep and retry once
if (err && err.name === 'QuotaExceededError') {
maybeEvict(cache, cacheName, /* force */ true).catch(() => {});
}
});
}
return response;
} catch (error) {
// Offline — let the map renderer show a blank tile
return new Response('', { status: 408, statusText: 'Offline' });
}
}
/**
* Run an eviction sweep on a cache, dropping the oldest 10 % of entries
* when over the per-cache limit. Heavy: only call periodically.
*/
async function maybeEvict(cache, cacheName, force = false) {
try {
const limit = TILE_LIMITS[cacheName] || 1500;
const keys = await cache.keys();
if (force || keys.length >= limit) {
const drop = Math.max(1, Math.ceil(limit * 0.1));
const toDelete = keys.slice(0, drop);
await Promise.all(toDelete.map((k) => cache.delete(k)));
}
} catch (err) {
console.warn('[SW] eviction sweep failed for', cacheName, err);
}
}
// ============================================================================
// MESSAGE HANDLING
// ============================================================================
self.addEventListener('message', (event) => {
const { type, payload } = event.data || {};
switch (type) {
case 'SKIP_WAITING':
self.skipWaiting();
break;
case 'CACHE_MODULES':
cacheModules(payload.modules);
break;
case 'CLEAR_USER_CACHE':
clearUserCaches();
break;
case 'GET_CACHE_STATUS':
getCacheStatus().then((status) => {
event.source.postMessage({ type: 'CACHE_STATUS', status });
});
break;
// ----- Tile-cache management (Phase 1 offline maps) -----
case 'GET_TILE_STATS':
getTileStats().then((stats) => {
event.source.postMessage({ type: 'TILE_STATS', stats });
});
break;
case 'CLEAR_TILE_CACHES':
clearTileCaches().then(() => {
event.source.postMessage({ type: 'TILE_CACHES_CLEARED' });
});
break;
}
});
// ============================================================================
// HELPER FUNCTIONS
// ============================================================================
/**
* Cache specific modules on demand.
*/
async function cacheModules(moduleNames) {
const cache = await caches.open(MODULES_CACHE);
for (const moduleName of moduleNames) {
try {
const moduleAssets = [
`/modules/${moduleName}/index.js`,
`/modules/${moduleName}/index.css`,
`/modules/${moduleName}/index.html`
];
await cache.addAll(moduleAssets.filter(async (url) => {
try {
const response = await fetch(url, { method: 'HEAD' });
return response.ok;
} catch {
return false;
}
}));
console.log('[SW] Cached module:', moduleName);
} catch (error) {
console.warn('[SW] Failed to cache module:', moduleName, error);
}
}
}
/**
* Clear user-specific caches (call on logout).
* Tile caches are NOT cleared here — those belong to the device, not the user.
*/
async function clearUserCaches() {
await caches.delete(API_CACHE);
await caches.delete(MODULES_CACHE);
console.log('[SW] Cleared user caches');
}
/**
* Get summary status of all caches (count of entries in each).
*/
async function getCacheStatus() {
const cacheNames = await caches.keys();
const status = {};
for (const name of cacheNames) {
const cache = await caches.open(name);
const keys = await cache.keys();
status[name] = keys.length;
}
return status;
}
/**
* Get per-provider tile cache statistics.
*
* Returns shape:
* {
* totals: { count, estBytes },
* byProvider: [{ key, label, count, limit, estBytes }, ...]
* }
*
* estBytes is an approximation (count × AVG_TILE_BYTES). For an exact size,
* the caller can use navigator.storage.estimate() on the page side.
*
* Result is cached for STATS_TTL_MS so rapid re-queries (e.g. multiple
* Settings opens) don't re-enumerate every cache.
*/
const STATS_TTL_MS = 10 * 1000;
let _cachedStats = null;
let _cachedStatsAt = 0;
async function getTileStats({ force = false } = {}) {
const now = Date.now();
if (!force && _cachedStats && (now - _cachedStatsAt) < STATS_TTL_MS) {
return _cachedStats;
}
const byProvider = [];
let totalCount = 0;
for (const cacheName of ALL_TILE_CACHES) {
let count = 0;
if (await caches.has(cacheName)) {
const cache = await caches.open(cacheName);
// matchAll returns a smaller payload than keys() on Safari, but neither
// is free. Done at most once per STATS_TTL_MS thanks to the cache above.
const keys = await cache.keys();
count = keys.length;
}
byProvider.push({
key: cacheName,
label: TILE_CACHE_LABELS[cacheName] || cacheName,
count,
limit: TILE_LIMITS[cacheName] || 0,
estBytes: count * AVG_TILE_BYTES,
});
totalCount += count;
}
_cachedStats = {
totals: {
count: totalCount,
estBytes: totalCount * AVG_TILE_BYTES,
},
byProvider,
};
_cachedStatsAt = now;
return _cachedStats;
}
/**
* Delete every tile cache. Frees the device storage used by cached map tiles.
* Does not affect app-shell, modules, or API caches.
*/
async function clearTileCaches() {
const results = await Promise.all(
ALL_TILE_CACHES.map((name) => caches.delete(name))
);
// Reset counters and invalidate stats cache
_tileInsertCounters.clear();
_cachedStats = null;
_cachedStatsAt = 0;
console.log('[SW] Cleared tile caches:', ALL_TILE_CACHES.filter((_, i) => results[i]));
}