ekke 8f48abc858 Docs for the database team: OSM re-import, external layer tables
Two documents, both for work the database team will carry out. No application
code changes.

OSM re-import runbook
---------------------
The roads endpoint returns geom, districtid, osm_id, surface, oneway and name
for district 1 — no `highway`. The road class was discarded at import, which is
what osm2pgsql's default.style does to any tag not on its list, so roads cannot
be styled or filtered by class. A discarded tag cannot be recovered by querying;
the only fix is to import again, and if that is happening anyway it is worth
doing on terms that allow the data to be kept current afterwards.

The runbook covers a flex import with --slim and --extra-attributes, producing
four tables — roads, lines, points, polygons — plus daily replication. Two
decisions are specific to LUPMIS:

  * The import goes into its own `osm` schema. osm2pgsql --create drops and
    recreates every table it owns, and `spatial` holds lu_parcels; the API is
    given views in `spatial` instead, so the existing get_osm_roads.php keeps
    working and simply gains `highway`.
  * districtid is stamped by a spatial join after each import, since osm2pgsql
    has no concept of a district.

Executed against osm2pgsql 2.2.0 and a scratch PostGIS database with a hand-made
extract covering every shape the config handles. All four tables imported
cleanly; updatable and attributes both true; amenities mapped as node, closed
way and multipolygon relation all reached the points table, as did a
healthcare-tagged node with no amenity — three cases the current layout cannot
serve. An --append then applied a diff and left districtid NULL on the changed
row while untouched rows kept theirs, which is the behaviour the incremental
re-stamp in section 5.2 depends on.

Not verified: anything needing the LUPMIS database itself — the current import
state, the district boundary table's real name, row counts. Section 1 is the
set of checks to run first.

External layer tables
---------------------
DDL for storing layers added through the Add External Layer dialog:

  spatial.hlp_layer_types     available layer types (wms, wfs, xyz, cog)
  spatial.es_external_layers  one row per layer a user has added

Layer types live in a table rather than a CHECK constraint so a new kind can be
introduced by inserting a row. The rule that WMS and WFS require a layer name
moved there too as requires_layer_name — left in the schema it would have meant
a migration for every new type anyway. Validity is enforced by a foreign key and
the conditional rule by a trigger, since a CHECK cannot read another table; the
trigger raises check_violation so existing API error handling still applies.

Beyond the dialog's four fields the table carries districtid (NULL = every
district), userid and is_shared, plus the columns needed to represent the
external layers the application already hard-codes — style, opacity, z_index,
attribution, legend_url, online_only. Without those, moving the existing
DEAfrica slope layer into the database would lose information the map relies on;
it is included as a seed row to prove the schema can hold it.

Executed against PostgreSQL 16. Nine cases pass, including: a new type added as
data with no DDL and usable immediately, that type enforcing its own
requires_layer_name, a retired type blocked for new layers while existing ones
stay editable, and deletion of a type still in use being refused.

The application still hard-codes the four types and will until
get_layer_types.php exists, so the dialog and hlp_layer_types must be changed
together until then — noted in the file, as the two fail in opposite directions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-20 10:42:16 +02:00
2026-03-04 12:59:40 +01:00
2026-03-04 12:59:40 +01:00
2026-03-04 12:59:40 +01:00

LUPMIS PWA with Offline SQLite and Maps

A Progressive Web App with:

  • OpenLayers map with ol-ext LayerSwitcher for base map selection
  • SQLocal for SQLite database in the browser (via OPFS)
  • BroadcastChannel for cross-tab synchronization
  • Service Worker for asset caching and offline support
  • Vite for development and building

Features

  • 🗺️ Interactive map with 5 base layers (OSM, Satellite, Topo, Carto Light/Dark)
  • 📍 Click map to set coordinates, markers colored by category
  • 💾 Offline SQLite database (data persists in browser)
  • 🔄 Cross-tab sync via BroadcastChannel
  • 📴 Works offline (cached assets + up to 500 map tiles)
  • 📱 Installable as PWA on mobile and desktop

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                         Browser                                  │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  ┌──────────┐    ┌──────────┐    ┌──────────┐                  │
│  │  Tab 1   │    │  Tab 2   │    │  Tab 3   │                  │
│  │ main.js  │    │ main.js  │    │ main.js  │                  │
│  │ SQLocal  │    │ SQLocal  │    │ SQLocal  │                  │
│  └────┬─────┘    └────┬─────┘    └────┬─────┘                  │
│       │               │               │                         │
│       └───────────────┼───────────────┘                         │
│                       │                                         │
│              BroadcastChannel                                   │
│         (notifies other tabs of changes)                        │
│                       │                                         │
│                       ▼                                         │
│              ┌────────────────┐                                 │
│              │     OPFS       │  ← Single database file         │
│              │ (lupmis.db)    │    shared by all tabs           │
│              └────────────────┘                                 │
│                                                                  │
│  ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│                                                                  │
│              ┌────────────────┐                                 │
│              │ Service Worker │  ← Caches assets for offline    │
│              │    (sw.js)     │                                 │
│              └────────────────┘                                 │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

Why This Architecture?

Initially we considered using a SharedWorker to manage a single database connection. However:

  1. SQLocal already uses its own internal worker - It handles OPFS access internally
  2. OPFS handles file coordination - Multiple SQLocal instances can access the same database file
  3. Simpler is better - BroadcastChannel provides easy cross-tab notification without the complexity of SharedWorker bundling issues in Vite

The result is simpler code that works reliably with Vite's build system.

File Structure

project/
├── index.html              # Entry HTML with map container
├── main.js                 # App entry point
├── vite.config.js          # Vite configuration
├── package.json
│
├── src/
│   ├── components/
│   │   └── MapView.js      # OpenLayers map with ol-ext LayerSwitcher
│   ├── database.js         # SQLocal + BroadcastChannel
│   └── pwa.js              # PWA utilities (install, offline)
│
└── public/
    ├── sw.js               # Service Worker (caching)
    ├── manifest.json       # PWA manifest
    ├── offline.html        # Offline fallback page
    └── icons/              # PWA icons

Setup

# Install dependencies
npm install

# Start development server
npm run dev

# Build for production
npm run build

# Preview production build
npm run preview

Usage

Basic Database Operations

import { sql, dbReady, addLocation, getLocations } from './src/database.js';

// Wait for database to be ready
await dbReady;

// Add a location
await addLocation('Water Point', -1.5234, 7.4567, {
  description: 'Main village well',
  category: 'water'
});

// Get all locations
const locations = await getLocations();

// Direct SQL queries using tagged templates
const results = await sql`SELECT * FROM locations WHERE category = ${'water'}`;

MapView Component

import { MapView } from './src/components/MapView.js';

// Create map centered on Ghana
const map = new MapView('map-container', {
  center: [-1.5, 7.5],    // [longitude, latitude]
  zoom: 7,
  basemap: 'osm'          // 'osm' | 'satellite' | 'topo' | 'carto-light' | 'carto-dark'
});

// Set category-based marker colors
map.setCategoryStyles({
  'water': { color: '#3b82f6' },
  'school': { color: '#f59e0b' },
  'health': { color: '#ef4444' },
});

// Add markers from database
const locations = await getLocations();
map.addMarkers(locations);

// Handle map clicks
map.onClick((lon, lat, feature) => {
  if (feature) {
    // Clicked on existing marker
    console.log('Selected:', feature.get('name'));
  } else {
    // Clicked on empty space - use coordinates
    document.getElementById('longitude').value = lon.toFixed(6);
    document.getElementById('latitude').value = lat.toFixed(6);
  }
});

// Zoom to a location
map.zoomTo(-1.5, 7.5, 14);

// Fit view to show all markers
map.fitToMarkers();

// Select a marker by ID
map.selectMarker(locationId);

Available Base Maps

Name Key Source
OpenStreetMap osm OpenStreetMap
Satellite satellite Esri World Imagery
Topographic topo OpenTopoMap
Carto Light carto-light CARTO
Carto Dark carto-dark CARTO

Cross-Tab Synchronization

import { onDatabaseChange } from './src/database.js';

// Listen for changes from other tabs
onDatabaseChange((change) => {
  console.log('Database changed:', change);
  // { table: 'locations', action: 'INSERT', id: 5, timestamp: 1234567890 }
  
  if (change.table === 'locations') {
    refreshLocationsList();
  }
});

PWA Features

import { initPWA, isOnline, onOfflineChange } from './src/pwa.js';

// Initialize PWA
await initPWA();

// Check online status
if (isOnline()) {
  syncWithServer();
}

// React to offline/online changes
onOfflineChange((offline) => {
  if (offline) {
    showOfflineBanner();
  } else {
    hideOfflineBanner();
    syncWithServer();
  }
});

Deployment

Required Headers

Your web server must send these headers for OPFS to work:

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp

Nginx Configuration

server {
    listen 443 ssl http2;
    server_name your-domain.com;
    
    root /var/www/dist;
    index index.html;
    
    # Required for OPFS/SQLite
    add_header Cross-Origin-Opener-Policy "same-origin" always;
    add_header Cross-Origin-Embedder-Policy "require-corp" always;
    
    # Cache static assets
    location ~* \.(js|css|wasm|png|jpg|ico|svg)$ {
        expires 1y;
        add_header Cache-Control "public, immutable";
        add_header Cross-Origin-Opener-Policy "same-origin" always;
        add_header Cross-Origin-Embedder-Policy "require-corp" always;
    }
    
    # SPA fallback
    location / {
        try_files $uri $uri/ /index.html;
    }
}

Apache (.htaccess)

<IfModule mod_headers.c>
    Header always set Cross-Origin-Opener-Policy "same-origin"
    Header always set Cross-Origin-Embedder-Policy "require-corp"
</IfModule>

OpenResty (Docker)

add_header Cross-Origin-Opener-Policy "same-origin" always;
add_header Cross-Origin-Embedder-Policy "require-corp" always;

Browser Support

  • Chrome/Edge 89+ - Full support
  • Firefox 111+ - Full support
  • Safari 15.2+ - OPFS supported
  • Mobile - Chrome Android, Safari iOS 15.2+

Troubleshooting

"SecurityError" or "NotAllowedError"

The COOP/COEP headers are missing. Check your server configuration.

Database not persisting

  1. Check that you're using HTTPS (or localhost)
  2. Verify COOP/COEP headers are present (DevTools → Network → check response headers)
  3. Check browser DevTools → Application → Storage → OPFS

Changes not syncing between tabs

The BroadcastChannel should handle this automatically. Check the browser console for any errors.

Vite HMR WebSocket errors

The cross-origin isolation can break Vite's hot reload. Options:

  1. Use vite-plugin-cross-origin-isolation (add to vite.config.js)
  2. Or manually refresh the browser after changes

License

MIT

Description
progressive web app with sqlocal as a mobile app
Readme 49 MiB
Languages
JavaScript 75.4%
HTML 11%
CSS 9.1%
PHP 2.3%
Python 1.2%
Other 1%