FIMViz.js

Package roadmap — proposed additions

Forward-looking design for engine capabilities we want. Records the shape, the seam each hangs off, and the high-value first slice, so work can start without re-deriving the design. Builds on the Dataset/Layer ADT (DECISIONS_TRADEOFFS_INCOMPLETE_ITEMS.md §1.1) and the provider seam (mapProvider.js).

1. Cross-provider event-filtering & interaction policy

Problem. Mouse interaction was ad hoc — hover/draw attached google.maps.event listeners directly to the map, and a raster overlay swallowing events was fixed with a blunt clickable:false. No provider-neutral, layer-aware model for who receives a click/drag/hover.

Design. A normalized, precedence-based event dispatcher on the map instance, provider-neutral:

✅ Landed: onMapEvent (Google+Leaflet) for click/hover, FimMap z-order dispatch with absorption, simultaneous mode, modal capture / region-draw, pixel-level hit-test (RasterLayer.hitTest = a real non-noData pixel under the point, so clicks fall through transparent areas), and the interactive opt-in (raster overlays default non-interactive via addRasterImage(..., { interactive }) — the hardcoded clickable:false retired). Still open: the fuller vocabulary (drag/dblclick/contextmenu end-to-end).

2. Dataset operations (clip / mask / transform / 3-D aggregate)

Problem. We already bundle GDAL — high-value spatial ops turn Dataset from "a thing you render" into "a thing you analyze."

Design. Each is a lazy op returning a new Dataset (immutable, memoized), forced through a registered seam (keeping GDAL out of Dataset's import graph). N-ary ops use the #inputs[] node.

✅ Landed as pure-JS lazy ops on the decoded grid (package/rasterOps.js; need no GDAL, node-tested; masked/unmatched pixels → NaN, which colorize/Stats treat as transparent): ds.clip(bbox), ds.mask(polygon,{invert}), ds.reclassify(rules,{unmatched}) build lazy nodes forced at terminal and round-trip through toRecord/fromRecord. A UI — ui/operationsPanel.js createOperationsPanel, styled like the tools panel — drives them on a live layer via deriveSources. Also band math — ds.combine([b,…], { op })/ds.difference(b) (difference/ratio binary, sum/mean/min/max N-ary), the first real N-ary op: LHS-conform (others resampled onto THIS grid via geo/resample), forcing the multi-input #inputs node, round-tripping through records — and ds.zonalStats(zones) (a terminal returning per-zone min/max/mean/sum/count/area). Both pure-JS + node-tested; the pure grid forms (combineGrids/zonalStats) are on the barrel.

✅ The panel now covers the whole set, driven by an op TABLE rather than a form per op: clip · mask · reclassify · slope · aspect · hillshade · resample · reproject · combine · reduce · zonal stats · group by, plus rasterize on the vector side, grouped and filtered by the layer's kind. Multi-input ops take an operand pool (layers); the two table-returning terminals report through onResult instead of onApply, so a read is never mistaken for a change to the layer. See DECISIONS §1.1.

✅ Landed: terrain, as pure JS (slopeGrid/aspectGrid/hillshadeGrid in rasterOps.js, node-tested) — Horn's 1981 3×3-window gradient, the same algorithm gdaldem slope/aspect/hillshade use, run directly on the decoded grid instead of shelling out to GDAL (the same "pure-JS substitute" choice already made for clip/mask/reclassify vs. gdalwarp). ds.slope({zFactor,cellsizeX,cellsizeY,unit}), ds.aspect(), ds.hillshade({altitude,azimuth,zFactor,cellsizeX,cellsizeY}) are lazy unary ops, round-tripping through records like the others. cellsizeX/Y default to the grid's own pixel size in the bounds' units (degrees for WGS84 — pass metres for a true-scale result, same unit contract as zonalStats' area); aspect has no cellsize param, matching gdaldem's own square-pixel assumption. Not GDAL-backed, so no wasm/worker cost.

✅ Landed: rasterize (rasterizeFeatures in rasterOps.js + ds.rasterize({width,height,bounds,field, burnValue})) — the kind-changing vector→raster half of "vectorize/rasterize". Point-tests each pixel centre against every feature's polygon (SpatialFilter, reused rather than re-implemented), burning a property value or a constant; later features win on overlap. Polygon/MultiPolygon only (no point/line); holes are ignored (rings union, the same simplification SpatialFilter already makes elsewhere). Node- tested, including the toRecord/fromRecord round-trip and the vector-only-op guard.

✅ Landed: axis reduce (ds.reduce(op, {axis,method,variant}), op ∈ sum|mean|min|max) — collapses a Dataset's selection axis (e.g. a stage/time series) to one grid. Implemented as sugar over select()+combine(): resolves every axis entry to a child Dataset, then LHS-conforms/reduces them exactly like combine() — no new grid math, reusing the already-tested N-ary reducer. Node-tested with a 2-entry axis stub series (mean/sum/min/max, the lazy-not-forced guarantee, the empty-axis and unsupported-op error paths).

✅ Landed: the ops reach a Layer directly — layer.clip(bbox).mask(poly).reclassify(rules) applies each op across every source immediately (synchronously, via the same deriveSources path), returns the layer so they chain, and draws once at the next render(); layer.dirty reports an applied-but-undrawn op and layer.reset() restores the pre-op sources. Dataset.fromGrid(grid) closes the loop the other way, wrapping an already-decoded grid as a pre-materialized Dataset so the chain is reachable from a bare grid. N-ary ops (combine/difference) are deliberately absent from the layer — "which source is the left operand" has no answer there — and layer.rasterize() throws, since a kind change cannot return the same layer.

✅ Landed: groupBy — reduction grouped by another raster's values (groupByGrid in rasterOps.js + ds.groupBy(by, opts), node-tested, barrel-exported). The third kind of reduction: reduce() collapses a selection axis, zonalStats() collapses space by geometry, and this collapses space by value — "mean depth per land-use class", "rainfall binned by elevation", a rating curve. That is what "one variable as a series against another" reduces to in practice, and it is a distinct verb rather than an overload because the grouping key comes from data rather than from the axis model or from geometry (the same "one word must not mean both" rule that keeps reclassify apart from ColorScale). Two modes: discrete (each distinct value of by is a class — classification rasters) and binned (bins: 5 cuts by's own range into equal-width bands, bins: [edges] uses those, last bin closed so the maximum lands somewhere). by is conformed onto the left grid by the same LHS rule and the same resampler combine uses, so the two agree on what "aligned" means; a pixel counts only where both rasters have a value, honouring NaN and each side's noData. Note the division of labour it completes: producing a new grid from two variables is combine/difference; groupBy is the table-producing half of the same question.

Still open: polygonize (raster→vector — the other half of "vectorize/rasterize"; unlike rasterize, this needs contour tracing (marching squares / connected-component boundary tracing), a materially different and larger algorithm than the point-in-polygon tests the rest of §2 reuses — deliberately not folded in with the above), and the gdalwarp -cutline/gdal_calc fast paths for large rasters (a GDAL-backed performance tier over the pure-JS path above, for rasters too large to hold as decoded grids).

3. Proprietary & additional file formats

Problem. parseSource handles geotiff/geojson/kml/kmz/shp; real hydrology/hydraulics work lives in domain and geospatial formats we can't yet ingest.

Design. The materializer/parser registry already generalizes this: Dataset.registerMaterializer(format, fn) (generic geo formats, engine) + domain adapters (parseFimScenario-style, app-tier, per lab schema). Format detection extends detectFormat; browser parsing uses WASM (GDAL covers many) or a JS parser.

Candidates (roughly by value): NetCDF (.nc, multi-dim stacks → maps onto Dataset.axes; netcdfjs), HEC-RAS (.hdf/.g0x/…, the upstream model FIM derives from; HDF5 via h5wasm), SWMM (.inp/.out/.rpt, a domain adapter), GRIB/GRIB2 (weather/precip), GeoPackage (.gpkg, GDAL/ sql.js), COG (.tif, range-read large rasters; geotiff.js supports it), FlatGeobuf (.fgb), LAS/ LAZ (LiDAR point clouds), Esri FileGDB (.gdb, GDAL), ASCII Grid/DEM (.asc).

First slice. NetCDF (exercises the axes model end-to-end, unlocks temporal/ensemble) + COG range reads (§4). Lab-specific schemas (HEC-RAS/SWMM quirks) stay app-tier adapters; generic containers (GeoPackage/FlatGeobuf) in the engine. Research needed: browser viability + size of each WASM/JS parser; which belong in the engine vs. an optional plugin so a google-only consumer doesn't download HDF5/GDAL-full.

✅ Landed: CSV and XYZ (io/parse.js, pure JS, no new dependency; node-tested) — the two formats that had no open design question, so they went first. Both build vector Datasets (format: "csv"/"xyz") and share the existing generic vectorMaterializer (registered in io/materializers.js) since they're already GeoJSON by the time a Dataset exists — no new materializer needed.

✅ Landed: WaterML/NWIS (an app-tier domain adapter, node-tested) — USGS's Instantaneous/Daily Values JSON response (?format=json, "WaterML JSON"). Mirrors parseFimScenario.js's shape: a pure transform (parsed JSON in, generic engine Datasets out), so it never touches io/parse.js. value.timeSeries[] (one entry per site×variable) groups into one Gauge per site — { siteCode, siteName, location, series } — each series entry a Dataset with a time axis, one axis entry per reading. Differs from FIM Scenario's axis shape in one deliberate way: an NWIS reading has no separately-fetchable file (the whole series arrives in the one response), so every entry's ref is null and the reading itself ({ value, qualifiers }) lives directly in meta — select() would be the wrong tool here, so Gauge adds its own latest()/at()/series_() accessors instead of routing through Dataset.select. coord is the ISO timestamp string (not a numeric epoch), so — matching parseFimScenario's time axis — lookups are exact-match/by-index only, not nearest; latest() covers the common case directly. Malformed per-site/per-reading rows are skipped, not fatal, same policy as parseFimScenario. Not yet done: fetching a live NWIS URL and rendering a Gauge (marker + sparkline, or feeding a stage into the flood-extent slider) — that consumer wiring is open UI/app work, deliberately out of scope for the adapter itself.

4. Data import / export mechanisms

Problem. Ingest is file upload + URL fetch (proxiedUrl); export is Dataset.download. Real deployments need more transports and container handling.

Design. A source registry keyed by scheme, mirroring the provider/materializer seams: registerSource(scheme, fn) where a URL's scheme (http/https/ftp/ws/s3) dispatches to a handler yielding bytes for parseSource. Compression is a parallel registerDecompressor(kind, fn) applied before parse.

First slice. Native gzip/tar decompression (zero-dep, immediate value) + the registerSource seam with a WebSocket streaming source (live data is the differentiated capability). FTP/cloud are proxy/ credential-bound — spec the proxy contract first.

5. Headless UI module (ui/) — user-mounted, engine never calls it

Problem. The library is headless, but a package user not rebuilding FIMViz still wants to stand up an interface quickly (a hover value, a marker info window, a toast, a layer palette/opacity panel) without hand-rolling it or lifting app-tier code welded to widget.html.

Design. A separate module inside the package (src/ui/) the host may use, preserving headlessness by two rules: (1) no core module (package//io//geo//layers/) imports ui/; (2) ui/ touches document/window only inside functions, never at import time (same discipline as Dataset.download()) — so importing the barrel still resolves under Node. The engine's only outbound channels stay the host bus (notify/error/emitHost) + console; the engine never auto-wires or calls UI — a ui/ helper may take the bus (connectToast(fim)), but the engine never reaches for it. The hostEvents.test.mjs guard is rescoped from "no engine module touches window" to "no core module does; ui/ may (lazily)."

Contents by coupling:

Explicitly out of the package (decided): the unified Layer Panel (too app-opinionated — the host composes its own from layer.settings + read-models); the bind*Tools event-inversion binders (an artifact of a fixed widget.html, obsolete once settings + effect events exist); forced layer exclusivity — velocity/ensemble mutual-exclusion is host policy, so Layer.exclusive is opt-in, default false (mechanism stays, imposition goes).

First slice — the one-shot build (ordered by testability)

  1. LayerSettings change-model (DATASET_LAYER_ADT §6d) — engine, node-testable: base + RasterSettings
    • VectorSettings, effect-class → event mapping, layer.settings, the new recomputed event.
  2. Event-dispatch first slice (§1) — onMapEvent on google+leaflet for click/hover, layer.hitTest, FimMap z-order dispatch with absorption (geometry + z-order pure → node-tested; provider wiring browser).
  3. UI module — createToast, createTooltip, marker info window, createToolsPanel + rasterControls/vectorControls, renderLegend/renderStats.
  4. Exclusivity → opt-in — Layer.exclusive defaults false.
  5. Example — examples/05-ui-toolkit.html: the manual acceptance check for the browser-only paths.

✅ Landed: all of the above, plus simultaneous mode (§1), modal capture / region-draw (captureInteraction/releaseInteraction + ui/regionDraw.js → a SpatialFilter scoping layer.getStats({ filter })), and the fimviz/ui subpath export (dist/ui.js, its own webpack entry pulling only the small pure deps — no geotiff/Maps-loader/GDAL — so a headless consumer importing fimviz/ui downloads only that). Also createInfoWindow/bindFeatureInfo/propsTable, bindHoverValue, and createOperationsPanel (§2).

✅ Landed: the layer tier (ui/layerPanel.js — createLayerPanel/createLayerSelect/layerLabel, plus FimMap.applyLayerOrder(), FimMap.whenIdle() and the applyLayerOrder/whenIdle provider methods). A list of fim.layers with show/hide, reorder, fit and remove, and map clicks resolved to the layer stack under the pointer with cycling through overlaps. Four decisions worth keeping:

FimMap also emits layers:changed now (add/remove): layers is a plain public array, so a view over it could otherwise only poll.

✅ Landed: the selection tier. createRegionDraw grew from a click-per-vertex polygon into four tools — polygon · rectangle · freehand · brush (REGION_MODES, setMode()) — with Enter/Esc/Backspace keys, undo(), and a live onPreview. The reason it was cheap: all four already shared an output — rings of {lat,lng}, which SpatialFilter, dataset.mask() and layer.getStats({ filter }) have always taken — so the modes differ only in how they fill the ring list, and the brush's per-sample stamps are just a multi-polygon, which SpatialFilter already unions. Two supporting decisions, both in DECISIONS §1.1/§2.1: the drag modes take pan-by-drag away for the length of a stroke via a new optional provider method setDraggable (tracing and panning are the same gesture), and the trailing click that follows every press-drag-release is swallowed rather than allowed to fall through to layer dispatch.

The first browser pass then found the half that was missing: nothing was drawing the shape. ui/regionOverlay.js (createRegionOverlay/regionGeoJSON) renders it through a new engine seam, FimMap.addScratchVector — the provider's ordinary vector tier, but deliberately not a Layer. Vertices appear from the first click, an open edge at two points, a closed ring at three. The same pass added viewMetrics to the provider contract so brushRadius can be a screen size ('2vw'), re-resolved per stamp, instead of a ground size that doubles under the cursor as you zoom.

Deferred, with a finding: damage/velocity control presets are deliberately NOT shipped. VelocityLayer is a Google-only animated canvas driven by positional params (particle density, colour stops, fade alphas — no simple settings hook), and DamageLayer is app-tier, so a velocityControls/ damageControls preset would bind to knobs that don't exist. The generalized createToolsPanel(root, { layer, controls }) already serves them (pass a custom control spec) — the real missing piece is each subsystem surfacing its knobs as a LayerSettings sub-model, which is subsystem/app work (and browser-only), not an engine preset.

6. Tree-shakeable subpaths (fimviz/core)

Problem, as originally traced. The barrel forces every consumer to download the whole engine: mount.js (→ FimViz/mount/parseFile, which nearly every consumer imports) reaches io/parse.js, which statically imported @tmcw/togeojson, shpjs, jszip, geotiff, and @turf/turf — unconditionally, whether or not the consumer ever parses a file. Separately, io/materializers.js (reached because the barrel re-exports registerBuiltinMaterializers from it) runs registerBuiltinMaterializers() at module scope — a real side effect, which named-export tree-shaking cannot remove regardless of usage analysis. Meanwhile Dataset, Storage, ColorScale, Legend, Stats, Filter/SpatialFilter/PredicateFilter, Layer/RasterLayer/VectorLayer, createMap/ registerMapProvider, ComparisonLayer/EnsembleAggregationLayer, and the raster-grid ops have zero static third-party dependencies (the Google/Leaflet SDK loaders in mapProvider.js are already import()-lazy) — a consumer who only wants to render/analyze Datasets they already have (inline data, or decoded themselves) pays for parsing dependencies it never touches.

Design. A second subpath, fimviz/core (own webpack entry + exports map entry, mechanically the same shape as the already-shipped fimviz/ui split): everything in the zero-dependency cluster above, minus mount/FimViz/parseFile/csvHeaders/wktToGeometry/the vendored parser re-exports (fromArrayBuffer/kml/shp/Loader). Requires moving registerBuiltinMaterializers()'s auto-run out of io/materializers.js's module scope into an explicit call the full fimviz entry makes once. The GDAL reprojector no longer needs the equivalent treatment — it's wired as materialize.js's lazy registerDefaultReprojectorLoader fallback (a closure over a dynamic import()), so fimviz/core gets ds.reproject(crs).grid() for free with no eager GDAL cost either way; only the materializer auto-run is the thing a fimviz/core entry would need to opt out of. fimviz (unchanged) stays the zero-setup default for the common boot-a-map-and-parse-files case; fimviz/core is for a consumer building/rendering Datasets without the parse layer.

Not started, and the case for it is now much weaker — because the payload problem was attacked at the dependency level instead, which needed no new entry point and no API change:

dist/fimviz.js went 678 KB → 201 KB (−71%), and what is left is roughly half our own source. A fimviz/core entry would now be carving up that half — a much smaller prize than the original tracing suggested, for a permanent second entry point, a second exports mapping, a docs split, and the "which subpath is this class in?" question every consumer then has to answer. Two subpaths (fimviz + fimviz/ui) remain the shipped shape; the barrel's name count was addressed separately by moving each registry onto the type it serves (Layer.registerType, ColorScale.registerPalette, …).

7. GDAL lifecycle abstraction (over callGdal)

Problem. callGdal(method, ...params) (landed — geo/gdal.js, barrel-exported from lib.js via the same dynamic-import() pattern as registerGdalReprojector) is a deliberately raw, generalized escape hatch: it just resolves the gdal3.js singleton and calls Gdal[method](...params). That means every caller repeats the full gdal3.js lifecycle by hand for the common case — construct a File, callGdal('open', file), run the actual operation, callGdal('close', dataset), callGdal('getFileBytes', outPath) to get bytes back, with no wrapper absorbing any of that ceremony. warpTo/warpToGrid already hide exactly this boilerplate, but only for gdalwarp — the one utility they were written for; callGdal intentionally didn't generalize that lifecycle handling when it landed, only the method dispatch.

Design (not started — this section records the shape, not a landed slice). Two convenience layers over callGdal, matching the two shapes its own doc comment already categorizes:

First slice. runGdal (the dataset-in/bytes-out wrapper) — highest value, and a close enough mechanical match to warpTo/warpToGrid's existing internals that it's low-risk to extract. The info-only wrapper and the Dataset-level convenience are follow-ons once real usage shows which GDAL utilities beyond gdalwarp callers actually reach for.

8. Multi-dimensional formats & real temporal datasets (SciWrid Toolkit as a materializer)

Problem. §3 names NetCDF as its first slice — "exercises the axes model end-to-end, unlocks temporal/ensemble" — and it is still unwritten, along with GRIB2 and Zarr. Meanwhile the axes model itself has only ever been driven by FIM Scenario, whose shape is one file per timestep: an axis entry's ref is a URL, and select() "resolves one axis entry to a child URL-rooted Dataset". Every real multi-dimensional scientific format is the inverse — one file, many timesteps, addressed by index or date. So the engine has a temporal axis that has never met a temporal file, and no way to read one.

Writing GRIB2/NetCDF/HDF5/Zarr decoders ourselves is the §3 "research needed" item (browser viability and size of each WASM/JS parser) and is a project in its own right.

Design. Treat SciWrid Toolkit — a sibling lab library, WASM + JS, reading GRIB2/NetCDF3/NetCDF4-HDF5/Zarr/TIFF-COG/Parquet/Kerchunk — as an implementation behind Dataset.registerMaterializer, not as a dependency of the engine. The seam already exists and is exactly one function wide; nothing in package/ learns these formats.

The shapes line up unusually well, which is what makes this a materializer rather than a port:

SciWrid extractGrid FIMViz RasterGrid
pixels data: Float32Array pixels
order row-major, row 0 = maxLat row 0 = north (rasterOps.js's zonalStats/maskGrid row→lat math)
missing NaN NaN → transparent on colorize, excluded from Stats
extent bbox [minLon,minLat,maxLon,maxLat] bounds {west,south,east,north}
CRS resampled to WGS84 lat/lon EPSG:4326 — what both providers already accept

The CRS row is the quiet payoff: extractGrid resamples onto a geographic bbox, so these formats arrive already renderable and never touch the GDAL warp. Unlike reproject(), this path also runs under Node, so it is coverable by npm test rather than joining §5.3's owed browser verification.

Known friction, priced in rather than discovered later:

How it is wired today (not yet a published dependency). SciWrid is vendored as a packed tarball, vendor/sciwrid-toolkit-<version>.tgz, referenced from package.json as "sciwrid-toolkit": "file:vendor/sciwrid-toolkit-0.1.0.tgz". npm pack honours SciWrid's own files: ["dist", "README.md", "LICENSE"], so what lands is the built bundle only — 11 files, ~386 KB unpacked, no examples/ fixtures (194 MB), no .git (219 MB), no C sources. Installed with --omit=optional: h5wasm/numcodecs/hyparquet/jsfive/jpeg-js are SciWrid's optionalDependencies and are lazy-loaded from a CDN in the browser anyway — add only the ones a landed slice actually needs under Node (NetCDF4 will want h5wasm). The tarball is the version pin, and it keeps a fresh clone installable without a second repository. To refresh:

cd ../SciWrid-Toolkit && git pull && npm run build      # dist/ is what gets packed
npm pack --pack-destination ../FIMViz.js/vendor
cd ../FIMViz.js && npm install file:vendor/sciwrid-toolkit-<version>.tgz --omit=optional

First slice. NetCDF4 only, one vertical: generalized axis ref → materializer → select()/ reduce() → RasterLayer.render() → a time slider on an example page. That proves the seam end-to-end on the format §3 already picked; GRIB2 and Zarr then become repeat applications of the same glue rather than new design. Stats/SpatialFilter/PredicateFilter/ColorScale/Legend are grid-agnostic and need no work — a temporal statistic is a loop over select().

✅ Landed: the axis-ref generalization (package/dataset.js, node-tested). An entry's ref may now be { select: {…} } — an in-file selector — alongside the existing URL and named-variant forms; select() then returns a child rooted on the parent's own bytes/URL (no second fetch, same format, no axes of its own) and the selection reaches the decoder as root.select. Discriminated on an object-valued select key, so a named URL variant that happens to be called select is still a variant. Round-trips through toRecord/fromRecord. Existing materializers are untouched: root.select is simply absent for ordinary sources.

✅ Landed: the NetCDF4 vertical (io/sciwrid.js + test/sciwrid.test.mjs). parseSciwrid(source, opts?) turns a scan() into a lazy Dataset carrying a time axis of selector refs, one entry per timestep; registerSciwridFormats() registers the decoder for netcdf4/netcdf3/grib2/zarr. Verified end-to-end against a real 120-step NLDAS-2 NetCDF4 (Hurricane Idalia): select() → one decoded slice, clip()/Stats unchanged on it, and reduce('mean'|'max') collapsing all 120 timesteps in ~3 s with no new grid math — the predicted payoff of routing selectors through select(). Four decisions worth keeping:

h5wasm is a devDependency only: SciWrid lazy-loads it from a CDN in the browser and from npm under Node, so it is needed to run the tests and not to ship. A source-scan test asserts io/materializers.js never names the adapter, which is what keeps the wasm out of the default bundle.

✅ Landed: GRIB2 and Zarr v2, the predicted "repeat applications of the same glue" — nearly true, with one format difference worth recording. Both are node-tested against vendored Idalia fixtures (Zarr as-is at 1.9 MB; the 26 MB Stage IV GRIB2 trimmed to 4 messages / 930 KB with SciWrid's own trim()), so all three formats the adapter registers are now exercised rather than merely claimed.

✅ Landed: NetCDF3, exercised rather than assumed — and it is the weakest of the four, which is the point of testing it. It decodes (shape parses, pixels come back correct), but scan() surfaces neither an extent nor a time axis for it: the extent because of the WASM-path gap above, the times because SciWrid's own docs record that a wp_nc3_get_time_units_json accessor is still needed. So a NetCDF3 file arrives as a single grid with a mandatory extent override — select() and reduce() have no axis to work on. Registered and usable, but the temporal half of §8 does not apply to it, and a test pins exactly that so the limitation cannot quietly change.

⚠️ Correction to the paragraph above: "no time axis" was the wrong conclusion from "no times". Driving a real 24-step NetCDF3 (the CMIP tos_O1_2001-2002 sample) showed that extractGrid's time index works fine on that path — three indices returned three demonstrably different fields. What is missing is only the labels, which is what the outstanding CF-units accessor supplies. The dimension itself is declared in the variable's own shape and is perfectly indexable. Reading "scan() surfaces no times" as "the dimension is unreachable" cost NetCDF3 its entire temporal half for no reason, and — worse — turned a modellable dimension into an allowExtraDims acknowledgement, which is the API telling the user to accept a silent collapse that was never actually necessary. Superseded by the generalized axis below.

✅ Landed: these formats are no longer opt-in — they are just formats. detectFormat gained .nc/.nc4/.cdf/.grib/.grib2/.grb2/.zarr, and parseSource routes them to the adapter, so fim.addDataset(file) opens a NetCDF exactly the way it opens a GeoTIFF. The original design made the adapter opt-in as the mechanism for keeping its payload out of everyone's bundle; that conflated two things, and separating them costs nothing:

✅ Landed: the series axis and the grid conventions are parameters, not assumptions. Everything the adapter used to infer silently is now a documented default with an override beside it, so a file that does not match CF's overwhelming majority is a configuration problem rather than an unsupported one.

✅ Landed: io/netcdf3.js — the one container format we read ourselves, and only its header. NetCDF3 needed both a hand-supplied extent and (before the index axis) an allowExtraDims acknowledgement. Neither limitation was ever a property of the files: they carry lon/lat/time coordinate variables with CF units in the header, and only SciWrid's WASM path fails to surface them. So a small, dependency-free reader now supplies exactly that, and a NetCDF3 file opens with no options at all — extent from cell edges, real timestamps, select()/reduce() working.

This is a deliberate exception to "we do not write decoders" (§3's "research needed" item), and it is narrow on purpose:

What is deliberately NOT a parameter, and why. Which dimension is the series axis. scan() reports a variable's shape as bare numbers with no dimension names (VariableInfo carries shape and ndims, nothing more), and extractGrid exposes exactly one index knob — time. So the series axis is necessarily the outermost non-spatial dimension, and an option to select a different one would be an option nothing downstream could honour. Offering it would be worse than not having it: the caller would believe a claim the library cannot keep. Closing this needs either dimension names on scan() or a general index selector on extractGrid — both upstream, both preferable to decoding the container ourselves, which would mean a second reader for every format.

8.1 Which grids we actually support (scope, and the silent-failure guard)

Recorded because "it reads NetCDF" is far too coarse a claim: the format is rarely the hard part — the coordinate geometry is. What we support is rectilinear geographic grids, with a manual extent override for anything else. Everything below is a horizontal-grid type real files use:

Geometry How coordinates are stored Typical sources Status
Regular lat/lon 1-D lat[]/lon[], even spacing NLDAS, AORC, most reanalysis ✅ works
Gaussian 1-D lat at quadrature points, uneven ECMWF/IFS, GRIB template 40 ⚠️ decoded as evenly spaced (below)
Projected rectilinear 1-D x[]/y[] in metres + grid_mapping HRRR, RAP, NAM, WRF, MODIS sinusoidal ✅ now rejected loudly
Rotated pole 1-D rlat/rlon in rotated degrees CORDEX, COSMO, HIRLAM 🔴 still silently wrong
Curvilinear 2-D lat(j,i)/lon(j,i) NEMO/MOM/POP, ocean tos, NCEP Stage IV ✅ throws; override works
Unstructured / mesh lat(ncells)+connectivity ICON, MPAS, FESOM, ADCIRC, SCHISM ❌ different data model
Reduced Gaussian rows of differing length ECMWF GRIB ❌
Cubed-sphere / icosahedral tiles/faces GFDL FV3, MPAS ❌
Swath 2-D geolocation, irregular, bowtie gaps MODIS L2, VIIRS, TROPOMI ❌
DGGS cell IDs, no coordinates H3, S2, geohash ❌

Read that table as being about geometry, not about which files work. Two independent things decide whether a file lands on the map: its grid geometry (above) and whether its format path reports an extent at all. scan() populates a bbox only on the netcdf4/zarr/parquet paths, so every GRIB2 and NetCDF3 file needs an extent supplied regardless of geometry — NCEP Stage IV appears in the curvilinear row because it genuinely is curvilinear, but it would need an override even if it were a plain lat/lon grid. Conversely a curvilinear NetCDF4 file fails for the geometry reason. The thrown error distinguishes them, because the two send you looking in completely different places.

The guard, and why it earns its place. scan() derives a bbox from the min/max of whatever 1-D variables are named like coordinates — and its matcher accepts x/y and rlat/rlon (sciwrid-lib.js's coordinate detection). A projected file therefore yields something like [-2699020, -1588806, 2697980, 1588806]: four finite numbers, max > min, and metres rather than degrees. That passed every check we had, so the data would have been placed confidently and wrongly. A wrong extent is worse than a missing one — a missing one throws, a wrong one produces a map nobody questions — so geographicBboxProblem() now range-checks any bbox as degrees (|lat| <= 90, |lon| <= 360, both conventions allowed) and rejects it into the same actionable "pass an extent" error a curvilinear file already gets.

Two known gaps, stated rather than papered over:

Both are upstream fixes, not ours; the useful thing we can do is not pretend they're handled.

✅ Landed: a guard for unmodelled dimensions. T(time, level, lat, lon) is ordinary output (ERA5/GFS/CMIP), and the adapter models only (lat, lon) + one time axis. Left alone that fails in the worst available way — a normal-looking time scrubber over a vertical level nobody chose, correctly placed and quietly wrong. extractGrid exposes no level/member/band selector, so we cannot resolve it either; parseSciwrid therefore throws when a variable declares more dimensions than are modelled, and allowExtraDims: true is the caller's acknowledgement, recording meta.extraDims and the raw shape rather than losing them.

The check is arithmetic on the declared shape, not on what the reader admits to — which is what makes it useful. It immediately caught our own NetCDF3 fixture: sample.nc3 is (time=3, lat=4, lon=5), but since scan() surfaces no times for netcdf3, dimension 0 was invisible and unmodelled, and had been collapsed unannounced since the format landed. Trusting the reader's silence would have hidden exactly the bug the guard exists to find.

That finding has since been taken one step further, and it is the more useful reading. The guard was right that dimension 0 was unmodelled; the mistake was concluding it was unmodellable. The synthesized index axis (§8 above) models it, so sample.nc3 now yields a 3-step axis and trips nothing — the guard fires only for what remains genuinely unreachable, which for a 4-D variable it still does. A guard that says "something is being dropped" is doing its job; it is not evidence that the thing must stay dropped.

Beyond the horizontal grid, three axis families matter for this domain and none is modelled yet: vertical coordinates (pressure/height/depth are directly usable; sigma, hybrid sigma-pressure and ocean s-coordinates are dimensionless and need formula_terms plus a surface field to become real altitudes), forecast reference time × lead time (a genuinely 2-D temporal structure, native to GRIB2 — expressible with our N-axis model, but a different mental model from one time line), and ensemble member / threshold / percentile axes, which map onto EnsembleAggregationLayer and probabilistic flood products. Folding those in is later work; §8's slice is one time axis.

Surfaced by the example, fixed in the engine: ColorScale.getColor(null)/('') coerced to 0 and returned the domain minimum's colour — "no data" rendering as "the lowest reading", the one confusion missingColor exists to prevent — while NaN escaped as the malformed string rgb(NaN, NaN, NaN), missingColor was ignored on the palette path entirely, and getValues() returned undefineds for every continuous scale. All four fixed with regression tests; the raster path was never affected because colorizeGrid pre-filters NaN itself.

Deliberately out of this slice:

Cross-cutting notes