Farm stack foundation layer: spec v0.1#

Oct 7, 2026 · @anun333

Status: draft. Revises v0, which is kept unchanged as the record of the original draft. Changes and their rationale are listed in Changes from v0 at the end. Options for every undecided item are in open-questions.md.

Purpose and scope#

Layer zero is the data and device contract every farm app builds on: identity, time, records, observations, devices, messaging and commands. It ships no end-user features. Its job is to make sensor data trustworthy, farmer-owned and portable.

It is for small and mid-size diversified farms, DIY ag builders and researchers, and for developers building apps on their behalf.

In scope:

• A canonical data model for records and sensor observations
• Device, calibration and health tracking
• A message format, link drivers and store-and-forward delivery
• A command contract with acknowledgements and node-side limits
• Local-first storage, export and revocable sharing
• A driver SDK, conformance tests and import/export adapters

Out of scope (built on top): rules and alerts, irrigation logic, dashboards and UI, robots and guidance, ML.

Done means three different apps (an irrigation controller, a livestock water monitor and a research data logger) can each be built without changing layer zero. This is an acceptance test, not a slogan: §15 records what each of the three needs and which clause provides it.

What v0 actually ships#

The full scope above is the contract. The first release implements one vertical slice of it, end to end:

Since the table was written, sharing grants were built after all (sharing.py, Q16), with tokens for the data service (§8.2). The rest of the deferred column stands.

The substrate is a simulated farm, not hardware. The contract is built and exercised against a deterministic fictional farm first, because layer zero's hardest clauses are all about failure — reboots, backlog drains, drift, lost acknowledgements — and waiting for a real farm to supply one of each is a multi-year sampling problem. This moves the pilot farm off the development critical path and leaves it on the claims path: §9.2's growing-season requirement is unchanged, and a simulator never satisfies it. See VIRTUAL_FARM.md.

Commands are deferred on purpose. They carry the liability, they are the hardest part of the contract to change once hardware is in the field, and they benefit most from a season of evidence on the observation path first. Nothing in this spec is weakened by shipping them second; the contract is written now so node firmware can be built against it, and frozen after the first season.

Design principles#

Every rule below traces to a pain point farmers or field deployments report.

Each principle has an enforcement point, and a principle with no enforcement point is a wish. §16 maps each one to the clause that makes it true.

1. Identity#

Every entity gets a UUIDv7, generated offline with no coordination.

• Labels are not keys. Human names ("north bed") are separate, editable labels. Renaming breaks nothing. A label is a log record (sdk/model.py Label), so names are exported, replayed and audited like everything else.
• A rename appends; it never overwrites. The new label supersedes the old one, which stays in history, because names outlive themselves in other documents: a certifier's report from last spring says "north bed", and after the bed is renamed the question "which entity was that?" MUST still be answerable. A rename MUST NOT supersede another entity's label, or a name could be silently re-pointed.
• One label per entity per language (BCP 47). Layer 4 is bilingual (§11). A missing translation falls back — a regional variant to its primary language, then to the farm's default — and the caller MUST be able to see which language it actually received, so a UI can mark the gap rather than pass English off as the Spanish it asked for.
• IDs do not collide because they are UUIDs. Every entity also belongs to a site, but the site is for scoping, authorization and sharing — not for uniqueness. Site MUST NOT be part of any key, or renaming or merging a site would break references.
• UUIDv7 order is ingest order, not measurement order. Observation IDs are minted by the gateway at ingest, so a reading that spent three days in a node's buffer sorts after readings taken later. Implementations MUST NOT order observations by id. Ordering rules are in §2.
• Radio nodes send short IDs. A 16-byte UUID in every LoRa packet wastes airtime. Nodes send their native short ID (LoRaWAN DevEUI or Meshtastic node number) plus a channel index; the registry maps that pair to the full point ID.
• Node message identity is (site, node, boot, seq). seq alone is not enough: a solar node that browns out nightly restarts its counter, and a gateway deduplicating on (node, seq) would silently discard every reading after the first reboot. boot is a counter the node increments once per cold start and holds in non-volatile memory — a few bytes of EEPROM or flash, and the only non-volatile state a node is required to keep.
• seq is 16-bit, wraps to zero, and resets to zero on each boot. Wrap is unambiguous because boot advances on any power cycle and the gateway tracks the highest seq seen per (node, boot); a decrease of more than half the range within one boot is a wrap, not a replay.

Wire identity for a reading is (node, boot, seq, channel, index) — see §5. channel is one byte, assigned by the device driver and local to the node, so a node can be swapped without reprovisioning a shared index space. A registry-assigned 16-bit point index is an optional extension for nodes with more than 255 points.

2. Time#

Every observation carries five time fields, so ordering survives outages and bad clocks.

• Nodes with no clock send an age, stamped at transmit time. The node reports "measured N seconds ago" and the gateway computes measured_at = received_at − N. A node draining a store-and-forward buffer MUST recompute the age for each message as it transmits. Stamping the age at sample time and then buffering for three days yields a measured_at wrong by the dwell time — the single easiest way to corrupt a dataset silently, and the one a driver is most likely to get wrong.
• measured_at_uncertainty is mandatory and never zero. It is the honest width of the estimate: sub-second for gps, the sync error plus link latency for network, the drift budget since last sync for rtc_unsynced, and at least the node's sample interval plus transmit latency for none. A research logger cannot use a timestamp it cannot bound, and clock_quality is a category, not a number.
• Everything is stored in UTC. The farm's timezone is a property of the site and is used only for display.
• Ordering. Across nodes, order by measured_at, and treat two observations whose uncertainty intervals overlap as unordered. Within one node, (boot, seq) is authoritative and MUST be preferred over measured_at, because a node's own counter is reliable even when its clock is not.
• SensorThings mapping. measured_at is phenomenonTime. The other four have no equivalent and travel in the observation's extension object — parameters in SensorThings 1.1, renamed properties in the 2.0 draft. See notes/sensorthings-2.0.md.

3. Core data model#

Layer zero has two models joined by shared IDs: records follow farmOS conventions, sensing follows OGC SensorThings, and both write to one log.

A sensor asset and its Thing share one ID, as do a field or bed asset and the feature of interest it represents. Each observation belongs to a datastream, which ties one sensor, one observed property and one unit to its Thing.

Externally, records are served through a farmOS-compatible API and sensor data through a SensorThings-compatible API, so existing tools in both ecosystems work on day one.

3.1 Datastreams#

A datastream ties one sensor, one observed property and one unit to its Thing. The SensorThings 2.0 draft permits several observed properties per datastream; one property per datastream is a subset of that, so this model maps unchanged to 1.1 and to 2.0.

Sampling and latency are different numbers, and the second one spends the battery. A soil-moisture series sampled hourly and a trough float sampled every five minutes can share a node, but what the radio costs is set by how quickly each series must arrive, not by how often it is read. Reading a sensor is a rounding error — measured at under 2% of a node's consumption in every realistic configuration — while one transmit at the slowest data rate costs more than a day of sampling. A series that is merely informative declares a loose max_latency and rides along in whatever message is next; a series someone will act on declares a tight one and pays for it.

Feature lives on the datastream, not on each observation. For a fixed installed sensor the observed feature never changes, so repeating its UUID on every row wastes storage and airtime for no information. An observation MAY carry its own feature, and does so only when it genuinely varies: a handheld meter, a sample, a sensor on a moving thing.

3.2 Observations are immutable, and calibration is a separate series#

An observation is written once and never altered. There is no mutable field on it — not the result, not the quality flag. Two consequences, and they are the heart of Honest data:

Raw and calibrated values are separate datastreams.

• The raw datastream holds exactly what the node sent. It is never superseded, never recalculated, and is the archival truth. If everything else is lost or wrong, this is what a farmer still owns.
• The calibrated datastream is derived from it, one per raw datastream, carrying derived_from and, on each observation, the calibration record that produced that value.

A client that wants raw values reads one datastream; a client that wants usable values reads the other. Both are ordinary SensorThings Datastreams, so this is honest and standard, rather than honest by convention.

Recalibration appends; it never rewrites and never duplicates a timestamp.

A new calibration produces new observations in the calibrated datastream, each carrying supersedes pointing at the row it replaces. The superseded row stays in the log forever. Exactly one live observation exists per (datastream, measured_at), where live means nothing supersedes it.

This is the clause that makes the SensorThings promise survive the first recalibration. Keeping two results in one series — the v0 design — would have put two observations at the same phenomenonTime in one Datastream, so every existing client would double its data points or show a discontinuity, which is exactly the "works on day one" claim failing on the most ordinary maintenance event there is.

• Calibration is selected by measured_at, never by ingest time. The gateway MUST apply the calibration that was in force when the reading was taken. Applying whatever is current at ingest would silently give wrong coefficients to every reading that arrived late from a buffer — a violation of Honest data produced entirely by an unstated default.
• Derivation is reproducible. (raw value, calibration record, driver version) MUST be sufficient to recompute a derived result exactly.
• A calibration MAY declare an output resolution. 0.001 x 86 is exactly 0.08600000000000001 in IEEE 754 — correct, reproducible, and a claim of seventeen significant figures that no probe can support. Rounding to a declared resolution is not cleaning the data; it is declining to assert precision the instrument does not have. Because the resolution is part of the calibration record, the derivation stays reproducible, and a client is not handed false precision to display. A derivation that depends on anything else — ambient state, wall-clock time, another series — MUST be recorded in provenance.

A forecast is not a measurement, and is not filed as one (Q26). It has two times — the hour it is valid for (measured_at, served as phenomenonTime) and the run that produced it (issued_at, served as resultTime, the case O&M defines that field for) — and it is issued again and again: every run predicts tomorrow afternoon anew. So it lives on a datastream of kind = forecast, whose sensor is the forecast model and which derives from nothing on the farm. Several observations there share a measured_at, one per issue, and the live one is the latest issued_at: no supersedes is written, because a newer run replaces an older one as the best guess rather than correcting it, and a run that arrives late, after a newer one, must not take over. Every run stays in the log, which is what lets a forecast be scored against the farm's own sensors. A forecast may be of a probability — the chance of rain — and then it declares the event it is the chance of (probability_of: "≥ 0.254 mm within the hour"), so the number cannot be read as a depth. A measurement carries no issued_at: its result time is not recorded, and received_at is not it. A forecast datastream's staleness (§4.2) is judged from its latest issue, since its newest measured_at lies in the future. Of §4.2's checks only the range check runs on a forecast: a value outside what the variable can be is a broken feed or a unit confused upstream, and saying so is honest, but a forecast holding one value for a day is not a stuck probe, a front is not a spike, and a forecast has no redundant neighbour to cross-check.

An aggregate is a node's summary of a window, not a reading, and is labelled as one (Q20). A node that samples often and reports seldom may send, for a low-urgency series, the window's sample count, minimum, maximum, and how long its value has been unchanged — the length of the current run of identical samples, carried across windows. Those four numbers are what the flatline and range checks need, so a stuck probe is seen at the sampling resolution for the airtime of the reporting one. The node has computed something, so the series is not raw: it lives on a datastream of kind = aggregate with a declared window_s, its observations carry the four numbers as an object, and measured_at is the window's end (served as a phenomenonTime interval). It is never superseded, like raw. Its calibrated series is derived, carries the same window_s, and moves the extremes through the calibration — exact for a monotonic calibration, which a probe's is — leaving the count and the run alone. Only range and flatline run on an aggregate: rate, fall and spike need the order of readings inside the window, which the node did not send, and that is the price of the airtime saved.

3.3 Quality is an assertion with provenance#

Quality is not a column on the observation. It is its own append-only record, because a quality judgement is a claim made by a particular check, under a particular threshold, at a particular time — and all three change.

Why this shape:

• A single enum cannot express reality. A sensor can be stuck and out of range at once. flags is a set; usable is the one value an app that does not care about detail can read. Two conforming implementations now agree on the same input, which a single enum did not guarantee — fatal for a suite meant to certify conformance.
• Changing a threshold does not rewrite history. Every assertion names the threshold version it used. Retuning a threshold produces new assertions that supersede the old ones, and the old ones remain visible, so a false alarm can be traced to the threshold that caused it and the fix can be audited.
• A person's override is a new assertion, not an edit. assertor = person with a higher precedence than foundation. The original machine assertion stays.
• Staleness is not here. See §4.2.
• Device warnings are not here. Battery and signal are properties of a device, not of a reading; see §4.3.

One assertion per assessment run. check names an assessment, not a single check, and flags carries the union of everything that fired in it. §4.2 lists four checks that each produce one flag, and the precedence rule below selects exactly one live assertion — so if each check wrote its own assertion, two firing together would mean one wins and the other's flag silently disappears. That is the loss this model exists to prevent, so the foundation runs the suite, unions the flags, and records per-check detail in evidence. A person's override is still a separate assertion, which is what precedence is for.

Precedence, so implementations agree: the live assertion for an observation is the non-superseded one with the highest precedence, where person outranks foundation; ties break by asserted_at, then by id. usable is derived as bad if stuck or out_of_range is set, questionable if only suspect or rate_exceeded is set, good if flags is empty — unless a person asserts otherwise.

SensorThings mapping. 1.1 carries resultQuality on Observation, typed DQ_Element with multiplicity zero-to-many, which fits this model natively. The 2.0 draft has no resultQuality attribute at all — Table 17 lists only id, phenomenonTime, result, resultTime, validTime and properties. Under 2.0 the live assertion travels in properties. This is worth raising with OGC while 2.0 is unpublished; see notes/sensorthings-2.0.md.

3.4 Units and observed properties#

• Units use UCUM codes, declared once on the datastream. SensorThings 1.1 already says unitOfMeasurement values SHOULD follow UCUM, and the 2.0 draft moves the unit inside a SWE-Common resultType, which also carries UCUM codes — so this is the standards-aligned choice in both versions rather than a preference.
• Observed properties reference a shared vocabulary by URI, never free text. Decided 2026-10-07 (open-questions.md Q4), layered by what each vocabulary is for rather than forcing one to cover everything:
  • CF Standard Names for physical measurands, served as resolvable URIs under http://vocab.nerc.ac.uk/standard_name/.
  • ODM2 / CUAHSI (http://vocabulary.odm2.org/variablename/) for the instrument and water terms CF does not carry — battery voltage, signal strength. CF describes the environment, not the device, and §4.3 needs both.
  • AGROVOC for the records side — crops, treatments, practices — where its multilingualism serves layer 4. It is not a vocabulary of measurands and MUST NOT be used to declare what a sensor reports.
  • A local term only for a genuine gap in all of the above, which MUST be declared in one place with its reason and MUST be published upstream rather than accumulated. The durable namespace for local terms is open as Q4c.
• A declared URI MUST resolve. This is the clause that makes the previous one mean anything: an unresolvable identifier is free text with extra syntax, and it fails silently, because nothing about .../cf/soil_moisture_content looks wrong until someone tries to fetch it. Resolution is checked against a registry, not at declaration time: a farm gateway may not see the internet for days, so the check at declaration is offline — the term must be registered, and the unit must be one of that term's — and the registry is what gets fetched, when a connection exists. Implemented as vocab.check, run by State.validate on every datastream, and make vocab-resolve.
• The unit a term is declared in MUST be consistent with the term's own definition. A fill fraction declared under a term defined as a length is a unit error that passes every syntactic check, and it is the specific mistake this project made and corrected — see the Q4 entry.
• A datastream with no meaningful unit (a boolean, a state) sets the unit to null rather than inventing one.
• A raw datastream declares the unit its values are in (Q21). A node that sends scaled integers — a probe's 280 for 0.280 cm³/cm³ — sends numbers, so its raw series declares UCUM's dimensionless 1, and the scale lives in the calibration, the record that is allowed to change. A node that sends a value already in a unit (a digital thermometer's 21.5 °C) declares that unit. A raw series that a calibration scales by a power of ten while declaring the calibrated series' unit is reported by validation: it is the plausible-looking declaration no other check falsifies.

4. Devices, calibration and health#

Sensors are maintained assets with their own history, because drift and hardware failure are the most common reasons field data goes wrong.

4.1 Device record#

The list below describes a view, not a record. Read literally it asks one record to hold "the current boot counter", and a record in an append-only log cannot hold current anything without being mutated — which §3.2 forbids, for reasons that apply here unchanged. So the device is three records and a projection:

Last seen and the current boot counter are derived because they are observations about the device, not claims the device makes about itself. Deriving them also means nobody can edit them, which is the right property for a field a certifier may rely on.

What each part MUST carry:

• Install location, depth and orientation, with dates. On the Installation, never on the device. The question is "how deep was this probe when it took that reading", and a field overwritten on every move cannot answer it. depth_m is positive downwards from the surface — a sign convention nobody states is one somebody gets wrong — and orientation_deg is a compass azimuth. Both are numbers rather than prose, so a wrong one can be caught.
• Installation intervals MUST be half-open and MUST NOT overlap. [installed_at, removed_at), so the instant of a move belongs to exactly one siting. Two open installations make the depth question answer twice, which is worse than not answering: nothing looks wrong. Moving a sensor therefore has two halves, and closing the old installation is the half that gets forgotten — implementations MUST report an overlap rather than picking one.
• Firmware version, also stamped into each reading's provenance.
• Calibration history: method, reference used, coefficients, date, who did it, and the interval it is in force for. A calibration record carries a validity interval, because §3.2 selects calibration by measured_at and that requires knowing what was in force when.
• A maintenance log: cleaned, moved, replaced, recalibrated, serviced. The action is an enumeration, not prose — "when was this probe last cleaned" is a question a certifier asks and it cannot be answered by grepping notes. The vocabulary is closed by this clause, so a missing action is a spec change rather than a local convention. serviced is that change once made (Q22): routine work on a device that is neither a replacement nor a recalibration — a battery changed, a panel cleaned, a desiccant pack swapped — with the detail in the note.
• Declared capabilities from the driver manifest (§8): dedup persistence, delivery confirmation, safe states. An undeclared capability MUST read as unknown, never as a default. §6.3 branches on precisely this: a node that cannot persist its dedup table across a reboot has to say so, and silence is not a denial.
• Datastream.sensor names a Device. It was a bare identifier with no record behind it in v0, which is how a probe could be moved between beds with nothing recording that it had happened.
• A calibration is scoped to the raw channel it converts. One device can report several measurands — a soil probe gives moisture and temperature; a multi-depth probe gives six moistures — so a calibration selected by sensor alone competes with its siblings. Implementations MUST select by sensor and raw datastream, preferring a channel-specific calibration to a sensor-wide one and never applying one that names a different channel. Overlap is checked per channel.
• An installation is a claim, and SHOULD be checked against physics where the physics allows. Soil damps and delays the daily temperature wave with depth, so a probe that reports soil temperature carries its own depth in its record: each damping depth down the swing shrinks by e and the peak arrives about 3.8 hours later. A check that infers depth this way and compares it with the installation record finds the move nobody logged — frost heave, a cultivator catching a cable, a probe pulled to look at and pushed back shallower — which no amount of correct record-keeping can catch. One window is noisy and a real move is permanent, so a displacement MUST persist before it is acted on — five of the last seven daily assessments — and a displacement seen today but not yet persistent is reported as unconfirmed, not as consistent. Where a profile's deeper probes carry too small a wave to calibrate on over two days, the check SHOULD also run over a longer window in which they do, so that the fit is robust: a profile calibrated on two peers cannot outvote a displaced one, and blames the probe extrapolated from it. A persistent displacement from a two-peer fit MUST therefore be reported as concerning the profile, and SHOULD name the probe whose own records — an installation, a maintenance event, a recalibration, or a long silence in its readings — changed before the alarm began, saying so plainly when none or several did. It produces suspect on every reading the device took while displaced — moisture as well as temperature, because the moisture was attributed to a depth it was not measured at — with the implied and recorded depths as evidence. The flag is added to each reading's live judgement rather than replacing it, so a reading already stuck stays bad; it declines where the live judgement is a person's, because a machine flag superseding a human override would erase the override from the live view; and it is idempotent across the overlapping daily windows it runs over. It MUST calibrate against peers rather than a constant, because diffusivity varies with water content, and it MUST use a robust fit, because the peers may be displaced too. It MUST report indeterminate rather than consistent where it cannot see: below the reach of the daily wave, and for a move between beds at one depth, which leaves no thermal signature. That blind spot is covered by a second signal: a farm's own irrigation timers wet each bed at its own time, so a probe's moisture record says which bed's water it answers to. A lateral check correlates each probe's wetting events with every other probe's, by recorded feature, tolerating the lag of a deeper probe. It MUST require a discriminating match before confirming a siting — rain wets every bed at once, so a probe moved out of its bed still matches the bed it left in a rainy window, and a confirmation earned by the weather is not one. Both checks MUST judge only readings taken under the current installation: a window reaching back past a recorded move judges the new record by the old bed's readings.
• A node that keeps GNSS time SHOULD report its position, and its installation SHOULD be checked against it. The fix is computed anyway for timing; discarding it throws away the one signal that sees a lateral move — a node carried to another bed or paddock — which the temperature check cannot. Use medians, so a multipath bounce of a kilometre changes nothing, and a tolerance wider than the fix scatter, so a node a few metres off its post is never flagged for something GNSS cannot resolve. Geometry stays out of the log (§10.1) and is passed to the check.
• Battery, signal and position are properties of the node, and the node is a Device. v0's harness modelled each node's battery and signal as two separate sensor devices — a node's battery named as though it were a probe in the ground.

4.2 Health checks produce quality assertions#

Checks run in the foundation and write assertions (§3.3), never mutations:

A feature with one sensor has no drift detection, and drift is the commonest real failure. Measured on the virtual farm, scoring each fault against ground truth: a single probe catches a disconnected sensor (100% recall) and a frozen one (87%), and catches drift, bias and intermittent noise not at all — 0% recall on all three. (Since the shape checks below, noise is caught alone 40% of the time with every flag correct; drift and bias remain invisible to a single probe.) Every reading is individually plausible and the series looks healthy, so no check that looks at one series has anything to go on. Adding a second probe on the same feature takes those to 79%, 100% and 19%, entirely through cross-check. So redundancy is not a refinement: on a series anyone will act on, it is the difference between seeing three fault modes and seeing none, and that belongs in an installation decision rather than a wish list.

For soil water, a rate limit MUST NOT stand in for a spike check. A rate allowance grows with the gap between readings: 0.00015/s, derived from drip irrigation at fifteen-minute sampling, permits 0.54 m³/m³ in an hour — nearly the whole physical range — and over ISMN's hourly Tennessee record it caught none of 1,849 spikes and jumps. What tells a glitch from the weather is its shape: a spike leaves its level and comes straight back, which rain does not do. Readings that leave their level by at least a threshold, stay out no longer than a short duration, and return MUST be flagged rate_exceeded — judged one reading late, by appending an assertion when the reading that shows the return arrives, under the same rules as any late flag (added to the live judgement, never over a person's, idempotent). And soil fills fast but empties slowly, so a cap on falls is meaningful where a cap on rises is not: a fall faster than soil drains is suspect — not worse, because a wetting front draining at shallow depth can occasionally exceed it. A rise has no cap from rate at all, because without a rain gauge a fault and an irrigation event rise alike. Measured on 1,797 SCAN probes: at 0.05 m³/m³ the shape check raised 0.46 flags per probe-year on readings ISMN rates good, never on the median probe, and most of those on sensors that are plainly broken; at 3-hourly sampling it fires less, not more. A fall cap of 0.05 per hour caught 61% of ISMN's negative jumps. max_rate_per_s remains for measurands where a rate really is the physics.

A step the maintenance log explains is not a fault (Q22). A new battery lifts a node's voltage in one reading; a probe lifted out to clean reads air and comes back. Before the rate, fall or spike check flags a step, it MUST consult the maintenance log of the device the series comes from — that device only, so a node's battery change explains nothing about the soil probe it carries — for an event between the two readings, or within the spike. An explained step raises no flag, and what the check would have said is kept in the assertion's evidence as explained, with the event, so the judgement stays auditable.

Cross-check says suspect and never worse. When two probes disagree it cannot say which is wrong, so calling either one bad would be a guess dressed as a measurement. A farmer deciding which probe to pull needs that distinction.

Flatline latency depends on when the fault begins, not only on the threshold. Part of an identical run can precede the fault, because a healthy probe already repeats overnight when evapotranspiration is near zero. A probe that freezes at dawn is caught sooner than one that freezes at midday, through no merit of the detector — so a latency figure needs the onset time beside it to mean anything.

Flatline MUST NOT be applied to a regulated quantity. A float-controlled trough level, a thermostat-held temperature, a pressure held by a regulator — these are constant by design, and constant is the healthy state. Measured on the virtual farm: a working float holds a trough at one reported value for 226 consecutive samples, nearly nineteen hours. Flatline on such a series is a category error that reports a fault every day. The check that catches a stuck float is range.

A flatline threshold MUST exceed the quantisation dwell. A sensor reports in steps, and when the true value moves more slowly than one step the readings are identical and correct. Measured: a soil probe reporting three decimals, sampled every fifteen minutes, repeats a value for up to eight hours overnight when evapotranspiration is near zero. A six-hour threshold therefore fires nightly on a healthy probe. The floor for flat_seconds is how long the true process takes to move by more than one reported step, which is a property of the sensor's resolution and the process's rate — not a number to pick by feel.

A series MAY learn its flatline from its own history, inside bounds its soil sets (Q24). The default comes from the probe's depth, the soil texture at that depth and the climate — twice the healthy 99.9th-percentile dwell measured for that combination. After a season, a series may replace it with twice its own 99.9th-percentile run of one value, learned only from readings not already flagged stuck, and appended as a new threshold version that records what it was learned from. It MUST NOT learn below the default nor above twice it. Below, because one quiet year is not the soil's range: measured on 30 Tennessee probes learning from their first year, thresholds allowed to fall to half the default raised ten times the false stuck runs in the years after. Above twice, so a probe stuck since installation cannot teach itself that stuck is normal.

A threshold lives on the datastream whose values it bounds. max: 0.55 is soil moisture, so it belongs on the derived series; a limit on the integer a probe reports belongs on the raw one. They are different numbers in different units and putting one on the wrong series makes it silently inert — the check runs, finds nothing to compare against, and reports no objection.

Thresholds are expressed in physical units, not in samples. "Identical values for longer than physically plausible" is a duration, and how long a frozen reading is implausible is a property of the measurand rather than of how often a node samples. A count-based threshold means different things on two farms: twelve samples is twelve hours at hourly sampling and one hour at five-minute, so the "same" threshold catches a fault on one farm and misses it on the other. Flatline therefore takes flat_seconds, with a sample count kept only as a confidence guard — two equal readings an hour apart prove little however long the span between them.

These four contribute to one assertion per run, not one each (§3.3). Cross-check flags only suspect, never worse: when two probes disagree it cannot say which is wrong, and calling either one bad would be a guess dressed as a measurement.

Thresholds live on the datastream with sensible defaults, so apps can tune them without code. Every threshold is a versioned record, and every assertion names the version it used (§3.3). Defaults are deliberately loose — a default that fires often costs the farmer's trust in every flag, which is more expensive than a missed detection.

Staleness is a datastream state, not a reading flag. "No reading for 3× expected_interval" is a statement about a series at a moment in time, not about any particular observation. Flagging the last good reading as stale would mean reaching back and mutating a record that was correct when written, because a later reading failed to arrive — an append-only violation that also destroys the meaning of the value. Staleness is exposed as current datastream health, and the absence itself is already recorded honestly as a gap (§5.4).

4.3 Device health is observations, not side-channel state#

Battery voltage, RSSI and SNR are measurements of the device, so they are datastreams on it like any other. Trends, thresholds, quality assertions, export and audit then work on them for free, instead of needing a parallel mechanism. The SensorThings 2.0 draft explicitly blesses this case: an observation whose target is a sub-part of the Thing itself, battery level being its own example.

A low-battery warning is therefore a threshold on a device datastream, surfacing as device health — not as a flag on an unrelated soil-moisture reading, which is what a device warning value in a reading-level quality enum would have meant.

4.4 Resources#

Pumps, wells and power sources are resource assets with declared capacities (flow, pressure, watts). Bookings against them are enforced in layer zero's command path (§6.5), not only by the layer-1 scheduler. Tasks for people are pending logs, following farmOS, rather than a separate entity.

5. Messaging and link layer#

One compact message format rides over any link; the gateway expands it into full observations.

5.1 A reading's path#

1. The node samples, applies nothing, and packs a compact binary message.
2. The node buffers it until delivery is confirmed by whatever means its link offers (§5.2).
3. A link driver carries it: Meshtastic, LoRaWAN via ChirpStack, wifi, cellular or wired.
4. The gateway resolves IDs through the registry, computes times (§2), drops duplicates on (site, node, boot, seq), writes the raw observation, then derives the calibrated one (§3.2) and runs health checks (§4.2).
5. Raw and derived observations are written to the append-only log and the time-series store, then published on the bus (MQTT for edge devices; bridged to ROS 2 for robots and machinery).

5.2 Messages and batching#

A message carries a format version, (node, boot, seq), and one or more readings. A reading's value is the integer the node sampled or, on a channel bound to an aggregate datastream, the four integers [n, min, max, unchanged] of a window (Q20); a value whose shape is not its channel's is recorded as an event and written to no series. Batching is not an optimisation to add later: one reading per message cannot fit a useful sample rate inside a 1% duty cycle, and retrofitting it would change the dedup contract and the wire format.

message  := version, node_short_id, boot, seq, base_age, dropped, reading+
reading  := channel, age_delta, value

dropped is the count of readings this node evicted from its buffer since its last message, which §5.3 requires it to report so a loss is recorded as a gap rather than as readings that quietly never existed.

Ages are whole seconds, and that bounds the sample interval. Two readings on one channel that compute the same measured_at would be two live observations at one instant, which §3.2 forbids and which breaks every SensorThings client. So a node MUST NOT sample one channel faster than the age field can distinguish: a driver declaring a sub-second interval is a configuration error, not a runtime condition. A gateway that receives such a collision anyway records it and keeps the first reading, because writing both would violate the invariant and discarding silently would lose a reading without saying so.

A reading's identity is (node, boot, seq, index) where index is its position in the message. The dedup key stays (site, node, boot, seq) — one key per message, not per reading — so deduplication, gap detection and batching do not interfere.

base_age is stamped at transmit time (§2); each reading's age_delta is relative to it.

• Link drivers are interchangeable and share one interface: send, receive, link health, and a declared delivery-confirmation capability: confirmed, gateway_receipt or none.
• "Buffer until the link confirms delivery" is not available on the primary target links. Meshtastic and unconfirmed LoRaWAN uplinks return no receipt, and confirmed LoRaWAN uplinks spend a downlink per message, which exhausts the downlink budget and the battery. For a none link, the gateway periodically sends a cumulative acknowledgement — the highest contiguous seq it holds for that (node, boot) — and the node releases its buffer up to that point. One small downlink retires many uplinks.
• Messages must fit the smallest link's payload. In the EU 868 MHz bands a 1% duty-cycle sub-band allows about 36 seconds of transmit time per hour, per sub-band, per device; EU868 also defines 0.1% and 10% sub-bands, and LoRaWAN deployments add a network fair-use policy on top. Treat 36 s/hour as the common case, not as the limit, and take the real budget from the link driver.
• Encoding: CBOR packed arrays with a CDDL schema, versioned with the spec and carrying its version in every packet. Decided on measurement; see notes/q5-encoding-measurement.md. A hand-packed layout is documented as an escape hatch for a node draining a very long backlog at the slowest data rate, distinguished by the version byte.
• node_short_id is optional on the wire, present per link driver. On LoRaWAN the session already identifies the device, so the bytes are waste; on Meshtastic they are required. Measured at four bytes a message, which is a reading every other message at DR0.
• Batch size is computed from the current data rate, never fixed at build time. Sizing a batch to exactly the payload limit is a trap: LoRaWAN MAC commands piggyback in FOpts and shrink the available FRMPayload, and ADR moves the data rate under a running node. The encoder takes a maximum-payload argument and packs what fits, with headroom.
• A cumulative acknowledgement MUST be able to abandon a hole. It cannot pass a frame that no longer exists anywhere, and without a way to give up it never tries: one unrecoverable gap blocks release permanently, including for every reading that arrived perfectly well after it. Measured on a three-day outage — a node holding 800 readings, saturated and evicting, because the gateway's contiguous mark was stuck behind frames that were gone for good. So a hole below the highest sequence seen, which nothing has filled for a stated grace period, is declared permanent and the mark moves past it. The gap was already logged when it was noticed (§5.4); declaring it permanent hides nothing and only admits it is not coming. The grace is measured from when the hole appeared, not from the last frame received: on a busy node the high-water mark keeps advancing while the hole sits untouched, so a grace keyed to the mark never elapses.
• A node's give-up bound MUST be expressed in time, not in attempts. A node holds a reading until it is acknowledged or until a stated horizon passes, after which the reading becomes a recorded gap. An attempt count looks equivalent and is not: four attempts at a half-hour deadline is two hours, so a node configured that way silently loses everything in a three-day outage while appearing to work. The horizon is the farmer-facing number — how long this node holds a reading — and it must exceed the longest outage the deployment intends to survive. Some bound is required, because a resend goes first by design and a frame that can never be acknowledged otherwise blocks every reading behind it.
• A resend MUST reuse its sequence number. This is the same rule §6.3 states for commands, and it holds here for the same reason: the dedup key is (site, node, boot, seq), so a retry under a fresh number is a new message to the gateway. It then writes the same readings again at timestamps that differ by the rounding of a recomputed age — phantom readings a second away from the real ones, in a series nobody can later untangle. Measured on a 6%-loss link: 2498 observations where 768 were expected.
• An acknowledgement deadline is mandatory wherever cumulative acknowledgement is used. A node MUST give up waiting and resend after a stated interval. Without one, a single lost frame stalls the gateway's highest contiguous sequence at the hole below it permanently, and the node holds every later reading until it begins evicting — a node that looks healthy and delivers nothing.
• A node has one clock and MUST use it at both ends of an age. The cancellation that makes §2 work depends on it: subtracting two readings of the same wrong clock removes the error exactly, so a node six hours wrong still reports correct times. Taking the sample time from one source and the transmit time from another destroys the property silently, and nothing downstream can detect it.

5.3 Backlog policy#

A node with a five-minute interval accumulates about 288 readings a day. After a three-day outage it holds ~900 readings and can spend 36 seconds an hour transmitting. What it discards, and in what order, is a contract decision, because the target apps want opposite answers: a livestock water monitor needs the newest reading first, a research logger needs the series unbroken.

buffer_policy on the datastream declares:

A node that evicts MUST report the count, so the loss is recorded as a gap rather than disappearing.

• Under fresh_first, a reading never sent that is newer than every frame awaiting resend goes before the resends, and the resends go newest-first. Through a long outage every frame goes unacknowledged, so when the link returns everything awaiting resend is old. A node that resends first delivers the trough's midnight level at noon, which is the one thing fresh_first exists to prevent. A resend still keeps its sequence number (§5.2), and a node MUST keep draining while anything awaits resend, or the resends wait behind every new sample until the horizon turns them into a gap.
• Only the gateway's highest-contiguous acknowledgement is cumulative. A link that confirms each frame says that frame arrived and nothing about the frames before it; a node that reads it as cumulative releases readings that never arrived. The livestock water monitor's acceptance run measured it: 21 of an outage's 144 trough readings lost, before any reordering, and nearly all of them once the newest frame went first.

5.3.1 The transmit schedule, and where a sensor is installed#

A node's transmit interval is the tightest max_latency among the series it carries. A trough float that must be known within five minutes forces the radio awake every five minutes however sleepy everything else on that node is, so latency is not a per-series cost — it is imposed on every series sharing the radio.

Two consequences, both measured (notes/power-budget.md):

• Slowing a sensor down to spare the sensor saves nothing. Sampling is under 2% of a node's energy. What a higher sample rate costs is the airtime to send what it produced: twelve readings an hour instead of one nearly triples a node's consumption, and almost none of that is the probe.
• Which node a sensor sits on is the power decision, and it outweighs every configuration choice. The same hourly soil probe lasts around 1500 days on its own node and around 130 days on a node a five-minute trough float keeps awake. Nothing in the sample-rate or encoding space comes within an order of magnitude of that.

A latency requirement is a ceiling, not a schedule. max_latency is the longest a reading may wait, not how often the radio should speak. A node sampling twelve readings an hour and sending one message an hour that holds five is permanently behind and will lose data forever while appearing healthy — inside its latency ceiling the whole time. So a node MUST also drain whenever it holds more than one frame's worth and its duty cycle affords the airtime.

An infeasible configuration MUST be refused at registration. Compare what a node produces — the sum of 3600 / sample_interval across its channels — against what it can deliver, which is the readings that fit one frame at the current data rate multiplied by the frames the duty budget affords. If production exceeds delivery, no buffer policy saves it: the node will lose data permanently, and the farmer will discover it as unexplained gaps in month three. The foundation says so when the device is registered, with the three things that fix it: sample less often, raise the data rate, or split the channels across nodes.

So the installation rule: do not co-locate a series with a loose latency on a node whose tightest latency is short, unless the wiring makes it unavoidable. An extra reading that fits a message the node was already sending is close to free; an extra transmission is not. A driver manifest declares both intervals per point (§9.1) so the foundation can tell a farmer which of their nodes is short-lived and why.

5.4 Gaps#

Gaps in (boot, seq) are logged as gaps and never filled with guesses. A boot increment is a reboot, not a gap, and is logged as such. Modules watch for new observations and events through a stable subscription API (§7.5); nothing above layer zero polls.

6. Command contract#

Commands state a desired outcome, carry their own safety bounds, and are safe to send twice, so a lost or repeated message can never leave a pump or valve in a bad state.

6.1 What a command carries#

• A UUIDv7 command ID, held server-side
• A short command sequence, node-scoped, which is what actually travels on a constrained link (§6.2)
• The target point and the desired state, not a toggle: "valve 3 open", never "flip valve 3"
• A maximum duration or expiry, after which the node reverts to its safe state on its own
• Who issued it and why — a user, a regimen, a rule — for the log
• An optional dry_run flag

6.2 Identity on constrained links#

A 16-byte UUID does not belong in a downlink. §1 bans it for uplinks on airtime grounds and the same arithmetic is worse downward: LoRaWAN Class A offers two short receive windows after an uplink, and DR0 gives about 51 bytes of payload for the whole command.

So the UUIDv7 stays server-side, and the wire carries a short command sequence scoped to (node, session) — the same registry-mapping pattern already used for point IDs. Issuer and reason are stamped by the gateway and never transmitted. The registry mapping is part of the audit chain: "which command did the node actually run" resolves through it, so the mapping is itself logged.

6.3 Idempotency has a stated bound#

"Repeats with the same ID are ignored" is only true while the node remembers the ID, and a node's memory is small and clears on power loss. Unbounded, the claim fails in exactly the case the contract exists to prevent: a retry across a brownout runs the pump twice.

• A node MUST ignore a repeated command sequence until that command's expiry has passed, plus a margin declared in its driver manifest.
• A node MUST either persist that set across cold starts, or declare dedup_volatile in its manifest. For a dedup_volatile node, the foundation MUST refuse to issue any command whose double execution is not bounded by a node-side maximum runtime — the remaining protection being the max-runtime limit and the safe state on power loss, which is then stated rather than assumed.
• A retry MUST reuse the same command ID and sequence. Minting a new ID for a retry is a conformance violation, because it defeats every deduplication guarantee above.
• How a node persists it (Q13): FRAM, or a lazily written ring in flash, where the hardware allows; where it does not, the node declares dedup_volatile and the rule above applies. A node never silently has volatile dedup.

6.4 Acknowledgement, and the honest absence of one#

The node answers accepted, rejected with a reason, or completed.

No acknowledgement within the link timeout means unknown, not "not delivered." A command lost on the way out and an acknowledgement lost on the way back are indistinguishable from the server, and the node may well have acted. Recording that state as "not delivered" invites a reissue as a fresh command, which double-executes. The contract is: the state is unknown, the resolution is a retry with the same ID (§6.3), and the backstop is the node's own expiry and maximum runtime.

A dry_run acknowledgement MUST name the limit that would have been hit, if any. A bare rejection teaches a backtest nothing.

6.5 Limits and shared resources#

Node-side limits are configured on the node, not sent with each command: maximum runtime, minimum off-time, interlocks with other outputs, and the safe state on power loss or link loss. The server cannot override them.

Shared-resource capacity is checked in layer zero, beside per-point authorization. v0 put the resource registry in layer zero but the booking in the layer-1 scheduler, while also declaring layer 1 non-safety-critical and expected to be down sometimes — so "two modules can never overdraw one well" rested on a component designated optional, and per-node interlocks cannot see another node's valve. Capacity is the same kind of check as permission, on a path that already exists, so it belongs here. The layer-1 scheduler still books ahead; layer zero refuses a command that would exceed a declared capacity regardless of who issued it.

Hardwired interlocks for machinery and high-energy loads sit below all of this and are outside software's reach.

6.6 Standards alignment#

Tasking has never been part of SensorThings Part 1, in either version — it lives in a separate document, Part 2: Tasking Core (OGC 17-079r1). So the 1.1-versus-2.0 choice in §10 does not bear on this one. The choice here is Part 2 Tasking Core as it stands, or OGC API — Connected Systems Part 2, whose ControlStreams are published and purpose-built for commands flowing to and from systems. Decided (Q3b): our own contract — this section's safety properties are more than either standard offers — with a Connected Systems adapter, and Connected Systems Part 2 read properly before this section is frozen.

7. Storage, ownership and sharing#

The farm's own machine holds the authoritative copy; the cloud is optional and never required to operate.

7.1 The log is authoritative#

Every change to records, every observation, every command and every decision is an event. Current state is derived from the log, so history can always be replayed and audited.

farmOS is a projection of the log, not the backend. v0 listed farmOS as the records backend and as an adapter at once, which cannot hold: if a mutable Drupal database owns records, then the authoritative copy is not the log, state is not derived from the log, removing an "adapter" destroys the records, and there are two sources of truth with no reconciliation. Instead:

• Writes append to the log, always.
• A projector materializes farmOS one-way, so existing farmOS clients keep working.
• Writes arriving through the farmOS-compatible API are translated into appended events, not applied to Drupal directly.

Every principle survives this; the alternative split is recorded in open-questions.md Q8, where this choice was confirmed, with serving the farmOS API directly and retiring the projector as the end state.

7.2 Two stores#

Derived state is indexed, not recomputed. The promise below is not free. A reference implementation that rebuilt the live set for every quality check, re-sorted the whole event log on every append, and located a row in its series by value equality took a minute to ingest one simulated day of a five-node farm, and got worse with every day — all three quadratic, against the guarantee in the next paragraph. A conformance suite should include a season-scale ingest, because nothing smaller reveals it.

High-frequency sensor data goes to a time-series store with downsampling and retention rules. Records — assets, logs, plans — go to the main store. Years of readings never slow everyday records. Both are projections of the log; neither is authoritative.

No retention rule deletes a raw observation (Q15). Raw is kept at full resolution for as long as the farm keeps its log; downsampled series are derived views that can be dropped and rebuilt. Only the farmer deletes raw, by an explicit act the log records. Where a hub's disk is the limit, old raw moves to farmer-owned cold storage in the archive format of §7.6, compressed and checksummed, rather than being deleted.

7.3 Corrections and erasure#

• A correction is a compensating event. Nothing is ever updated in place. The farmOS-conventions side of the model is CRUD-shaped and will tempt an implementer to mutate a row; the conformance suite tests that they did not.
• Erasure, when it is genuinely required — a worker's personal data, for instance — is performed by destroying the payload's encryption key and retaining the event skeleton, so the log stays continuous and auditable while the content becomes unrecoverable. An append-only log with no erasure story is a liability the first time someone has a legal right to removal.
• The keys (Q17). Personal text is sealed to a key belonging to that one person, with the person bound into the seal. The persons' keys are wrapped by a farm key and kept in a key ring that may be backed up with the log. The farm key MUST NOT be backed up with the data; it lives on the gateway with one offline recovery copy. Erasing a person destroys their key and rotates the farm key: every other key is re-wrapped under a new one, and the old farm key and its recovery copy are destroyed. That is what reaches a backup taken before the erasure: it still holds the person's wrapped key, and nothing that can unwrap it exists. The erasure itself is a log event naming the person's id and not the content.

7.4 Offline edits#

Devices that edit while disconnected sync later. Conflict resolution happens at projection time, deterministically, and never by discarding an event.

Ordering uses hybrid logical clocks, not wall clocks. Last-writer-wins on wall-clock time is unsafe in this system by its own admission: §2 records clock_quality precisely because farm devices' clocks cannot be trusted, and a node reporting clock_quality = none must never win a last-writer contest against a GPS-synced one on the strength of a bad timestamp. Every event carries an HLC stamp; LWW-per-field applies to HLC order.

Where two writers touched the same field concurrently and the field is one where that matters, both versions are retained and a conflict event is emitted for a person to resolve. The conflict event is itself in the log. The fields where it matters (Q14): inputs applied, animal treatments and their withdrawal periods, harvest lots, field and bed boundaries, and anything an interlock or a node-side limit reads. Set-like fields — tags, animal groups — merge by union rather than conflicting.

Concurrency is decided by what each writer had seen, not by the stamps. An HLC stamp puts every event in one order, but two stamps cannot say whether one writer knew of the other. So every edit MUST name the versions of the record its writer had seen (its base). An edit with another in its history follows it, whatever either clock said; two edits neither of which had seen the other are concurrent. A conflict is concurrent versions of a protected field that disagree. A person resolves it by editing the field having seen every candidate — an ordinary edit, so the resolution is history like everything else.

• A void concurrent with any protected edit is a conflict. Field by field the two never meet, and the void would quietly take a withdrawal or a spray off a record while another device was correcting it.
• A removal from a set removes only what its writer had seen added, so an addition concurrent with a removal survives it.
• Checks read every candidate. A treated animal's withdrawal and a harvest inside a pre-harvest interval are judged on the cautious candidate, so a disagreement between two devices can never shorten a withdrawal or hide a breach.

Implemented as sdk/records.py (land, animals, inputs applied, animal treatments, harvest lots) and sdk/edits.py.

7.5 Subscriptions#

"Nothing above layer zero polls" implies delivery guarantees, so they are stated: at-least-once, with a durable cursor each subscriber can replay from. A module that restarts, silently misses the events it was down for, and cannot replay them will make wrong decisions while its decision events claim full provenance. Implemented as sdk/subscribe.py.

• The cursor MUST be an offset into arrival order, not a position in HLC order. §7.4 merges events from other replicas by stamp, so an edit made offline yesterday and synced today lands in the middle of history. A cursor that is an index, the last stamp seen, or the last event id is already past it, and the subscriber never sees it. This is a silent miss that happens only in the offline case — the one this spec was written for and the one least likely to be tested. Arrival order only ever grows at the end, and is already durable because the log file is appended in the order events arrive.
• Act first, then save the cursor. A crash in between costs a redelivered batch; the opposite order costs a lost one. Handlers MUST therefore be idempotent on the event id.
• Delivery is in arrival order. A subscriber that needs causal order sorts by stamp itself. Completeness and causal order are different promises, and only completeness can be made without buffering forever.
• A cursor belongs to one replica, and MUST refuse to run against another. Arrival order differs between replicas, so a borrowed offset delivers the wrong events. Each cursor carries the id of the last event it covered, and a mismatch is an error rather than a guess — resetting to zero redelivers a season, and trusting the offset delivers the wrong things.
• A filtered subscriber's cursor advances over what it skips, or it re-scans every non-matching event forever.
• Wakeups are a convenience; the cursor is the guarantee. A subscriber that is down when it would have been woken misses the wakeup, not the event.

7.6 Export and sharing#

• Full export. Everything exports in open formats (CSV, JSON, GeoJSON for geometry) at any time, with no fee and no support ticket. Raw and calibrated series, quality assertions and their threshold versions are all included; an export that carried only calibrated values would be the silent cleaning this spec exists to prevent. Implemented as sdk/export.py.
• The unit exported is the event, not the record. §7.3's compensating events carry a reason and no record at all, so an export assembled from records drops every correction — the audit trail minus the part that says something was wrong. That is the easier mistake to make, because the record interface is the more convenient one.
• Exactly one file is the archive, and the others MUST say they are not. The archive MUST be byte-compatible with the log's own on-disk form, so that importing it is a replay rather than a second decoding that could disagree with the first. CSV is for a spreadsheet and GeoJSON is a view for GIS clients; a GeoJSON export carries current values and summaries, not history, and MUST declare itself lossy. An export that claims to lose nothing and quietly does is worse than one that declares its losses, because the loss is discovered years later by whoever needed the missing field.
• Losslessness MUST be verified rather than asserted. Each value is decoded back from what was written and compared before the export claims the format kept it. Reasoning about which values are safe is not sufficient: the first implementation reasoned, and was wrong about a check_version of "1", which a CSV cell returns as the integer 1.
• An export MUST be verifiable as a copy. It carries a manifest with per-file checksums and counts, and a verify operation that re-checks them. Without that, the manifest is a claim about files rather than a check on them, and the export is not evidence.
• A feature with no geometry MUST be reported, never silently omitted. A GIS client discards a geometryless feature without a word, which is how a missing bed becomes invisible instead of noticeable.
• Size is part of the promise. A simulated week measures 35.1 MB, or 3.4 MB gzipped — 10.4x — putting a 180-day season near 900 MB against 87 MB. A farmer who cannot move the file has not been given their data, so compression MUST be available; the uncompressed form stays canonical because that is the one the log itself reads.
• Revocable sharing. A grant states what data, to whom, for what purpose and until when. A grant MUST also state whether it covers observations that arrive after the grant was made, and whether the grantee may re-share. Both matter immediately for research and co-op use, and leaving them unstated means two implementations will behave differently on the farmer's most sensitive operation. By default a grant covers only what has arrived when it is made (Q16); one that also covers later arrivals MUST carry an end date, so an open-ended stream of a farm's data is never made by accident, and a standing arrangement renews by a new grant. "Arrived" is received_at: a backlog that reaches the gateway after the grant is new to the farm too. Grants and revocations are log records.
• Revocation stops future access and does not retract what was already delivered. Say this plainly to the farmer at the moment they grant, because "revocable" reads as "I can take it back".

7.7 Decisions and required records#

Automated actions are logged as decision events: the observations used as inputs, the rule or model and its version, the output, and the trust level it ran under. Any action can be traced back to why it happened.

A decision says which question it answers and how to say it, so that anything that carries it to a person — a notifier, a page — needs nothing from the module that made it. Its topic is the question ("is the north trough empty?"; by default its rule and subject), and the latest decision on a topic is the standing answer; valid_until, when given, is when that answer goes stale (tonight's frost watch, at dawn). Its message names a key, its parameters, and the parameter its subject's label fills in the reader's language, the key chosen when the rule is judged rather than when it is read. A module that writes only when an alert is raised or cleared and one that writes a fresh report every morning then read the same way.

The log model must be able to produce certification and compliance records — organic, produce-safety water testing, pesticide application — without custom code. Exact requirements vary by country and program. Note that compliance records depend on quality assertions, which depend on thresholds: anyone who can edit a threshold can change what a compliance report says, which is why threshold edits are privileged and logged (§8.2).

8. Security#

Secure by default, because no farm will harden its network later.

8.1 Devices and links#

• Device onboarding. Each device gets its own key when it is registered. Unregistered devices are heard but not trusted, and their data is quarantined until someone approves them.
• Signed firmware. Updates are signed and the version recorded. LoRa links are too slow for over-the-air updates, so the spec plans for physical update visits on those nodes.
• Encrypted links where the link allows it. LoRaWAN and Meshtastic both offer link encryption; wired and wifi links use TLS to the gateway.
• No default passwords, no open admin ports, and the log records every login and permission change.

8.2 Users, roles and privileged operations#

Roles: owner, manager, worker, viewer, service account. Commands are authorized per role and per point. Three operations are privileged beyond ordinary writes and were unassigned in v0:

Node-side limits are not in this table because no role can change them remotely; they are configured on the node (§6.5).

8.3 Modules#

Modules are permissioned like users. Each declares what it may read, write and command, scoped per point; the farmer grants it at install and the foundation enforces it, which is what makes third-party modules safe to run. Packaging and isolation (Q11): for v0, each module is a process under systemd holding a capability token for exactly what it was granted, and it reaches the foundation over a unix socket that checks the token on every call; WASI's capability sandbox is the stated target, adopted when the Python toolchain is ready.

Built as sdk/modhost.py and deploy/systemd/:

• A manifest is a ceiling; a grant is the farmer's. A module's module.json says what it may ever ask for. A grant names the points, within that, and is a log record like a sharing grant. The log holds the token's hash, never the token.
• No module writes the foundation's records. Observations, quality assertions, thresholds, calibrations, corrections, labels and every kind of grant are refused whatever a manifest declares, and commands are deferred with §6.
• Writes carry the module's name and grant, so every decision is traceable to the module that made it, and a module reads back only what it wrote: modules never reach into each other (§11). The one exception is granted, read-only and named in the grant record: an owner may let a module read every module's decisions (§7.7) and what was done about them — notifications sent, acknowledgements — and the names of what those decisions are about, followed by arrival cursor (§7.5). It is what a notifier and a page need, and it reads a layer-zero record, not another module: neither imports the module whose decision it carries. A decision carries what it was judged on, so the grant says so plainly; readings, records and grants are never readable this way.
• A refusal is logged, once per module, operation and reason, and counted after that, so a module looping on a refused call cannot fill the farm's disk with the record of it.
• The module's view is as of a moment. A module reads a reading's quality as asserted by the time it asks about (State.quality_as_of), the same rule a backtest needs.

9. Driver SDK and conformance tests#

Anyone can add hardware without touching the core, and a passing conformance suite is what makes a driver or implementation official.

Two driver kinds, one plugin interface each:

• Device drivers turn a protocol into raw readings and accept commands: Modbus RTU/TCP, SDI-12, I2C, 1-Wire, 4–20 mA and 0–10 V analog, BLE, local HTTP APIs, CAN/J1939/ISOBUS, NMEA and NTRIP.
• Link drivers move messages: Meshtastic, LoRaWAN via ChirpStack, MQTT, wifi, cellular, wired.

9.1 Driver manifest#

A driver declares, and the foundation handles the rest:

Python is the first SDK language, with Rust for the edge daemon.

9.2 Conformance suite#

• Simulated devices for every driver kind — these are not fixtures written per test but the devices of one canonical deterministic farm, which the suite also uses as a completeness gate: a clause of this spec that nothing on that farm exercises is a clause nothing tests. See VIRTUAL_FARM.md
• Replay of real recorded field data. The usable source is the International Soil Moisture Network: 71 networks, 2,842 stations, data from 1952, harmonised to hourly UTC in m³/m³ — our cm3/cm3 exactly — and, decisively, flagged values are retained rather than deleted, which is the difference between a labelled fault dataset and a list of holes. Its C flags are geophysical-range violations, which is out_of_range; its D flags come from consistency and spectrum checks, the family flatline and rate belong to. SCAN's own API is live and free but deletes suspect values, so it supplies labelled gaps and regional threshold evidence rather than labelled bad values. See notes/validation-data.md
• Public data cannot test the transport contract, and the suite should not pretend otherwise. Every archive has already discarded the layer §5 exists to solve: no frames, no sequence numbers, no ages, no duty cycle. A harmonised hourly series is what survives after those problems were handled by somebody else. The transport half is tested by the simulator and then by a season
• Scored detection, not asserted detection. A health check's claim to catch drift or a stuck probe is measured against ground truth — precision, recall, false-alarm rate on clean rows, detection latency from onset, and whether each fault episode was caught at all. "It flags the obvious case" is not a measurement. The scorer must not be able to see what was injected; a detector scored against the fixture that produced it is scored against itself
• A control drill, every time. A run with nothing wrong must flag nothing. This is not a formality: the first scored run on a clean farm reported a 99.8% false-alarm rate, because the cross-check was comparing a derived series in physical units against a neighbour's raw series in wire units — 0.280 against 280. No amount of fault-detection recall would have revealed that
• Fault injection: drift, stuck values, dead batteries, dropped and repeated messages, clock jumps, node reboots with counter resets, and buffer drains after multi-day outages
• Command safety tests: lost acks, duplicates, link loss mid-command, retry across a node reboot
• Append-only tests: that a correction appended rather than mutated, that recalibration superseded rather than overwrote, that a threshold change did not alter historical assertions

Reliability claims require at least one full growing season of field data, not a few weeks in the lab.

The suite cannot certify anything until the blocking open questions are closed. A conformance test needs a decided wire format, a decided unit system and a decided point-index scheme. "A passing suite is what makes an implementation official" is a claim the suite cannot yet support; open-questions.md marks which questions block.

10. Interoperability adapters#

Ship on 1.1: the 2.0 draft is unpublished, and its serialization differences are confined to one layer if the adapter is written with that in mind.

10.1 Query options, and the rule about refusing#

The SensorThings endpoint MUST support $filter, $orderby, $top, $skip, $count, $select and $expand, with server-driven paging via @iot.nextLink.

An unsupported query option MUST be an error, never silence. A client that sends $filter=result gt 0.4 and receives every reading has been told something false: it cannot distinguish "no filtering happened" from "nothing matched the filter", and a map or a chart built on that shows wrong data with no indication of it. Anything the service cannot evaluate MUST return 400 (malformed) or 501 (valid OData, not implemented), with the offending text identified. $top beyond the server's cap MUST be refused rather than clamped, for the same reason.

The service root MUST carry serverSettings.conformance, and MUST NOT claim a requirements class unless every requirement in it is met (1.1 §9.2.1). A client reads it to decide what it may rely on, so overclaiming is worse than underclaiming; this implementation declares the nine data-model requirements and not request-data (no geospatial $filter), resource-path (no $ref), create-update-delete, batch, data-array, multi-datastream or MQTT. Every link the service emits MUST resolve. A self link or navigation link that answers 404 is worse than an absent one, because a client learns only after following it. A service MUST NOT advertise a navigation link to an entity set it does not serve — and the fix for that is to serve the set, not to drop the link: a Thing's Locations link answers with the Locations derived from its feature's geometry, and its HistoricalLocations link answers with an empty collection, truthfully, because Things here do not move and devices that do are recorded as §4.1 Installations, which 1.1 has no entity for. This clause exists because two such links shipped — see notes/qgis-validation.md — and both were invisible to tests written from the inside.

An entity key that is not a UUID MUST be percent-encoded in a link, and decoded only after the path is split. §3.4 makes an observed property's identity its vocabulary URI, whose slashes are otherwise indistinguishable from path separators.

observationType MUST be derived from the datastream's declaration, not from the type of a result. A declared unit means a measured quantity; that is what a unit is for. Typing a series from a sample's runtime type made a scaled-integer raw series an OM_CountObservation carrying a unit, which is self-contradictory, and left the type unknowable until the first reading arrived. Only an undeclared unit lets the values decide.

Geometry is served, but is not a log record. A feature of interest's geometry reaches the serialization through the same side table as human names (§1), because a farm's field boundaries are the farmer's material and belong in site-private storage, not in an append-only log that may be published. A GIS client silently drops a feature with no geometry, so a service that has geometry available MUST serve it.

Both versions are served side by side once 2.0 is published, from one internal model, at separate base paths. This is the ecosystem's expected pattern rather than an accommodation: FROST-Server, the most widely deployed implementation, ships 1.1 and 2.0 API plugins that can be enabled simultaneously over a single data model. The internal model is deliberately richer than either serialization, so neither version's Observation shape may leak into it — that constraint is what keeps the migration a day's work. Validate both serializations against FROST-Server from the first release, since in practice it is what defines conformance in the field.

11. Layers above#

Everything above layer zero is optional and depends only on the layers below it.

Where each layer stands (2026-10-11):

• Layer 1. modules/rules/ judges CEL conditions and writes each judgement as a §7.7 decision; its frost watch reproduces on a real season the backtest that chose its threshold. Starlark regimens of steps, waits and checks run in regimen.py (Q10). modules/notify/ carries any module's alerts to people by push or SMS, in their language, escalating through a route's tiers until someone acknowledges by reply or button. Not built: a scheduler that acts, resource conflicts (both need §6), and analytics.
• Layer 2. modules/water/, the livestock water monitor (Q12), passes its §15 acceptance test on the virtual farm. modules/soilwater/, the soil-moisture advisory, is backtested on recorded summers, and its watering check, the first regimen, says when a watering never reached the bed. modules/zoneflow/ sets each irrigation zone's runs beside its usual flow and its heads, and says when a head is clogged or broken; it is drilled on simulated seasons. modules/sowing/ says each morning whether a bed's soil at seed depth is warm enough to sow each crop meant for it, from Harrington's germination tables. modules/salinity/ sets a bed's salt, from its probe's bulk conductivity, beside its crop's tolerance. Not built: irrigation control, which needs §6, and the modules the farm's records were built for (Q8, Q27).
• Layer 3. modules/advise/ answers what grows where and when. modules/twin/ runs GreenLight as an unheated tunnel (Q28): checked against the virtual farm's tunnel, as a covered-crop frost watch each evening, and against a tunnel's own sensor. Not built: cost per field or crop.
• Layer 4. modules/board/, the thinnest slice: one read-only page per language of the alerts that stand, who was told and who is on it, the latest readings, and a plain warning when it has stopped updating. Not built: a field app, voice, an assistant.
• Layer 5. Nothing yet.

CEL and Starlark both refuse I/O, which is what makes a decision event's provenance claim true. Conditions are expressions ("soil moisture below 18% for 6 hours"), cheap to evaluate, store and display; regimens are sequences that need control flow.

Rules for building upward:

• Dependencies point down only. A module never reaches into another module, only into the layers below, so any module can be removed.
• Trust ramp per module. Every automation runs as advisory, approval or autonomous. Farms move up one step at a time, and the level is recorded on every decision event.
• Nothing above layer zero is safety-critical. If layers 1 to 4 go down, nodes keep running their local schedules and limits, and layer zero still refuses commands that exceed a declared capacity or a per-point permission.
• Shared limits are enforced, not hoped for — in layer zero (§6.5), not only in the layer-1 scheduler.
• Cost visibility early. Showing cost and return per field answers the barrier farmers name most often: unclear value.

12. Versioning and governance#

The spec is published separately from the code, so others can build on it or write their own implementation.

• Semantic versions. Breaking changes only in major versions, each with a written migration. The message format carries its version in every packet.
• Spec, reference implementation, conformance suite are three versioned artifacts released together.
• Change process. Public proposals with a comment period before a breaking change lands.
• Liability, sustainability, legal entity. A fiscal host now and an own entity later; no commands in the first release, hardwired interlocks below everything, insurance once an entity can buy it (open-questions.md Q9). The license is decided — spec CC-BY-4.0, SDK and conformance suite Apache-2.0, reference implementation AGPL-3.0-or-later; see LICENSING.md.
• Partners. University extension services already have farmer trust and run research plots, which makes them natural pilot partners.

13. Open questions#

All questions, with every option and a recommendation, are in open-questions.md. Those that block the first release, because a conformance test cannot be written around them:

14. Changes from v0#

15. The three-apps acceptance test#

Each of these required a layer-zero change under v0, so v0 did not yet meet its own done criterion. Re-run this test against any future revision.

The livestock water monitor passes (modules/water/, 2026-10-10), after its first run found that the clause it rests on did not hold in practice: resends went ahead of fresh readings, and per-frame confirmations were read as cumulative (§17). The irrigation controller waits on §6; the research logger is not built.

The test runs against the simulated farm, where each app has a feature built to host it — beds with drip for the controller, a paddock with troughs for the monitor, a plot with duplicated probes for the logger. Passing it there proves the contract is sufficient; it does not prove the thresholds are right, which is what the first season is for.

16. Principle enforcement map#

17. Amendments after v0.1#

Recorded here rather than folded in silently, because §12 makes the spec a versioned artifact and a change with no stated reason is the thing this project exists not to do. None of these is breaking: each adds an optional field or settles an ambiguity in the only direction that was coherent.