History and replay

This page explains the two ways the gateway gives you the past: history reads, which draw a trend of one tag over a range, and replay, which plays every tag, the clock and the alarms back to one /sync socket at the recorded pace. It is for anyone who trends tags or looks into an incident, and for developers using the SDK. The routes are in the history API and the /sync protocol; a worked example is Replay an incident.

Where history comes from

Every accepted value passes through the event backbone (NATS JetStream) before anyone sees it. Two stores keep those values:

Store	Keeps	Default retention
The event backbone's value stream	Every accepted value of every tag, as the runtime's own record	EventBackbone__ValueRetention, 7 days
The historian module, io.scadex.historian	Every accepted value of every tag, in a SQLite file of its own	90 days (retentionDays, changed in the module's settings)

The historian is an ordinary consumer of the value stream (IModuleValueConsumer). It stores each batch together with its stream position in one transaction, so a restart resumes without loss or duplicates, and downtime is caught up for as long as the backbone still retains what it missed. Its store opens in the background, so a slow disk does not hold up the gateway: until it is open, its routes answer 503 and history reads come from the backbone. The data file is <data>/modules/io.scadex.historian/history.db. A renamed tag keeps its history, and a deleted tag's history stays readable by its last path until retention removes it. See Historian.

Per-tag history configuration can be set in the dashboard and is stored, but nothing acts on it yet: the historian records every tag.

The historian module's own API

The module answers under /api/v1/modules/io.scadex.historian while it runs. These routes follow the module rule: GET needs viewer, PUT needs engineer.

Request	Purpose
GET /status	Position in the stream, lag, oldest and newest sample, size on disk, gaps
GET /points?path&start&end&limit&cursor	Raw samples, oldest first. truncated and nextCursor page through them
GET /buckets?path&start&end&buckets	Count, min, max, average, first and last of Good numeric values per time bucket
GET, PUT /settings {retentionDays}	How long to keep samples. Shortening it prunes at once; otherwise pruning runs hourly

start and end are Unix milliseconds. Until the store is open these routes answer 503.

How a read chooses its source

A history read (tag.replay on /sync, which the dashboard's trends use) answers from a running historian module if there is one, otherwise from what the backbone retains. If the module fails or takes longer than 15 seconds, the backbone answers instead. The reply says who answered and what it covers, so a client can be honest about the rest:

Field	Meaning
source	The module's id (io.scadex.historian), or backbone
retainedFrom	The oldest time the source has. A range starting earlier has no data before this
truncated	The read stopped short of the range's end, at coveredUntil. The backbone reads at most 10,000 stored batches from the start of a range
coveredUntil	The newest time the source read or holds
catchingUp	The module is behind the live stream, so recent values may be missing
gaps	{from, to} periods the module never received, because they expired from the backbone before it read them
fallbackReason	Why an installed module did not answer
samples	Stored samples in the range, before any reduction to maxPoints

The request is tag.replay {path, start, end, maxPoints} and the reply tag.replay {path, start, end, points, preBoundary, samples, source, ...}, or tag.replay.error {path, start, end, error}. Times are Unix milliseconds. The gateway returns at most 10,000 points for one request, reducing a denser range to maxPoints.

{ "event": "tag.replay", "data": { "path": "Plant.Line1.Flow", "start": 1762552800000, "end": 1762556400000, "maxPoints": 1000 } }

Trends in the dashboard

The dashboard turns every one of those fields into a sentence under the chart, in the words of the source: "Historian holds nothing before ...", "Event backbone is still catching up and has reached ...", "Historian never received values between ... and ...: they expired from the event backbone before it read them." Below the chart it says how many points were drawn from how many stored values, and from which source.

Where trends appear:

• Historian (Data, Historian): a list of trendable tags on the left (folders, types, instances, @gateway and @types are left out) and the trend of the chosen tag on the right. The header says whether the historian module is Recording or Stopped, from when it holds data and for how many days it keeps it; without the module it says trends only go back as far as the gateway keeps recent values.
• A tag's panel in Tags: the same trend pane for the tag you have open.
• Watchlist: the tags you keep an eye on, each with its live value and trend over 1 h, 6 h, 24 h or 7 days. The list is per person, holds up to 24 tags, and its first eight also show on the overview. Add a tag by path on the page, or press Watch in a tag's panel.
• Screens: the Trend widget draws a tag's history on a screen. See Widgets.

The trend pane offers Time range presets of 15 min, 1 h, 6 h, 24 h, 7 d and 30 d, or Custom with a start and end. It follows live values until you pause it, can be refreshed, and exports the drawn points as CSV. Ctrl+scroll zooms and dragging pans. Under the chart it shows Lowest, Highest, Average (marked approximate when the data was downsampled) and Change; for a boolean tag these read as On, Off and a percentage of time on.

Replay

A replay plays the recorded past back to one /sync socket: the tag values, the gateway clock and the alarms. It is private to that socket. Its frames are replay.value and replay.alarm, never tag.value or alarm.*, so live tag state, live alarms, other clients and the historian are not touched. It needs the viewer role.

In the dashboard

Replay is a mode of the whole browser. Start it from the clock in the top bar, or from Replay in the navigation (the same controls on a page, for a phone where the clock is not shown):

1. Choose Replay instead of Live.
2. Set Start with the date-time field, or press one of 5 min, 1 hour, 8 hours or 1 day under Ago: to start that long ago.
3. Set End if you want one. Left empty, the replay plays on until it has caught up with now.
4. Press Start replay. From the Replay page this opens the overview, which then shows the plant as it was.

While a replay is on, everything that shows a tag, a screen, a trend or an alarm shows the plant as it was at the replayed moment and plays on at the recorded pace. The clock in the top bar shows the replayed time. The controls offer Pause and Resume, Jump to another time with Go, a progress bar, and Back to live. Going back to live restores the live values at once: each subscriber is handed its tag's live value and the alarm list is read afresh.

Nothing can be written during a replay. A write, a screen script or an alarm action is refused and the dashboard shows a message saying so; see Writes during a replay.

The TypeScript SDK does the routing, so no page needs to know (startReplay and stopReplay on ScadexClient, useReplay in React): replay.value frames reach the same subscribers as tag.value, and live tag.value and alarm.* frames are held back from them for the duration. useReplay() returns null when the browser is live, or a ReplayModeState:

Field	Meaning
id	The replay's id on the gateway
start, end	Where the person asked it to begin and, if they did, to end (Unix milliseconds; end is null otherwise)
phase	starting, playing, paused, finished or error
position	The recorded time reached when the gateway last said so
reportedAt	Date.now() when position was reported, so a playing replay can be carried on from it between status frames
sent	How many frames the gateway has sent
message, warning	The gateway's message and warning, or null

See TypeScript SDK.

Checkpoints

The value stream only holds what changed, so without help a tag that stood still for a day would have nothing to replay from. Every hour on the hour the gateway publishes each tag's last value to its own JetStream stream, SCADEX_CHECKPOINTS_V1, kept for the value retention plus two hours. EventBackbone__CheckpointInterval sets the spacing (default one hour, a whole fraction of an hour).

What the gateway does

replay.start {id?, start, end?, paths?} takes start in Unix milliseconds, an optional end (without it the replay runs until it has caught up with the present) and optional paths: exact tag paths, or a prefix ending in *. Without paths every tag the session may read is replayed, which is what the dashboard asks for, so any subscription is served. The gateway then:

1. Finds the newest checkpoint at or before start, looking back up to six hours. With none, it folds in the last hour of values.
2. Folds the values recorded between that checkpoint and start into it, giving what every tag held at start, and sends that as replay.value frames with initial: true.
3. Sends the clock at start the same way, and the alarms outstanding at start as replay.alarm with kind: "snapshot".
4. Says so with a replay.status whose opened is true. The opening state is complete, so a client can show it all at once.
5. Reads the value stream on from start with one NATS consumer and sends each value as it comes round at its recorded pace, with the clock once a second and each alarm event at its own recorded time between the values.

A value timestamped after start is never part of the opening state.

Protocol summary

Client sends	Gateway replies
replay.start {id?, start, end?, paths?}	replay.status, then replay.value {id, path, value, quality, timestamp} and replay.alarm {id, kind, ...} frames
replay.pause {}	replay.status
replay.resume {}	replay.status
replay.stop {}	replay.status

replay.status {id, state, start, end, position, sent, opened, message?, warning?} is sent on every change and about four times a second while playing.

state	Meaning
playing	Values are being sent at the recorded pace
paused	Held where it is; values and alarm events wait
finished	The replay reached its end, or the end of the recording as it stood when it got there
stopped	Ended by replay.stop or by a new replay.start
error	It could not run or could not continue; message says why. A refused request is an error with its reason

position is the recorded time reached, which runs on smoothly with the clock rather than jumping from value to value. warning is something that did not stop the replay but that a person should know, such as alarms that could not be read.

Pacing and limits:

• Replays run at the recorded pace. There is no speed control.
• Gaps of more than 5 seconds in the recording are shortened to 5.
• A socket plays one replay at a time; starting a second ends the first. The gateway runs at most 8 at once.
• A replay with no end plays on to the end of the recording as it stands when it gets there (the present at that moment, which a replay at recorded pace can never overtake) and is then finished, with the clock on the moment it reached. With an end that is as far as it goes, and never past the present.
• Closing the socket ends the replay.

The full frame shapes are in the /sync protocol.

Writes during a replay

A socket that has started a replay is looking at the past, so the gateway refuses what would change the plant from it: tag.write (answered with tag.write.error) and screen.event, whose scripts can write, both with "Replay is on. Return to live to change values." Values private to the user or session, and @gateway.browser.*, are not the plant's and stay writable. The socket is in replay from replay.start until replay.stop, finished included, because what it shows is still the past.

The SDK refuses the same things before they are sent, and also alarm acknowledgements and shelving, which go over REST and so cannot be told apart from live ones by the gateway. It reports each refusal to a listener (onReplayBlocked), which the dashboard shows as a message.

Alarms in a replay

Alarms are replayed too, because a screen shows them: banners, alarm-colored symbols, counts and the alarm list. They are rebuilt from the alarm history (TagAlarmEvent) without touching the live alarm state:

• The alarms outstanding at start are the snapshot of replay.alarm {id, kind, alarms}, listed as a live alarm.snapshot is, most severe first. For each alarm, the state at that moment follows from its newest activation before it and the acknowledgement or clearing recorded after. An alarm that was removed or disabled while outstanding was cleared with a reason that says so, and is normal from then on. The tag's value at start replaces the one the alarm last changed state on.
• Each event after start is sent as replay.alarm {id, kind, alarm, at}, where kind is the history event (active, acknowledged, cleared, shelved, unshelved), alarm is the alarm in the state it left it, and at is when it was recorded. They are paced with the values by the same clock, so a pause holds them too.
• The alarm's name, severity, condition and setpoint are those it had at the time, copied into each history row. A shelf keeps the end it had (shelvedUntil on the shelved event).

What a rebuilt alarm cannot know, because history does not hold it: the alarm's deadband and on-delay (shown as 0), whether it was stale, and its value between transitions. History is kept for Alarms__EventRetention, a year by default; asking further back shows no alarms. If the alarm history cannot be read, the replay carries on with the values and says so in warning. See Alarms.

The clock tags

@gateway.time is the gateway's clock and @gateway.tz the zone it keeps it in (an IANA id, or UTC). The value of @gateway.time is an ISO-8601 string in that zone to the whole second, with the zone's UTC offset at that instant, for example 2026-11-03T14:02:11+01:00. It changes once a second and a subscriber gets it like any tag. The dashboard's clock is this tag.

The clock is not stored: recording it would put 86,400 values a day into the value stream, the checkpoints and the historian to say what a replay can work out for itself. A replay produces it from its own position, which makes it the one @gateway tag that replays. See Tags for the other @gateway tags.

Limits and gaps

• No speed control: a replay runs at the recorded pace only, with long gaps cut to 5 seconds.
• Tags that belong to one user or session (scoped memory tags, marked private on their values) are never replayed.
• @gateway tags other than the clock have no past and keep their live values in a replay.
• A replay reaches back only as far as the value stream's retention (7 days by default). It does not read the historian module, so a longer historian retention does not extend replay; it extends trends.
• Alarms in a replay are only as complete as the alarm history, with the unknowns listed above.
• At most 8 replays run at once across the gateway, one per socket.