Add revision numbers: snapshot r field, kanta.rev, CLI rN selectors, k1 meta migration
Revisions are absolute state positions between diffs: the empty initial state is revision 0 and each change record advances it by one. Snapshots persist the revision of the state they contain in a new r field, stamped with a meta format version k (kanta.structs.META_FORMAT, currently 1); change records carry no revision of their own. - kanta.rev exposes the current revision (raises before open) - replay restores the revision, trusting r on k-marked snapshots and counting change records from the start of the file for legacy (k0) ones - rotation preserves absolute revisions, counted from record positions - meta migration k1 rewrites a legacy file in full on writable open: leading snapshot preserved and re-stamped, all change records kept, closing snapshot with the total revision; read-only opens compute the same revision without touching the file - CLI range selectors accept rN bounds: rN is the record that produced state N with its state (the change, or the snapshot materializing it); single indexes must exist, ranges truncate Python-style - CLI snapshot indicator lines show the snapshot revision
This commit is contained in:
1 parent
ec8695ba0c
commit
de2476c59c
14 files changed
+793
-28
No files matched your search
+29
-4
@@ -51,7 +51,7 @@ Fields:
|
||||
Snapshot lines are prefixed with `SNAPSHOT `, followed by JSON:
|
||||
|
||||
```text
|
||||
SNAPSHOT {"ts":"2026-06-10T00:00:00Z","v":5,"state":{"users":{}},"m":"2026-06-10T00:00:00Z"}
|
||||
SNAPSHOT {"ts":"2026-06-10T00:00:00Z","v":5,"state":{"users":{}},"m":"2026-06-10T00:00:00Z","r":12,"k":"0.9.2"}
|
||||
```
|
||||
|
||||
Fields:
|
||||
@@ -59,13 +59,38 @@ Fields:
|
||||
- `v`: schema version represented by the snapshot.
|
||||
- `state`: full state dictionary.
|
||||
- `m`: optional domain modification timestamp.
|
||||
- `r`: revision of the snapshotted state — the total number of change records that occurred before this snapshot.
|
||||
- `k`: meta format version (integer), the module-level constant `kanta.structs.META_FORMAT` at write time. It is bumped manually when the meta format changes. `0` (a missing field in old files) marks a legacy snapshot whose `r` cannot be trusted; readers then count change records from the start of the file instead. k1 introduced revision numbers.
|
||||
|
||||
## Revision Numbers
|
||||
|
||||
A revision number identifies a *state position between diffs*, not a diff:
|
||||
|
||||
- The empty initial state is revision 0.
|
||||
- Each change record — including the bootstrap record — advances the position by one, so after the first record the state is at revision 1.
|
||||
- Change records carry no revision of their own; the revision of a state is counted from the preceding snapshot (`snapshot.r + n` records after it), or from the start of the file.
|
||||
- Revisions are absolute and survive rotation: the leading snapshot of a rotated file carries the revision of its cutoff state, so `r` values keep referring to the same state positions across history rewrites.
|
||||
|
||||
`kanta.rev` exposes the current revision at runtime: the number of change records applied since revision 0, advanced in memory as transactions queue records. Reading it before `open()` raises `DataIntegrityError`. The CLI accepts `rN` range bounds to select records by revision (e.g. `-r r10:r20`): a bound `rN` addresses state N together with the record that produced it — the change record resulting in state N, or, when that change is no longer in the file, the snapshot materializing state N. `-r r5` and `-r r5:r6` therefore show the same record and output state 5, while `-r r5..r6` also includes the next change and outputs state 6. Ranges truncate Python-style instead of erroring: on a rotated file whose leading snapshot holds revision 3, `-r r1:r4` starts at the beginning of the file and shows that snapshot. Snapshot indicator lines show the snapshot's revision.
|
||||
|
||||
## Meta Format Migrations
|
||||
|
||||
The snapshot `k` field carries an integer meta format version (`kanta.structs.META_FORMAT`, currently 1), bumped manually whenever the record-level format around the payloads changes. k1 introduced revision numbers.
|
||||
|
||||
When a writable `open()` finds a legacy (k0) last snapshot, the file is rewritten in full under the lock, before replay results are used further (meta migration k1, `kanta/metamigr.py`):
|
||||
|
||||
1. A leading snapshot, when the file begins with one, is preserved with its data and timestamps, re-stamped k1 with `r = 0` (the only count knowable from the file).
|
||||
2. All change records follow in order, re-framed; redundant intermediate snapshots are dropped.
|
||||
3. A closing k1 snapshot captures the final state with `r` equal to the total number of change records.
|
||||
|
||||
After the rewrite every snapshot in the file is k1 with a trustworthy `r`, so readers never deal with mixed r/k meanings in one file. Rotation composes with this: it runs first and counts revisions from record positions regardless of `k`, so rotating a legacy file already produces k1 snapshots and no separate migration is needed. A read-only open computes the same revision the migration would have produced but leaves the file untouched.
|
||||
|
||||
## Replay Model
|
||||
|
||||
1. Find the last snapshot in the file, if present.
|
||||
2. Initialize replay state from snapshot state (or `{}` if none).
|
||||
3. Replay subsequent change records in order using patch application.
|
||||
4. The final replay state becomes in-memory `kanta.data`.
|
||||
2. Initialize replay state from snapshot state (or `{}` if none) and the base revision from the snapshot's `r` (or a count of the preceding change records when the snapshot is a legacy one without `k`).
|
||||
3. Replay subsequent change records in order using patch application, counting them toward the revision.
|
||||
4. The final replay state becomes in-memory `kanta.data`, and the final count becomes `kanta.rev`.
|
||||
|
||||
This model provides fast startup for large logs while retaining append-only history.
|
||||
|
||||
|
||||
Reference in new issue
Block a user