Move the PoW state from a module-level dict keyed by WebSocket in
messages.py to a plain per-connection local in tracking.activity_ws,
making PoW a WS-level concern instead of being buried in the message
handler:
- activity_ws validates and burns the solution (raw.pop('pow')) before
forwarding a message frame, and issues a fresh challenge after every
registration and every vetted message frame; a failed validation is
answered with a 'required' challenge and the frame never reaches
messages.py.
- messages.py loses the _pow dict, _issue_pow and the disconnect
cleanup for it; visitor_frame needs no return value since a vetted
frame always triggers a fresh challenge (a too-long message simply
comes with contact-error and required=false, so the client does not
retry it).
The wire protocol and the frontend are unchanged.
The bridge routes {type: pow} frames to the panel. contact-main solves
each fresh challenge in the background with the backend-served easypow
solver module (imported by URL from a variable so Rollup leaves it
dynamic), aborting stale solves on replacement challenges. Sending
awaits the current solution and burns it with the message; a
'required' rejection re-solves and retries the message once.
Composer and ContactPanel need no changes: uploads stay ungated, and
the send path just awaits the in-flight solve.
New dependency: easypow (PBKDF2-SHA512 challenges, bundled JS solver).
Sends are PoW-gated: registration/resync and every accepted message
come with a fresh single-use challenge (~5 s solve), held in per-socket
RAM state only — nothing in the database. A message burns one
solution; a missing, stale or spent solution is dropped and answered
with a fresh 'required' challenge. The bundled browser solver is
served at /_contact/easypow.js straight from the installed package, so
client and validator versions always match.
Uploads are deliberately not PoW-gated (the transfer rate is the real
cost). Instead, uploads are tracked as pending per session and a quota
caps pending files at 10 — answered 429, not 403, which could trip
auth flows. A sent message strikes the files it references off the
pending set; when the session's last socket closes, the remaining
pending files are deleted from the store.
Rework attachment storage and serving per the refined separation
scheme:
- Non-media uploads are stored as {hash}.file blobs (store_attachment)
— the original extension never lands on the filesystem, so the store
directory holds nothing double-click-executable. The original name
lives only in the URL: /_f/{hash}/{original-name}. The .file suffix
itself is rejected in URLs.
- Serving is strictly separated: /_f/{name} serves media only (hash
with or without extension, .orig. still blocked) and 404s .file
names; /_f/{name}/{filename} serves only {name}.file blobs, never
media, never by hash alone.
- Every /_f/ response now carries bare 'content-disposition:
attachment' (the URL's last segment already names the download) and
x-content-type-options: nosniff — media included. Subresource
fetches (<img>, <video>, favicons, the zoom viewer) ignore the
disposition, so inline media is unaffected, while a direct
navigation can never render or sniff a store file into something
executable in the site's origin. The mime-based script-carrier
forcing and the header filename are gone (subsumed).
- Message uploads treat SVG as media (rasterized derivatives; an
as-is-served .svg is script-inert inside <img>), and undecodable
fake images fall through to the .file attachment path.
Drop the script-carrier blocklist: any file type (html, exe, svg, ...)
may be attached now, with safety from enforced downloads instead of
type filtering, verified twice:
- the attachment chip anchor carries a download attribute (already),
- the link is /_f/{name}/{original-filename}, a new file-store route
that answers content-disposition: attachment naming the original
file (ASCII-safe fallback + RFC 5987 UTF-8 form). The filename rides
in the URL itself — no filename registry in the database.
Close the residual gap on the plain /_f/{name} route: a stored file
whose mime executes script in origin (html/xml/svg/js) is forced to
download there too, so stripping the filename segment off an
attachment URL cannot turn it into an in-origin page. All /_f/
responses now carry x-content-type-options: nosniff. Subresource
fetches (<img>, favicons) ignore Content-Disposition, so inline
images and icons are unaffected.
Undecodable fake images (store_image falls back to as-is storage) now
attach as files instead of being marked inline images.
The serializer treated only DIV/P as blocks: a formatting element
directly under the editable root (a bolded segment of an undivided
line is a top-level <b>) became a separate PLAIN block, losing both
the formatting and the paragraph structure. Top-level inline runs now
accumulate into one paragraph, and style-based formatting (Firefox's
font-weight/font-style spans) maps back to markers.
Attachments: the upload allowlist rejected everything but a handful of
types with a console-only 415. The logic inverts — raster images get
derivatives, script carriers (svg/html/js/xml...) are refused, and any
other file stores as a download chip. Upload failures now show in the
composer, and toolbar buttons keep the contenteditable selection on
click (mousedown.prevent).
docs/messages.md covers the trigger, both socket protocols, the storage
model, the XSS stance (grammar restriction via a strict renderer, never
HTML sanitization), uploads, and the MVP gaps (PoW seams, moderation,
orphan GC, i18n). AGENTS.md and frontend-runtime.md learn the new
module, bundles and asset keys.
The analytics viewer gains a messages tab over the same socket: frames
with a type field are inbox payloads (thread list on connect, live
thread upserts), the type-less snapshot stays analytics. MessagesView
shows the thread list with unread dots and the selected thread with the
shared composer for replies; opening a thread marks it read server-side.
The header badge counts unread threads.
The composer is a constrained contenteditable (plain-text paste,
toolbar formatting, upload chips for images/attachments) serialized to
a Markdown subset by richtext.js — the wire format is text and the
server re-renders it, so no user markup ever reaches a page. Uploads go
to /_contact/files with the session token (admins share the component
via /_api/files). ContactPanel shows the thread pushed live over the
activity socket: pagerite.js lazy-loads the bundle on trigger clicks,
routes socket pushes to it, re-registers the session token on every
reconnect (the resync), and dots trigger buttons on unread replies.
A lone {contact: Label} line in article Markdown renders a
.contact-trigger button that opens the visitor panel (pagerite.js binds
the click and lazy-loads the contact bundle; inert in the editor
preview, whose resolver now matches the served page). The on-demand
bundle's URLs ride in the pagerite-assets config like langselect's, and
the build gets a contact entry point. Description/card mining renders
resolve the tag to nothing.
PUT /_contact/files/{name} stores panel uploads in the content-addressed
file store: raster images through the usual derivative pipeline, a small
allowlist of document types as-is. SVG/HTML are excluded (script
carriers). Gates for now: a session-shaped token plus size/type caps;
the proof-of-work stamp plugs in here later. The dev proxy learns the
/_contact path.
/_ws routes frames carrying a session token to pagerite.messages (and
now also pushes thread state back down the same socket); the visitor
registry is cleaned on disconnect. /_api/ws/analytics becomes
bidirectional: admins get the contact inbox on connect, visitor messages
are pushed live, and reply/read frames are handled. Analytics payloads
keep their type-less shape; messaging frames always carry a type field.
Data.contact holds visitor threads keyed by blake3(session token).
Message bodies are a Markdown subset rendered with a strict MarkdownIt
(html=False): the stored html is always markdown-it output, never
sanitized user markup. pagerite/messages.py owns the store ops, wire
payloads and both socket protocols (visitor frames on /_ws, admin frames
on /_api/ws/analytics); the socket wiring follows.
Beads now take their connector's color — accent for internal links,
plain text color for external ones (matching the tconn-exit styling) —
and render as an elongated pulse along the direction of travel instead
of a plain dot: a rounded capsule with a gradient-faded tail, rotated
to the edge's angle (cached per flow). Flow objects carry an 'external'
tag from buildFlows; the gradients live in the svg defs with stop
colors resolved from CSS so theme variables apply.
The inactivity cutoff timer cannot fire while the computer is asleep, and
performance.now() keeps running through sleep on some platforms, so after
a wake the whole slept span could be credited as reading time (inflating
min/visit stats). A gap longer than the inactivity timeout now closes the
span at its intended cutoff, and the cutoff timer clamps its accumulation
to its scheduled fire time — covering both event-first and timer-first
wake orderings.
An expired session failed the /_api/ws/editor handshake exactly like a dead
network, so the page/banner editors retried forever — the WS API cannot
open the login dialog. reconnect.js gains authReconnect(): on close, probe
/_api/settings (dialog-less fetchJson); a 401/403 runs the paskia login
flow and reconnects the moment it completes, a declined login holds at a
'session expired' ConnNote, and only transient failures take the paced
backoff. A paskia auth-success message from anywhere in the page
reconnects a waiting editor at once. AnalyticsView moves from its inline
copy of the same flow onto the shared helper.
/pages.md (front page /.md) serves the page's source Markdown — per-language
hybrid assembly, title injected as '# title' when the content has no h1 —
with the same ETag/Last-Modified/no-cache scheme as the HTML, which now
carries vary: accept. Pages advertise the counterpart via
<link rel=alternate type=text/markdown> and llms.txt links point at the
.md URLs.
- Expired session dropped the socket into an endless paced retry loop the
WS handshake can never escape: probe /_api/settings on close, run the
paskia login flow on 401/403 and reconnect the moment it resolves; an
aborted login leaves for the front page (history replace).
- Follow paskia's auth-success/auth-logout window messages: logins anywhere
in the page reconnect at once instead of waiting out the backoff; logout
leaves /_a. pagerite.js re-probes its pens on the same events.
- Recycle the socket on visibilitychange/online/bfcache restore: a socket
that survived sleep looks OPEN but is dead, so no updates ever landed.
- The stats range window was frozen at mount (no reactive time dependency);
it now slides on a minute clock.
- Transition map beads: fast-forward emitter schedules after long gaps
instead of spawning a bead per missed interval in one tick.
Node inclusion no longer depends on views in the window: every page
from /_api/pages is shown (the front page used to vanish when quiet).
Omission follows the navigation instead — index pages without content
of their own (category labels and empty pages, the nodes nav links
bypass to the first child) are omitted, children promoted, first
visible child taking the slot. Deleted pages named by traffic are
inserted as ghosts where they would have been, materializing missing
ancestors as content-less category lanes.
/_api/pages gains the "empty" flag (not node.chunks) for the nav
notion; has_content (chunks is not None) stays for the structure
editor's delete affordance.
Rendering every article per request is expensive. Bodies are now
RAM-cached keyed by the public base URL and cleared from
_invalidate_pages (the same hook that drops the page render cache on
any content/settings write). Responses carry a blake3 content-hash
ETag and answer 304, so polling feed readers revalidate cheaply;
matching page behavior, 304 revalidations are not recorded in
analytics.
New pagerite/feeds.py: /llms.txt (Markdown site map with excerpts for
LLM agents), /feed.json (JSON Feed 1.1) and /feed.xml (RSS 2.0 +
atom:link), all carrying every published article with full content and
absolutized URLs. Linked from every page <head> (feed alternates +
llms-txt) and from the sitemap, recorded in analytics like page GETs
and emoji-marked in the trails (🧠 llms.txt, 📡 feeds).
The robots and sitemap routes never called _record_get, so those
accesses were invisible. Recorded like page GETs, they surface as
crawler hits at display time (no activity message ever follows). In
the trail rendering, known non-article machinery paths (robots,
sitemap, future feed paths) get an emoji marker instead of a bare
slug.
The old 1.4-2.5 aspect window sent big near-square images (e.g.
1350x1200) to the compact layout despite ample resolution. The new
rule keeps square and portrait images compact (the compact box fits
them) while letting anything big and clearly wider than tall render
as a full-card cover.
Replace count-sorted, group-centered rows with a spring-like placement:
pills order by their weighted median target x and settle by clamped
coordinate descent, minimizing weighted horizontal connection distance
while keeping the minimum pill spacing.
The banner panel's card-image block gains a pasteboard button (an image
uploads; an image URL goes as the save message's image setting itself and
is fetched/stored server-side, cross-origin being CORS-blocked for the
browser) and a site-icon button saving the @favicon sentinel, which
resolves to Data.favicon at render time and follows favicon changes.
The resolved (inheriting) Node.image was checked before mining the
article, so a front-page banner image shadowed the articles' own
extracted figures everywhere: card covers, og:image and the editor's
card previews.
Render the seasonal typical estimate as a translucent muted fill (no
stroke, same 0.6 opacity as the data) instead of a grey line, on both day
and week views. Continue the fixed Monday-Monday week view with last
week's curve from the current bucket to the end of the week (secondary
accent, translucent fill), so the chart is useful early in the week and
last week is gradually replaced by the current one.
Gate the typical estimate on history spanning twice the view's full time
(third day / third week on) and the last-week tail on data reaching into
last week at all; the legend shows each entry only when present.
Replace the old "patches" format (path:lang composite keys, ordered
hunk lists, text-anchored matching) with Data.overrides:
path -> lang -> LangEdits, keyed throughout so a save's database diff
touches only the edited chunks. The old "patches" key is ignored on
decode, discarding legacy data without a migration.
- Whole-paragraph additions/deletions are structural: a drop flag on
the original chunk hash, and additions in their own dict anchored
from the neighboring chunks' before/after (first live referrer wins),
so they stay in place across retranslation and one-sided original
edits.
- Within-paragraph edits (up to a full paragraph rewrite or split) are
full-chunk replace patches applied by chunk hash alone: a
retranslation is overridden wholesale, so user edits survive AI
re-runs; editing the original changes the hash and orphans the patch.
The old search-matching staleness gate is gone.
- Saving a translation on a page without original chunks is rejected
(REST 400 / WS error); emptying the original afterwards renders the
translation empty, with the orphaned overrides inert.
- Added full article / with markdown mode for advanced LLMs, with local ollama and API support, new llm_translator script.
- Translate navigational titles as a whole (preserving site structure), and then retranslate as article headings with article context - potentially changing nav label
- Improved human overrides to translations, in particular when structure is affected
Reviewed-on: https://git.zi.fi/LeoVasanko/pagerite/pulls/2
Extra paragraphs exist only inside a patch hunk's replace text (anchored
on the preceding original block). Deleting one produced a (X, "") hunk,
and apply_patch removed the text but not a surrounding separator, leaving
an accumulating blank line in the served/editor Markdown. hybrid_markdown
now re-chunks the patched text (join_chunks(chunk_markdown(...))), which
drops the stray blank lines on the read path — repairing gaps left by
already-stored patches too.
Adjacent fix in make_patch: a hunk whose search text occurred more than
once in the page hit the FIRST occurrence at apply time, blanking or
rewriting the wrong instance when identical paragraph text appeared
earlier. Ambiguous hunks now grow block context (preceding block first)
until unique; unique hunks stay context-free so they keep surviving
translation refreshes.
load() closed the editor on every navigation, and the close raced the
navigation: the shell's close-time loadPlain re-render resolved last,
swapping the old page back in and replaceStating its URL. Navigation now
leaves the panel open and the existing pagerite:switch-editor retarget
fires while editing; the exception is /_a, where closeEditor({ navigating:
true }) closes the panel without the re-swap or title restore.
- Banner-corner auth link is now a button opening paskia-js's profile()
dialog (handles login too); auth re-probed when the dialog closes.
- Editor/analytics components call /_api via apiFetch/apiJson: an
expired session opens the login dialog and the request retries.
- pagerite.js: task-checkbox toggle uses apiJson (explicit edit attempt,
reverts on any failure incl. cancelled login); auth probes use
fetchJson (never a dialog); page-cache/navigation stay on plain fetch.
- Add the paskia npm dependency; document the convention.
Preview: the editor render handler now returns the complete article HTML
({cards} tags expanded, children's cards appended when the page has no
tag) and the frontend swaps article.innerHTML wholesale — the old
detach/keep/re-append hack for .cards (which duplicated stacks) is gone.
Semantics: one card per item, no subtree flattening (stacks removed).
Bare {cards} lists immediate children (front page: the other top-level
items); a page-less item is represented by its first leaf page, the
nav-link logic. {cards: path} renders that page alone, path/* its
direct children, path/** all published descendants.
The language-switch trail ping read <html lang> before the view
transition applied the swap, so it reported the previous language (and
read pings never update trail-item languages server-side): multi-
language visits recorded as single-language trails and the analytics
flag combining had nothing to show. The switch ping now passes the
picked tag explicitly.
Language negotiation no longer prefers the article's original language
anywhere in the Accept-Language list — nearly every browser lists
English as a fallback, so translations were almost never served. The
first servable header language now wins, the original counting only in
its natural position.
Area and bar fills at 0.6 opacity (were a near-invisible 0.15); the day
view's 'Last 24 hours' legend entry uses the same full-opacity accent
line as the other entries instead of a translucent bar swatch.
Port the seasonal.py demo algorithm to JS (analytics/seasonal.js): the
smoothed 5-minute history is folded onto a weekly grid with exponential
age decay (7-day half-life for the time-of-day pattern, 42 days for
per-weekday deviations, shrunk by effective weeks of data), capped at
180 days of history where the weights become negligible.
Week view: the fading past-week overlays are replaced by the full
Monday-first 'Typical week' estimate. Day view gains the same estimate
cut to the rolling 24h window as a smooth muted 'Typical <weekday>'
curve behind the bars. Longer ranges are unchanged.
Favicon flush to the badge's left edge, then the referer host and UTM
summary in one link, with a single newline-formatted tooltip (origin,
then each utm_* pair). Fixed --badge-bg/--badge-text/--badge-muted
palette (defined in pagerite.css, deliberately unthemed) so transparent
favicons stay legible in both light and dark themes; own-article trail
links remain plain, lifting referer info visually apart from them.
Replaces the outlined utm-tag pill.
GETs store the resolved page language, activity pings carry <html lang>
(including a new ping on in-place language switches, which also keeps the
switch's GET out of crawler classification), and trail items/crawler hits
surface it. The viewer shows flags discreetly: nothing on single-language
sites or primary-language-only rows, one leading flag for uniform visits,
transition flags on mixed trails, a flag row per crawler.
LeoVasanko