The admin inbox leaves the short-lived /_m page and joins the visitor panel as one overlay (ContactOverlay shell over paskia's shared blur backdrop, centered dialog, full-screen on mobile): the corner ✉️ icon opens the visitor thread for visitors and the wider sidebar inbox for admins, while {contact:} trigger buttons always open the visitor side so admins can test both. The icon highlights unread state without extra connections: visitors via localStorage seen/last-admin timestamps, admins via the contact_unread count added to the /_api/settings probe, both kept live by the bundle once loaded. MessagesView owns its admin socket only while the overlay is open.
The inbox moves out of the analytics tab to its own page at /_m (own pens-row icon, own entry and socket lifecycle; the admin-gated /_api/ws/analytics socket keeps serving both, with typed messaging frames ignored by AnalyticsView). Admin replies are labeled with the currently configured site name in the wire payload (the SSO name stays stored for audit), and the thread view is capped to roughly the visitor panel's width.
Email is no longer required and its format is not checked anywhere
(placeholder notes it's optional); only the name gates the first
message. Timestamps are just the analytics relative format — no full
date on hover.
- The first message of a thread must carry name + email introducing the
visitor; the panel shows the two fields until the thread has an
identity (send stays disabled while they look empty, so the draft is
never lost), prefilled from and saved to localStorage. The identity
is stored on ContactThread (msgspec defaults — old databases decode
unchanged) and fixed afterwards; it labels the visitor's messages
instead of 'you', and the admin inbox lists threads by name with the
email as a mailto in the thread header.
- Timestamps use the analytics relative formatter (formatWhen), with
the full date on hover.
- Message headers alternate alignment (own messages right, theirs
left) as the visual side cue; the sent-from page link is dropped
from the display (kept as stored metadata).
- Composer gains a sendDisabled prop for the intro gate.
Move the PoW state from a module-level dict keyed by WebSocket in
messages.py to a plain per-connection local in tracking.activity_ws,
making PoW a WS-level concern instead of being buried in the message
handler:
- activity_ws validates and burns the solution (raw.pop('pow')) before
forwarding a message frame, and issues a fresh challenge after every
registration and every vetted message frame; a failed validation is
answered with a 'required' challenge and the frame never reaches
messages.py.
- messages.py loses the _pow dict, _issue_pow and the disconnect
cleanup for it; visitor_frame needs no return value since a vetted
frame always triggers a fresh challenge (a too-long message simply
comes with contact-error and required=false, so the client does not
retry it).
The wire protocol and the frontend are unchanged.
New dependency: easypow (PBKDF2-SHA512 challenges, bundled JS solver).
Sends are PoW-gated: registration/resync and every accepted message
come with a fresh single-use challenge (~5 s solve), held in per-socket
RAM state only — nothing in the database. A message burns one
solution; a missing, stale or spent solution is dropped and answered
with a fresh 'required' challenge. The bundled browser solver is
served at /_contact/easypow.js straight from the installed package, so
client and validator versions always match.
Uploads are deliberately not PoW-gated (the transfer rate is the real
cost). Instead, uploads are tracked as pending per session and a quota
caps pending files at 10 — answered 429, not 403, which could trip
auth flows. A sent message strikes the files it references off the
pending set; when the session's last socket closes, the remaining
pending files are deleted from the store.
Rework attachment storage and serving per the refined separation
scheme:
- Non-media uploads are stored as {hash}.file blobs (store_attachment)
— the original extension never lands on the filesystem, so the store
directory holds nothing double-click-executable. The original name
lives only in the URL: /_f/{hash}/{original-name}. The .file suffix
itself is rejected in URLs.
- Serving is strictly separated: /_f/{name} serves media only (hash
with or without extension, .orig. still blocked) and 404s .file
names; /_f/{name}/{filename} serves only {name}.file blobs, never
media, never by hash alone.
- Every /_f/ response now carries bare 'content-disposition:
attachment' (the URL's last segment already names the download) and
x-content-type-options: nosniff — media included. Subresource
fetches (<img>, <video>, favicons, the zoom viewer) ignore the
disposition, so inline media is unaffected, while a direct
navigation can never render or sniff a store file into something
executable in the site's origin. The mime-based script-carrier
forcing and the header filename are gone (subsumed).
- Message uploads treat SVG as media (rasterized derivatives; an
as-is-served .svg is script-inert inside <img>), and undecodable
fake images fall through to the .file attachment path.
Drop the script-carrier blocklist: any file type (html, exe, svg, ...)
may be attached now, with safety from enforced downloads instead of
type filtering, verified twice:
- the attachment chip anchor carries a download attribute (already),
- the link is /_f/{name}/{original-filename}, a new file-store route
that answers content-disposition: attachment naming the original
file (ASCII-safe fallback + RFC 5987 UTF-8 form). The filename rides
in the URL itself — no filename registry in the database.
Close the residual gap on the plain /_f/{name} route: a stored file
whose mime executes script in origin (html/xml/svg/js) is forced to
download there too, so stripping the filename segment off an
attachment URL cannot turn it into an in-origin page. All /_f/
responses now carry x-content-type-options: nosniff. Subresource
fetches (<img>, favicons) ignore Content-Disposition, so inline
images and icons are unaffected.
Undecodable fake images (store_image falls back to as-is storage) now
attach as files instead of being marked inline images.
The serializer treated only DIV/P as blocks: a formatting element
directly under the editable root (a bolded segment of an undivided
line is a top-level <b>) became a separate PLAIN block, losing both
the formatting and the paragraph structure. Top-level inline runs now
accumulate into one paragraph, and style-based formatting (Firefox's
font-weight/font-style spans) maps back to markers.
Attachments: the upload allowlist rejected everything but a handful of
types with a console-only 415. The logic inverts — raster images get
derivatives, script carriers (svg/html/js/xml...) are refused, and any
other file stores as a download chip. Upload failures now show in the
composer, and toolbar buttons keep the contenteditable selection on
click (mousedown.prevent).
docs/messages.md covers the trigger, both socket protocols, the storage
model, the XSS stance (grammar restriction via a strict renderer, never
HTML sanitization), uploads, and the MVP gaps (PoW seams, moderation,
orphan GC, i18n). AGENTS.md and frontend-runtime.md learn the new
module, bundles and asset keys.
Beads now take their connector's color — accent for internal links,
plain text color for external ones (matching the tconn-exit styling) —
and render as an elongated pulse along the direction of travel instead
of a plain dot: a rounded capsule with a gradient-faded tail, rotated
to the edge's angle (cached per flow). Flow objects carry an 'external'
tag from buildFlows; the gradients live in the svg defs with stop
colors resolved from CSS so theme variables apply.
The inactivity cutoff timer cannot fire while the computer is asleep, and
performance.now() keeps running through sleep on some platforms, so after
a wake the whole slept span could be credited as reading time (inflating
min/visit stats). A gap longer than the inactivity timeout now closes the
span at its intended cutoff, and the cutoff timer clamps its accumulation
to its scheduled fire time — covering both event-first and timer-first
wake orderings.
An expired session failed the /_api/ws/editor handshake exactly like a dead
network, so the page/banner editors retried forever — the WS API cannot
open the login dialog. reconnect.js gains authReconnect(): on close, probe
/_api/settings (dialog-less fetchJson); a 401/403 runs the paskia login
flow and reconnects the moment it completes, a declined login holds at a
'session expired' ConnNote, and only transient failures take the paced
backoff. A paskia auth-success message from anywhere in the page
reconnects a waiting editor at once. AnalyticsView moves from its inline
copy of the same flow onto the shared helper.
/pages.md (front page /.md) serves the page's source Markdown — per-language
hybrid assembly, title injected as '# title' when the content has no h1 —
with the same ETag/Last-Modified/no-cache scheme as the HTML, which now
carries vary: accept. Pages advertise the counterpart via
<link rel=alternate type=text/markdown> and llms.txt links point at the
.md URLs.
- Expired session dropped the socket into an endless paced retry loop the
WS handshake can never escape: probe /_api/settings on close, run the
paskia login flow on 401/403 and reconnect the moment it resolves; an
aborted login leaves for the front page (history replace).
- Follow paskia's auth-success/auth-logout window messages: logins anywhere
in the page reconnect at once instead of waiting out the backoff; logout
leaves /_a. pagerite.js re-probes its pens on the same events.
- Recycle the socket on visibilitychange/online/bfcache restore: a socket
that survived sleep looks OPEN but is dead, so no updates ever landed.
- The stats range window was frozen at mount (no reactive time dependency);
it now slides on a minute clock.
- Transition map beads: fast-forward emitter schedules after long gaps
instead of spawning a bead per missed interval in one tick.
Node inclusion no longer depends on views in the window: every page
from /_api/pages is shown (the front page used to vanish when quiet).
Omission follows the navigation instead — index pages without content
of their own (category labels and empty pages, the nodes nav links
bypass to the first child) are omitted, children promoted, first
visible child taking the slot. Deleted pages named by traffic are
inserted as ghosts where they would have been, materializing missing
ancestors as content-less category lanes.
/_api/pages gains the "empty" flag (not node.chunks) for the nav
notion; has_content (chunks is not None) stays for the structure
editor's delete affordance.
Rendering every article per request is expensive. Bodies are now
RAM-cached keyed by the public base URL and cleared from
_invalidate_pages (the same hook that drops the page render cache on
any content/settings write). Responses carry a blake3 content-hash
ETag and answer 304, so polling feed readers revalidate cheaply;
matching page behavior, 304 revalidations are not recorded in
analytics.
New pagerite/feeds.py: /llms.txt (Markdown site map with excerpts for
LLM agents), /feed.json (JSON Feed 1.1) and /feed.xml (RSS 2.0 +
atom:link), all carrying every published article with full content and
absolutized URLs. Linked from every page <head> (feed alternates +
llms-txt) and from the sitemap, recorded in analytics like page GETs
and emoji-marked in the trails (🧠 llms.txt, 📡 feeds).
The robots and sitemap routes never called _record_get, so those
accesses were invisible. Recorded like page GETs, they surface as
crawler hits at display time (no activity message ever follows). In
the trail rendering, known non-article machinery paths (robots,
sitemap, future feed paths) get an emoji marker instead of a bare
slug.
The old 1.4-2.5 aspect window sent big near-square images (e.g.
1350x1200) to the compact layout despite ample resolution. The new
rule keeps square and portrait images compact (the compact box fits
them) while letting anything big and clearly wider than tall render
as a full-card cover.
Replace count-sorted, group-centered rows with a spring-like placement:
pills order by their weighted median target x and settle by clamped
coordinate descent, minimizing weighted horizontal connection distance
while keeping the minimum pill spacing.
The banner panel's card-image block gains a pasteboard button (an image
uploads; an image URL goes as the save message's image setting itself and
is fetched/stored server-side, cross-origin being CORS-blocked for the
browser) and a site-icon button saving the @favicon sentinel, which
resolves to Data.favicon at render time and follows favicon changes.
The resolved (inheriting) Node.image was checked before mining the
article, so a front-page banner image shadowed the articles' own
extracted figures everywhere: card covers, og:image and the editor's
card previews.
Render the seasonal typical estimate as a translucent muted fill (no
stroke, same 0.6 opacity as the data) instead of a grey line, on both day
and week views. Continue the fixed Monday-Monday week view with last
week's curve from the current bucket to the end of the week (secondary
accent, translucent fill), so the chart is useful early in the week and
last week is gradually replaced by the current one.
Gate the typical estimate on history spanning twice the view's full time
(third day / third week on) and the last-week tail on data reaching into
last week at all; the legend shows each entry only when present.
Replace the old "patches" format (path:lang composite keys, ordered
hunk lists, text-anchored matching) with Data.overrides:
path -> lang -> LangEdits, keyed throughout so a save's database diff
touches only the edited chunks. The old "patches" key is ignored on
decode, discarding legacy data without a migration.
- Whole-paragraph additions/deletions are structural: a drop flag on
the original chunk hash, and additions in their own dict anchored
from the neighboring chunks' before/after (first live referrer wins),
so they stay in place across retranslation and one-sided original
edits.
- Within-paragraph edits (up to a full paragraph rewrite or split) are
full-chunk replace patches applied by chunk hash alone: a
retranslation is overridden wholesale, so user edits survive AI
re-runs; editing the original changes the hash and orphans the patch.
The old search-matching staleness gate is gone.
- Saving a translation on a page without original chunks is rejected
(REST 400 / WS error); emptying the original afterwards renders the
translation empty, with the orphaned overrides inert.
- Added full article / with markdown mode for advanced LLMs, with local ollama and API support, new llm_translator script.
- Translate navigational titles as a whole (preserving site structure), and then retranslate as article headings with article context - potentially changing nav label
- Improved human overrides to translations, in particular when structure is affected
Reviewed-on: https://git.zi.fi/LeoVasanko/pagerite/pulls/2
Extra paragraphs exist only inside a patch hunk's replace text (anchored
on the preceding original block). Deleting one produced a (X, "") hunk,
and apply_patch removed the text but not a surrounding separator, leaving
an accumulating blank line in the served/editor Markdown. hybrid_markdown
now re-chunks the patched text (join_chunks(chunk_markdown(...))), which
drops the stray blank lines on the read path — repairing gaps left by
already-stored patches too.
Adjacent fix in make_patch: a hunk whose search text occurred more than
once in the page hit the FIRST occurrence at apply time, blanking or
rewriting the wrong instance when identical paragraph text appeared
earlier. Ambiguous hunks now grow block context (preceding block first)
until unique; unique hunks stay context-free so they keep surviving
translation refreshes.
- Banner-corner auth link is now a button opening paskia-js's profile()
dialog (handles login too); auth re-probed when the dialog closes.
- Editor/analytics components call /_api via apiFetch/apiJson: an
expired session opens the login dialog and the request retries.
- pagerite.js: task-checkbox toggle uses apiJson (explicit edit attempt,
reverts on any failure incl. cancelled login); auth probes use
fetchJson (never a dialog); page-cache/navigation stay on plain fetch.
- Add the paskia npm dependency; document the convention.
Preview: the editor render handler now returns the complete article HTML
({cards} tags expanded, children's cards appended when the page has no
tag) and the frontend swaps article.innerHTML wholesale — the old
detach/keep/re-append hack for .cards (which duplicated stacks) is gone.
Semantics: one card per item, no subtree flattening (stacks removed).
Bare {cards} lists immediate children (front page: the other top-level
items); a page-less item is represented by its first leaf page, the
nav-link logic. {cards: path} renders that page alone, path/* its
direct children, path/** all published descendants.
The language-switch trail ping read <html lang> before the view
transition applied the swap, so it reported the previous language (and
read pings never update trail-item languages server-side): multi-
language visits recorded as single-language trails and the analytics
flag combining had nothing to show. The switch ping now passes the
picked tag explicitly.
Language negotiation no longer prefers the article's original language
anywhere in the Accept-Language list — nearly every browser lists
English as a fallback, so translations were almost never served. The
first servable header language now wins, the original counting only in
its natural position.
Port the seasonal.py demo algorithm to JS (analytics/seasonal.js): the
smoothed 5-minute history is folded onto a weekly grid with exponential
age decay (7-day half-life for the time-of-day pattern, 42 days for
per-weekday deviations, shrunk by effective weeks of data), capped at
180 days of history where the weights become negligible.
Week view: the fading past-week overlays are replaced by the full
Monday-first 'Typical week' estimate. Day view gains the same estimate
cut to the rolling 24h window as a smooth muted 'Typical <weekday>'
curve behind the bars. Longer ranges are unchanged.
GETs store the resolved page language, activity pings carry <html lang>
(including a new ping on in-place language switches, which also keeps the
switch's GET out of crawler classification), and trail items/crawler hits
surface it. The viewer shows flags discreetly: nothing on single-language
sites or primary-language-only rows, one leading flag for uniform visits,
transition flags on mixed trails, a flag row per crawler.
The stored ua_pretty froze each record at the uarite version of its
record time (old records showed disguised Meta crawlers as
"Chrome/145 Windows"). Client now carries a display-time-only
"uarite" field holding the full uarite.UA dataclass (pretty/engine/
os/provider/kind/url), filled when the viewer payload is built, so old
data always follows the current uarite. uarite 0.2.0 fixes the Meta
misdetection itself; _compact_user_agent is gone, tracking.py calls
uaparse directly.
A closing ::: glued to a paragraph (no blank line before it) rode inside
the prose chunk and crossed to the translator as part of the text run;
when the model dropped it, validation passed and the splice lost the
fence — the rest of the page rendered inside the container (seen in the
Spanish translation). Container fence lines (::: openers and closers
alike) are now always their own prose-free chunk, never reaching the
translator. Affected pages re-chunk on next save and re-translate under
the new hashes, repairing themselves.
Replace the pre-classified store (visits/crawlers/abuse lists written at
collection time, plus abuse_ips and in-memory pending/session tables) with
a raw append-only log: one Get record per document GET (full path, true
HTTP status, referer origin, preload flag) and one Msg per /_ws activity
message. Visitor/crawler/abuse classification, visit grouping (30-minute
inactivity gap), status/referer/UTM attribution and all aggregates are
derived in Store.display(), so future rule changes never invalidate stored
data. The viewer payload keeps its exact shape.
Fixes structurally:
- Abuser 404s on slug-format paths showed up as "articles read": the
not-found branch recorded the request twice through separate status
plumbing. Each request is now recorded once with its true status.
- 404 trail links never rendered red: cache-served navigations issue no
GET and the only real GET (the idle preload) was discarded before status
recording. Preloads are now recorded with pre=True, never counted, and
used for status attribution.
- formatAbuseRows merged a path's 404 probes and 200 reads into one entry;
the collapse is now keyed by (path, status class).
Rule improvements enabled by the redesign:
- The plain-404 abuse threshold counts within a 1-hour sliding window, so
long-time readers accumulating misses never classify (scanners spray).
- Hidden (admin) clients never trigger abuse classification: editing means
visiting not-found pages.
- /.well-known/ probes (RFC 8615, e.g. Chrome devtools) are never abuse
evidence; //foo-style empty path segments are an instant telltale.
- Visits can no longer open on an external exit URL; favicon fetches skip
hidden clients' referers/exits.
Legacy analytics.json files are set aside as .bak-legacy on startup.
LeoVasanko