contact messages: easypow proof-of-work for sends, pending-upload cleanup
New dependency: easypow (PBKDF2-SHA512 challenges, bundled JS solver). Sends are PoW-gated: registration/resync and every accepted message come with a fresh single-use challenge (~5 s solve), held in per-socket RAM state only — nothing in the database. A message burns one solution; a missing, stale or spent solution is dropped and answered with a fresh 'required' challenge. The bundled browser solver is served at /_contact/easypow.js straight from the installed package, so client and validator versions always match. Uploads are deliberately not PoW-gated (the transfer rate is the real cost). Instead, uploads are tracked as pending per session and a quota caps pending files at 10 — answered 429, not 403, which could trip auth flows. A sent message strikes the files it references off the pending set; when the session's last socket closes, the remaining pending files are deleted from the store.
This commit is contained in:
1 parent
6b34719397
commit
ec35bc71e8
4 files changed
+114
-24
No files matched your search
@@ -15,7 +15,7 @@ Pagerite is a CMS. See `docs` for the full design and implementation details. Ke
|
||||
- `files.py` — `FileStore` (content-addressed, RAM-cached), image derivative helpers (`store_image`; `store_attachment` keeps non-media uploads as inert `{hash}.file` blobs), file routes (`/_api/files`, `/_f/` — media by hash, attachments only via `/_f/{hash}/{filename}`, all forced-download + nosniff, `/_themes/`, `/_fonts/`, favicon settings).
|
||||
- `api.py` — editor REST + WS: `/_api/pages`, `/_api/structure`, `/_api/settings`, `/_api/toggle-task`, `/_api/translations`, `/_api/ws/editor`, `/_translate/{key}`.
|
||||
- `tracking.py` — visit analytics: GeoIP, client enrichment, favicon fetch, `/_ws`, `/_api/ws/analytics`, the `/_a` page (docs/analytics.md). Both sockets also carry the contact-message protocol (docs/messages.md): `/_ws` routes token-bearing visitor frames, the analytics socket the admin inbox.
|
||||
- `messages.py` — visitor contact threads (docs/messages.md): strict message rendering (`html=False` markdown-it — threads never carry user markup), thread store ops, the visitor/admin socket protocols, and the public `/_contact/files` upload route.
|
||||
- `messages.py` — visitor contact threads (docs/messages.md): strict message rendering (`html=False` markdown-it — threads never carry user markup), thread store ops, the visitor/admin socket protocols, easypow PoW gating (single-use challenges in per-socket RAM, browser solver served at `/_contact/easypow.js`), and the public `/_contact/files` upload route (pending-upload quota, unreferenced files deleted when the session's last socket closes).
|
||||
- `pages.py` — public content pages: `/`, `/sitemap.xml`, `/robots.txt`, the `/{path:path}` catch-all. Articles serve their Markdown (title injected as `# title`, per-language hybrid) at a `.md`-suffixed URL, or at the plain URL when Accept explicitly names text/markdown; pages link it via `<link rel="alternate" type="text/markdown">`, and llms.txt links point at the `.md` URLs.
|
||||
- `feeds.py` — machine-readable exports: `/llms.txt` (compact index linking each article's `.md` Markdown URL), `/feed.json` (JSON Feed 1.1), `/feed.xml` (RSS 2.0 + atom:link); the feeds carry all published articles with full content, linked from every page `<head>` and the sitemap, recorded in analytics.
|
||||
- `data.py` — msgspec Structs for the kanta database.
|
||||
@@ -49,7 +49,7 @@ Server run by CLI entry point `uv run pagerite` (no auto reloads, build needed).
|
||||
|
||||
## Toolchain
|
||||
|
||||
- Python >= 3.14, managed with **uv**. Dependencies: `fastapi[standard]`, `fastapi-vue`, `html5tagger`, `kanta`, `markdown-it-py`, `mdit-py-plugins`, `platformdirs`, `pygments`, `tracerite`; dev group has `httpx`. Run anything via `uv run ...` (the venv is `.venv`).
|
||||
- Python >= 3.14, managed with **uv**. Dependencies: `fastapi[standard]`, `fastapi-vue`, `easypow`, `html5tagger`, `kanta`, `markdown-it-py`, `mdit-py-plugins`, `platformdirs`, `pygments`, `tracerite`; dev group has `httpx`. Run anything via `uv run ...` (the venv is `.venv`).
|
||||
- Key libraries:
|
||||
- **html5tagger** — all HTML generation (`E`, `Document`, `Template`, `HTML` for trusted/raw HTML).
|
||||
- To create stand alone pages, begin with `doc = Document(...)` that gives a HTML5 page header
|
||||
|
||||
+35
-16
@@ -49,12 +49,28 @@ no extra connection). Frames **to** the server:
|
||||
`pagerite.js` re-sends this on every socket (re)connect, which is the
|
||||
resync mechanism (the socket idle-closes after 5 min and reconnects on
|
||||
activity).
|
||||
- `{token, msg, fr}` — append a message; `msg` is the Markdown-subset
|
||||
source, `fr` the page path it was sent from (stored for context).
|
||||
- `{token, msg, pow, fr}` — append a message; `msg` is the Markdown-subset
|
||||
source, `pow` the solved proof-of-work (below), `fr` the page path it
|
||||
was sent from (stored for context).
|
||||
|
||||
Frames **from** the server: `{type: "thread", id, created, unread,
|
||||
messages: [{id, t, admin, html, page}]}` — a full thread snapshot
|
||||
(replace/upsert by `id`), or `{type: "contact-error", detail}`.
|
||||
(replace/upsert by `id`); `{type: "pow", challenge, required}` — a fresh
|
||||
single-use PoW challenge (`required` means the last message was rejected
|
||||
for a missing/stale solution and should be re-sent once solved); or
|
||||
`{type: "contact-error", detail}`.
|
||||
|
||||
### Proof of work (anti-spam)
|
||||
|
||||
Sends are gated by easypow (PBKDF2 challenges, ~5 s solving cost). The
|
||||
server hands a fresh single-use challenge with every registration/resync
|
||||
and after every accepted message; the panel solves it in the background
|
||||
with the bundled solver module (served from the installed package at
|
||||
`/_contact/easypow.js`, so client and validator versions always match),
|
||||
typically finishing while the user is still writing. Each message burns
|
||||
one solution — a send with a missing, stale or already-spent solution is
|
||||
dropped and answered with a fresh `required` challenge, which the client
|
||||
solves and retries once. Challenges live in per-socket RAM state only.
|
||||
|
||||
## Admin side
|
||||
|
||||
@@ -110,11 +126,16 @@ filename needed — the URL's last segment names the download) and
|
||||
rendered or sniffed into something executable in the site's origin;
|
||||
subresource fetches (`<img>`, `<video>`, favicons) ignore the
|
||||
disposition, so inline media is unaffected. The chip anchors
|
||||
additionally carry a `download` attribute. Gates today: a
|
||||
session-shaped `x-contact-token` header and a 20 MB size cap. Admin
|
||||
uploads from the same composer go to the admin-gated `/_api/files`
|
||||
instead. Files uploaded but never referenced by a sent message
|
||||
currently linger (no orphan GC).
|
||||
additionally carry a `download` attribute. Uploads are deliberately
|
||||
**not** PoW-gated (the transfer rate is the real cost there); the gates
|
||||
are a session-shaped `x-contact-token` header, a 20 MB size cap, and a
|
||||
pending-upload quota — 10 un-sent files per session, answered 429 (not
|
||||
403, which could trip auth flows). Every upload stays *pending* per
|
||||
session: a sent message strikes the files it references off the pending
|
||||
set, and when the session's last socket closes the remaining pending
|
||||
files are deleted from the store. Admin uploads from the same composer
|
||||
go to the admin-gated `/_api/files` instead. Files left behind by a
|
||||
server restart (the pending set is RAM-only) still linger — no GC.
|
||||
|
||||
## Storage
|
||||
|
||||
@@ -129,11 +150,9 @@ transactions (`contact` / `contact:reply` / `contact:read`).
|
||||
|
||||
## Limits and deliberate gaps (MVP)
|
||||
|
||||
- Message source ≤ 8000 chars; uploads ≤ 20 MB; payloads cap at 100
|
||||
threads × last 200 messages.
|
||||
- **No proof-of-work yet**: the send and upload paths are the two seams
|
||||
where the PoW stamp plugs in (challenge on token registration, verify on
|
||||
`msg`/upload; one stamp may cover several messages).
|
||||
- No thread moderation (delete/block), no orphan-upload GC, panel strings
|
||||
are English-only, and the admin inbox lists threads by token-hash prefix
|
||||
rather than enriched visitor identity.
|
||||
- Message source ≤ 8000 chars; uploads ≤ 20 MB and ≤ 10 pending per
|
||||
session; payloads cap at 100 threads × last 200 messages.
|
||||
- No thread moderation (delete/block), no orphan-upload GC for files
|
||||
orphaned by a server restart, panel strings are English-only, and the
|
||||
admin inbox lists threads by token-hash prefix rather than enriched
|
||||
visitor identity.
|
||||
+76
-6
@@ -10,6 +10,15 @@ Admins watch and reply over the analytics socket
|
||||
frames go to ``admin_frame``. Both sides get pushed full thread payloads
|
||||
(threads are small); there is no REST polling anywhere.
|
||||
|
||||
Sends are proof-of-work gated (easypow): registration/resync and every
|
||||
accepted message come with a fresh single-use challenge, solved in the
|
||||
background by the browser (the bundled JS solver is served at
|
||||
/_contact/easypow.js); a message burns one solution. Uploads are not
|
||||
PoW-gated — the transfer rate is the real cost there — but uploaded
|
||||
files stay pending per session and are deleted when its last socket
|
||||
closes without a sent message referencing them; a quota caps pending
|
||||
files.
|
||||
|
||||
Message bodies are a Markdown subset (emphasis, links, lists, code plus
|
||||
/_f/ image and file references) serialized client-side from a constrained
|
||||
contenteditable composer. They are rendered here with a strict MarkdownIt
|
||||
@@ -29,12 +38,14 @@ from pathlib import Path
|
||||
from urllib.parse import quote
|
||||
|
||||
import blake3
|
||||
import easypow
|
||||
from fastapi import APIRouter, HTTPException, Request, WebSocket
|
||||
from fastapi.responses import Response
|
||||
from markdown_it import MarkdownIt
|
||||
from markdown_it.common.utils import escapeHtml
|
||||
|
||||
from pagerite.data import ContactMessage, ContactThread
|
||||
from pagerite.files import _ext, store_attachment, store_image
|
||||
from pagerite.files import _ext, file_store, store_attachment, store_image
|
||||
from pagerite.state import data, kanta
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -51,6 +62,16 @@ MAX_MESSAGE = 8000
|
||||
_visitor_ws: dict[bytes, set[WebSocket]] = {}
|
||||
_admin_ws: set[WebSocket] = set()
|
||||
|
||||
#: PoW state (RAM only): the outstanding single-use challenge per visitor
|
||||
#: socket, and uploaded-but-unreferenced file digests per session — the
|
||||
#: latter are deleted when the session's last socket closes.
|
||||
_pow: dict[WebSocket, str] = {}
|
||||
_pending_uploads: dict[bytes, set[str]] = {}
|
||||
|
||||
#: easypow solving cost per message and the pending-upload quota.
|
||||
POW_SECONDS = 5.0
|
||||
MAX_PENDING_UPLOADS = 10
|
||||
|
||||
|
||||
# --- Strict rendering -----------------------------------------------------
|
||||
# A separate instance from markdown.py's page renderer: no raw HTML (the
|
||||
@@ -182,9 +203,18 @@ def _append(key: bytes, msg: ContactMessage, action: str, user: str | None) -> N
|
||||
# --- Visitor protocol (/_ws frames carrying "token") ------------------------
|
||||
|
||||
|
||||
async def _issue_pow(ws: WebSocket, *, required: bool = False) -> None:
|
||||
"""Send a fresh single-use PoW challenge (~POW_SECONDS of solving),
|
||||
replacing any outstanding one for the socket."""
|
||||
challenge = easypow.generate(POW_SECONDS)
|
||||
_pow[ws] = challenge
|
||||
await ws.send_json({"type": "pow", "challenge": challenge, "required": required})
|
||||
|
||||
|
||||
async def visitor_frame(ws: WebSocket, raw: dict) -> None:
|
||||
"""Handle one visitor frame: ``{token}`` registers/resyncs (the thread
|
||||
state is pushed back), ``{token, msg, fr?}`` appends a message and
|
||||
state is pushed back, with a fresh PoW challenge), ``{token, msg, pow,
|
||||
fr?}`` burns the single-use PoW solution, appends the message and
|
||||
broadcasts the updated thread to both sides."""
|
||||
token = str(raw.get("token") or "")
|
||||
if not TOKEN_RE.fullmatch(token):
|
||||
@@ -196,10 +226,19 @@ async def visitor_frame(ws: WebSocket, raw: dict) -> None:
|
||||
if not text:
|
||||
thread = data.contact.get(key)
|
||||
await ws.send_json(_payload(key, thread) if thread else _empty_payload(key))
|
||||
await _issue_pow(ws)
|
||||
return
|
||||
if len(text) > MAX_MESSAGE:
|
||||
await ws.send_json({"type": "contact-error", "detail": "message too long"})
|
||||
return
|
||||
challenge = _pow.pop(ws, None) # every send attempt consumes it
|
||||
try:
|
||||
if challenge is None:
|
||||
raise ValueError("no outstanding challenge")
|
||||
easypow.validate(challenge, str(raw.get("pow") or ""))
|
||||
except ValueError:
|
||||
await _issue_pow(ws, required=True)
|
||||
return
|
||||
now = datetime.now(UTC)
|
||||
_append(
|
||||
key,
|
||||
@@ -212,16 +251,26 @@ async def visitor_frame(ws: WebSocket, raw: dict) -> None:
|
||||
"contact",
|
||||
None,
|
||||
)
|
||||
# Files the message references are no longer pending cleanup.
|
||||
if pending := _pending_uploads.get(key):
|
||||
for digest in re.findall(r"/_f/([0-9a-f]{12})", text):
|
||||
pending.discard(digest)
|
||||
await _broadcast(key, data.contact[key])
|
||||
await _issue_pow(ws) # the next message needs a fresh solution
|
||||
|
||||
|
||||
def visitor_disconnect(ws: WebSocket) -> None:
|
||||
"""Drop a closed /_ws socket from every thread registry."""
|
||||
"""Drop a closed /_ws socket from every registry. When a session's
|
||||
last socket closes, its uploaded files that no sent message ended up
|
||||
referencing are deleted from the store."""
|
||||
_pow.pop(ws, None)
|
||||
empty = [key for key, sockets in _visitor_ws.items() if not sockets - {ws}]
|
||||
for sockets in _visitor_ws.values():
|
||||
sockets.discard(ws)
|
||||
for key in empty:
|
||||
del _visitor_ws[key]
|
||||
for digest in _pending_uploads.pop(key, ()):
|
||||
file_store.delete(digest)
|
||||
|
||||
|
||||
# --- Admin protocol (/_api/ws/analytics frames) -----------------------------
|
||||
@@ -318,16 +367,23 @@ async def contact_upload(name: str, request: Request) -> dict:
|
||||
"""Store a contact-panel upload in the content-addressed store.
|
||||
|
||||
Public like the messaging socket (only /_api is gated): the caller
|
||||
must present a session-shaped token, size is capped, and abuse
|
||||
limiting is meant to ride on the proof-of-work stamp once plugged in.
|
||||
must present a session-shaped token and size is capped. Uploads are
|
||||
deliberately not PoW-gated (the transfer rate is the real cost);
|
||||
instead the file stays pending per session and is deleted when the
|
||||
session's last socket closes without a sent message referencing it,
|
||||
and a quota (429, not 403 — no auth flows here) caps pending files.
|
||||
Images (raster and SVG) get the usual derivative treatment and are
|
||||
referenced extension-less; any other file is stored as an inert
|
||||
{hash}.file blob and referenced as /_f/{hash}/{original-name}, which
|
||||
is always served as a forced download. Returns the /_f/ path and
|
||||
whether the file is an inline-able image.
|
||||
"""
|
||||
if not TOKEN_RE.fullmatch(request.headers.get("x-contact-token", "")):
|
||||
token = request.headers.get("x-contact-token", "")
|
||||
if not TOKEN_RE.fullmatch(token):
|
||||
raise HTTPException(403, "no session")
|
||||
pending = _pending_uploads.setdefault(thread_key(token), set())
|
||||
if len(pending) >= MAX_PENDING_UPLOADS:
|
||||
raise HTTPException(429, "too many un-sent uploads — send your message first")
|
||||
try:
|
||||
length = int(request.headers.get("content-length") or 0)
|
||||
except ValueError:
|
||||
@@ -346,10 +402,24 @@ async def contact_upload(name: str, request: Request) -> dict:
|
||||
# Decodable images come back extension-less (GIFs and as-is SVGs
|
||||
# keep their extension); an undecodable body attaches like any file.
|
||||
if "." not in stored or ext in (".gif", ".svg"):
|
||||
pending.add(stored.partition(".")[0])
|
||||
return {"path": f"/_f/{stored}", "image": True, "name": orig}
|
||||
stored = await asyncio.to_thread(store_attachment, body)
|
||||
pending.add(stored.removesuffix(".file"))
|
||||
return {
|
||||
"path": f"/_f/{stored.removesuffix('.file')}/{quote(orig, safe='')}",
|
||||
"image": False,
|
||||
"name": orig,
|
||||
}
|
||||
|
||||
|
||||
@router.get("/_contact/easypow.js")
|
||||
async def pow_solver() -> Response:
|
||||
"""The easypow browser solver as an ES module, served from the
|
||||
installed package so the client solver and the server validator are
|
||||
always the same version."""
|
||||
return Response(
|
||||
easypow.js,
|
||||
media_type="text/javascript",
|
||||
headers={"cache-control": "public, max-age=86400"},
|
||||
)
|
||||
@@ -17,6 +17,7 @@ readme = "README.md"
|
||||
requires-python = ">=3.14"
|
||||
dependencies = [
|
||||
"blake3>=1.0.9",
|
||||
"easypow>=0.1.0",
|
||||
"fastapi-vue~=1.7.2",
|
||||
"fastapi[standard]>=0.141.1",
|
||||
"html5tagger>=2.0.0",
|
||||
|
||||
Reference in new issue
Block a user