Fix FileStorage on Windows: open fd with O_BINARY, UTF-8 release logs

os.open() on Windows defaults to O_TEXT, so the CRT translated \r\n in
binary ciphertext: writes grew, offsets shifted, reads came up short and
the RAF callbacks failed with EIO (probe/read/write depending on where
a 0x0A byte landed). Pass O_BINARY where it exists.

Also reconfigure stdout/stderr to UTF-8 in tools/release.py so the
✓/✗/⚠ summary lines don't crash on cp1252 Windows consoles.

.gitea/workflows/release.yaml

@@ -0,0 +1,89 @@
+name: release
+
+on:
+  push:
+    tags:
+      - "v*"
+
+# Runner host prerequisites: git, uv, zig, and a C compiler.
+jobs:
+  build-wheels:
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - os: macos
+            shell: bash
+          - os: windows
+            shell: cmd
+          - os: linux
+            shell: bash
+    runs-on: ${{ matrix.os }}
+    steps:
+      # Plain git clone: full history so setuptools_scm sees tags, and both
+      # shells work. Windows uses cmd: bash resolves to WSL (refuses SYSTEM
+      # accounts) and powershell hits the script execution policy under SYSTEM.
+      # --recursive pulls in the libaegis submodule needed by the build backend.
+      - name: Checkout
+        if: matrix.os != 'windows'
+        shell: ${{ matrix.shell }}
+        run: |
+          git clone --recursive "${{ gitea.server_url }}/${{ gitea.repository }}.git" .
+          git checkout -f "${{ gitea.sha }}"
+
+      # Windows runners execute as SYSTEM with the workspace under
+      # C:\Windows\System32\config\systemprofile. cl.exe is a 32-bit binary,
+      # so WOW64 redirects its file access to SysWOW64 and it cannot see the
+      # sources (C1083: aegis.h not found). Build from a path outside %WINDIR%.
+      - name: Checkout
+        if: matrix.os == 'windows'
+        shell: cmd
+        run: |
+          if exist C:\aeg-build rmdir /s /q C:\aeg-build
+          git clone --recursive "${{ gitea.server_url }}/${{ gitea.repository }}.git" C:\aeg-build
+          cd /d C:\aeg-build
+          git checkout -f "${{ gitea.sha }}"
+
+      # Builds sdist + all wheel variants for this platform, runs tests and
+      # benchmarks, leaves everything in dist/. Upload is left to publish-pypi.
+      - name: Build wheels for this platform
+        if: matrix.os != 'windows'
+        shell: ${{ matrix.shell }}
+        run: uv run tools/release.py --no-upload
+
+      - name: Build wheels for this platform
+        if: matrix.os == 'windows'
+        shell: cmd
+        run: |
+          cd /d C:\aeg-build
+          uv run tools/release.py --no-upload
+
+      - name: Upload wheels as artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: dist-${{ matrix.os }}
+          path: ${{ matrix.os == 'windows' && 'C:\aeg-build\dist\' || 'dist/' }}
+          retention-days: 1
+
+  # Publish to PyPI only once every platform's wheel build has succeeded.
+  # Jobs don't share a workspace, so the per-platform dist/ directories are
+  # passed around as artifacts and merged here. No Gitea release is created.
+  publish-pypi:
+    needs: build-wheels
+    runs-on: linux
+    steps:
+      - name: Download all platform wheels
+        uses: actions/download-artifact@v4
+        with:
+          pattern: dist-*
+          path: dist
+          merge-multiple: true
+
+      # The sdist is identical across platforms; --skip-existing also makes
+      # reruns of a partially-published release safe.
+      - name: Publish to PyPI
+        shell: bash
+        env:
+          TWINE_USERNAME: __token__
+          TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}
+        run: uvx twine upload --skip-existing dist/**

.gitignore

@@ -8,4 +8,5 @@
 /src/aeg/_aegis*.pyd
 __pycache__
 !.gitignore
+!.gitea/
 !.gitmodules

README.md

@@ -17,19 +17,6 @@ Or add to your project using [UV](https://docs.astral.sh/uv/getting-started/inst
 uv add aeg
 ```
 
-## Variants
-
-All submodules expose the same API; pick one for your needs. The 256 bit variants offer maximal security and use larger key and nonce, while the 128 bit variants run slightly faster and use smaller key and nonce while still providing strong security. The MAC length does not depend on the variant. Note that the x2 and x4 variants are typically the fastest (depending on CPU) by utilizing SIMD multi-lane processing for the highest throughput.
-
-| Variant        | Key/Nonce Bytes | Notes                   |
-|----------------|----------------:|-------------------------|
-| **aegis128l**  | 16              |                         |
-| **aegis128x2** | 16              | Fastest on Intel Core   |
-| **aegis128x4** | 16              | Fastest on AMD and Xeon |
-| **aegis256**   | 32              |                         |
-| **aegis256x2** | 32              | Fast on Intel Core      |
-| **aegis256x4** | 32              | Fast on AMD and Xeon    |
-
 ## Quick start
 
 Normal authenticated encryption using the AEGIS-128X4 algorithm:
@@ -46,6 +33,27 @@ pt = ciph.decrypt(key, nonce, ct)   # Raises ValueError if anything was tampered
 assert pt == msg
 ```
 
+## Variants
+
+All submodules expose the same API; pick one for your needs. The 256 bit variants offer maximal security and use larger key and nonce, while the 128 bit variants run slightly faster and use smaller key and nonce while still providing strong security. The MAC length does not depend on the variant. Note that the x2 and x4 variants are typically the fastest (depending on CPU) by utilizing SIMD multi-lane processing for the highest throughput.
+
+| Variant        | Key/Nonce Bytes | Notes                   |
+|----------------|----------------:|-------------------------|
+| **aegis128l**  | 16              |                         |
+| **aegis128x2** | 16              | Fastest on Intel Core   |
+| **aegis128x4** | 16              | Fastest on AMD and Xeon |
+| **aegis256**   | 32              |                         |
+| **aegis256x2** | 32              | Fast on Intel Core      |
+| **aegis256x4** | 32              | Fast on AMD and Xeon    |
+
+Instead of importing the submodules, you can obtain one by its name string:
+
+```python
+import aeg
+
+ciph = aeg.cipher("AEGIS-128X2")   # Also accepts "aegis128x2" and other forms
+```
+
 ## API overview
 
 Common parameters and returns (applies to all items below):
@@ -71,10 +79,6 @@ The MAC tag is handled separately of ciphertext:
 - encrypt_detached(key, nonce, message, ad=None, maclen=16, ct_into=None, mac_into=None) -> (ct, mac)
 - decrypt_detached(key, nonce, ct, mac, ad=None, into=None) -> plaintext
 
-No MAC tag, vulnerable to alterations:
-- encrypt_unauthenticated(key, nonce, message, into=None) -> ciphertext  (testing only)
-- decrypt_unauthenticated(key, nonce, ct, into=None) -> plaintext        (testing only)
-
 ### Incremental AEAD
 
 Stateful classes that can be used for processing the data in separate chunks:
@@ -108,6 +112,7 @@ The `Mac` class follows the Python hashlib API for compatibility with code expec
 Useful for creating pseudo random bytes as rapidly as possible. Reuse of the same (key, nonce) creates identical output.
 
 - stream(key, nonce=None, length=None, into=None) -> randombytes
+- stream_xor(key, nonce, data, into=None) -> data XOR keystream (unauthenticated; encrypts and decrypts)
 
 ### Miscellaneous
 
@@ -118,6 +123,23 @@ Constants (per module): NAME, KEYBYTES, NONCEBYTES, MACBYTES, MACBYTES_LONG, RAT
 - nonce_increment(nonce)
 - wipe(buffer)
 
+### Random Generator (CSPRNG)
+
+Seeded once from the OS, then a cryptographically secure AEGIS keystream with thread local key/nonce making it thread safe. `random_key()` and `random_nonce()` draw from it. The into variant can directly fill Numpy integer arrays and other structures that support Buffer API, making it far faster than any alternatives (e.g. Numpy's own random module).
+
+- random().bytes(n) -> bytearray
+- random().into(buffer) -> None
+
+### Random-access encrypted files (aeg.raf)
+
+pread/pwrite-style access to files split into independently encrypted chunks, with optional Merkle tree integrity. Storage is pluggable: a file path, io.BytesIO, aeg.raf.FileStorage / BytesIOStorage / StreamStorage, or a custom Storage.
+
+- raf.create(path_or_storage, key, cipher, chunk_size=65536, merkle=False) / raf.open(...)
+- f.read(size, offset), f.pread(size, offset), f.write(data, offset), f.pwrite(data, offset)
+- f.seek/tell/truncate/sync/close, context manager support
+- f.merkle_rebuild(), f.merkle_verify(), f.root_hash, f.verify_root(expected)
+- raf.probe(storage) -> RafInfo, raf.derive_master_key(key, context)
+
 ### Exceptions
 
 - Authentication failures raise ValueError.
@@ -251,6 +273,14 @@ For advanced use cases, the output buffer can be supplied with `into` kwarg. Any
 
 A `TypeError` is raised if the buffer is too small. For convenience, the functions return a memoryview showing only the bytes actually written.
 
+For sensitive data, `aeg.securebuf.SecureBuffer` provides memory locked against swapping and excluded from core dumps, wiped on close:
+
+```python
+from aeg.securebuf import SecureBuffer
+with SecureBuffer(1024) as buf:
+    pt = ciph.decrypt(key, nonce, ct, into=buf)
+```
+
 Foreign arrays can be used. This example fills a Numpy array with random integers.
 
 ```python
@@ -278,7 +308,7 @@ pt = ciph.decrypt(key, nonce, ct, into=buf)
 print(bytes(pt))
 ```
 
-Detached and unauthenticated modes can use same size input and output (no MAC added to ciphertext). Detached encryption instead of `into` takes `ct_into` and `mac_into` separately and returns memoryviews to both.
+Detached mode can use same size input and output (no MAC added to ciphertext). Detached encryption instead of `into` takes `ct_into` and `mac_into` separately and returns memoryviews to both.
 
 ## Performance
 

pyproject.toml

@@ -1,5 +1,5 @@
 [build-system]
-requires = ["setuptools>=61.0", "cffi>=2.0.0", "setuptools-scm>=8.0"]
+requires = ["setuptools>=84.0.0", "cffi>=2.1.1", "setuptools-scm>=10.3.4"]
 build-backend = "build_backend"
 backend-path = ["tools"]
 
@@ -8,14 +8,15 @@ name = "aeg"
 dynamic = ["version"]
 description = "AEGIS encryption easy to use Python binding. Wheels for major platforms."
 readme = {file = "README.md", content-type = "text/markdown"}
-requires-python = ">=3.10"
+requires-python = ">=3.12"
 classifiers = [
+  "Development Status :: 5 - Production/Stable",
   "Programming Language :: Python :: Implementation :: CPython",
   "Operating System :: OS Independent",
   "Topic :: Security :: Cryptography",
 ]
 dependencies = [
-    "cffi>=2.0.0",
+    "cffi>=2.1.1",
 ]
 
 [project.urls]
@@ -24,11 +25,12 @@ Repository = "https://github.com/LeoVasanko/aegis-python"
 
 [dependency-groups]
 dev = [
-    "auditwheel>=6.5.0",
-    "pytest>=8.4.2",
-    "ruff>=0.14.4",
-    "setuptools>=80.9.0",
-    "setuptools-scm>=9.2.2",
+    "auditwheel>=6.8.2",
+    "patchelf>=0.17; sys_platform == 'linux'",
+    "pytest>=9.1.1",
+    "ruff>=0.16.10",
+    "setuptools>=84.0.0",
+    "setuptools-scm>=10.3.4",
 ]
 
 [tool.setuptools]
@@ -36,6 +38,18 @@ package-dir = {"" = "src"}
 packages = ["aeg"]
 
 [tool.setuptools.package-data]
-aeg = ["*.h", "*.so", "*.pyd"]
+aeg = ["*.h"]
+
+[tool.ruff]
+# Disable formatting in Markdown files
+extend-exclude = ["*.md"]
+
+[tool.ruff.lint]
+ignore = [
+    "RUF022",  # __all__ is deliberately grouped with comments
+    "RUF023",  # __slots__ is deliberately grouped
+    "SIM117",  # nested with-statements are sometimes clearer
+    "BLE001",  # blind except is intentional at CFFI callback boundaries
+]
 
 [tool.setuptools_scm]

setup.py

@@ -1,35 +1,31 @@
 """Setup script for aeg - builds CFFI extension with libaegis C library."""
 
 import sys
+import sysconfig
 from pathlib import Path
 
 from cffi import FFI
 from setuptools import setup
 
-# Locate the static library (built by build_backend.py before this runs)
-lib_name = "aegis.lib" if sys.platform == "win32" else "libaegis.a"
-libaegis_static = Path("libaegis/zig-out/lib") / lib_name
-if not libaegis_static.exists():
-    raise RuntimeError(f"libaegis static library not found at {libaegis_static}")
-libaegis_static = str(libaegis_static.resolve())
+ROOT = Path(__file__).parent.resolve()
 
-# Include directory for headers
-libaegis_include = Path("libaegis/src/include")
-if not libaegis_include.exists():
-    raise RuntimeError(f"libaegis include directory not found at {libaegis_include}")
-include_dirs = [str(libaegis_include)]
+if sys.maxsize <= 2**32:
+    raise RuntimeError(
+        "aeg does not support 32-bit platforms; build with a 64-bit Python"
+    )
 
-# Read the CDEF header
-cdef_path = Path(__file__).parent / "src" / "aeg" / "aegis_cdef.h"
-cdef_content = cdef_path.read_text(encoding="utf-8")
+libaegis_static = ROOT / "libaegis/zig-out/lib" / (
+    "aegis.lib" if sys.platform == "win32" else "libaegis.a"
+)
 
-# Create CFFI builder
 ffibuilder = FFI()
-ffibuilder.cdef(cdef_content)
+ffibuilder.cdef((ROOT / "src/aeg/aegis_cdef.h").read_text())
+
+# Free-threaded Python does not support Limited API (abi3)
+is_free_threaded = sysconfig.get_config_var("Py_GIL_DISABLED")
 
-# Set the source
 ffibuilder.set_source(
-    "aeg._aegis",  # module name
+    "aeg._aegis",
     """
     #include "aegis.h"
     #include "aegis128l.h"
@@ -39,11 +35,15 @@ ffibuilder.set_source(
     #include "aegis256x2.h"
     #include "aegis256x4.h"
     """,
-    include_dirs=include_dirs,
-    extra_objects=[libaegis_static],
+    include_dirs=[str(ROOT / "libaegis/src/include")],
+    extra_objects=[str(libaegis_static)],
+    py_limited_api=not is_free_threaded,
 )
 
 if __name__ == "__main__":
     setup(
         cffi_modules=["setup.py:ffibuilder"],
+        options=(
+            {"bdist_wheel": {"py_limited_api": "cp312"}} if not is_free_threaded else {}
+        ),
     )

src/aeg/__init__.py

@@ -11,6 +11,8 @@ def cipher(alg: CipherName) -> Cipher:
     name = alg.lower().replace("-", "")
     if name == "aegis128":
         name = "aegis128l"  # AEGIS-128 is dead, the user meant AEGIS-128L
+    if not name.startswith("aegis"):
+        name = "aegis" + name
     if name in CIPHERS.values():
         return importlib.import_module(f".{name}", __package__)  # type: ignore[return-value]
     raise ValueError(f"Unknown algorithm {alg!r}. Valid options: {', '.join(CIPHERS)}")

src/aeg/_typing.py

@@ -1,6 +1,7 @@
 from typing import TYPE_CHECKING, Protocol
 
 if TYPE_CHECKING:
+    from .random import Random
     from .util import Buffer
 
 __all__ = ["Cipher"]
@@ -43,6 +44,8 @@ class Cipher(Protocol):
     Encryptor: type[_Encryptor]
     Decryptor: type[_Decryptor]
 
+    @staticmethod
+    def random() -> "Random": ...
     @staticmethod
     def random_key() -> bytearray: ...
     @staticmethod
@@ -97,7 +100,15 @@ class Cipher(Protocol):
         into: "Buffer | None" = None,
     ) -> "bytearray | Buffer": ...
     @staticmethod
-    def encrypt_unauthenticated(
+    def stream_xor(
+        key: "Buffer",
+        nonce: "Buffer",
+        data: "Buffer",
+        *,
+        into: "Buffer | None" = None,
+    ) -> bytearray | memoryview: ...
+    @staticmethod
+    def encrypt_unauthenticated(  # deprecated: use stream_xor
         key: "Buffer",
         nonce: "Buffer",
         message: "Buffer",
@@ -105,7 +116,7 @@ class Cipher(Protocol):
         into: "Buffer | None" = None,
     ) -> bytearray | memoryview: ...
     @staticmethod
-    def decrypt_unauthenticated(
+    def decrypt_unauthenticated(  # deprecated: use stream_xor
         key: "Buffer",
         nonce: "Buffer",
         ct: "Buffer",
@@ -120,3 +131,39 @@ class Cipher(Protocol):
         maclen: int = ...,
         into: "Buffer | None" = None,
     ) -> bytearray | memoryview: ...
+    @staticmethod
+    def nonce_increment(nonce: "Buffer") -> None: ...
+    @staticmethod
+    def wipe(buffer: "Buffer") -> None: ...
+
+    # random-access encrypted files (thin wrappers; see aeg.raf)
+    RAF_ALG_ID: int
+
+    @staticmethod
+    def new_raf_ctx(): ...
+    @staticmethod
+    def raf_scratch_size(chunk_size: int) -> int: ...
+    @staticmethod
+    def raf_create(key: "Buffer", ctx, io, rng, config) -> None: ...
+    @staticmethod
+    def raf_open(key: "Buffer", ctx, io, rng, config) -> None: ...
+    @staticmethod
+    def raf_read(
+        ctx, length: int, offset: int, *, into: "Buffer | None" = None
+    ) -> bytearray | memoryview: ...
+    @staticmethod
+    def raf_write(ctx, data: "Buffer", offset: int) -> int: ...
+    @staticmethod
+    def raf_truncate(ctx, size: int) -> None: ...
+    @staticmethod
+    def raf_size(ctx) -> int: ...
+    @staticmethod
+    def raf_sync(ctx) -> None: ...
+    @staticmethod
+    def raf_close(ctx) -> None: ...
+    @staticmethod
+    def raf_merkle_rebuild(ctx) -> None: ...
+    @staticmethod
+    def raf_merkle_verify(ctx) -> "int | None": ...
+    @staticmethod
+    def raf_merkle_commitment(ctx, hash_len: int) -> bytes: ...

src/aeg/aegis128l.py

@@ -3,11 +3,14 @@
 # DO NOT EDIT OTHER ALGORITHM FILES MANUALLY!
 
 import errno
-import secrets
+import sys
+import threading
+import warnings
 from typing import Literal
 
 from ._loader import ffi
 from ._loader import lib as _lib
+from .random import Random
 from .util import Buffer, new_aligned_struct, nonce_increment, wipe
 
 NAME = "AEGIS-128L"  #: Algorithm display name
@@ -19,25 +22,41 @@ ALIGNMENT = 64  #: Required alignment for internal structures
 RATE = 64  #: Byte chunk size in internal processing
 
 
+_random_tls = threading.local()
+
+
+def random() -> Random:
+    """Return this module's Random instance (per-thread singleton).
+
+    Each thread gets its own generator on first use, seeded once from the
+    OS, then deterministic: successive calls never repeat an output block.
+    No locking is involved. See aeg.random.Random for the API.
+    """
+    rng = getattr(_random_tls, "rng", None)
+    if rng is None:
+        rng = _random_tls.rng = Random(sys.modules[__name__])
+    return rng
+
+
 def random_key() -> bytearray:
     """
-    Generate a secret key using cryptographically secure random bytes.
+    Generate a secret key using the module's CSPRNG (see random()).
 
     It is recommended to wipe() the key after no longer needed.
     """
-    return bytearray(secrets.token_bytes(KEYBYTES))
+    return random().bytes(KEYBYTES)
 
 
 def random_nonce() -> bytearray:
     """
-    Generate a public nonce using cryptographically secure random bytes.
+    Generate a public nonce using the module's CSPRNG (see random()).
 
     Nonces (a number used once) are public data that may be sent together
     with the ciphertext, but they need to be unique for each use.
 
     See also: nonce_increment() can be used to derive sequential nonces.
     """
-    return bytearray(secrets.token_bytes(NONCEBYTES))
+    return random().bytes(NONCEBYTES)
 
 
 def _ptr(buf):
@@ -360,6 +379,55 @@ def stream(
     return out if into is None else memoryview(out)[: length or memoryview(out).nbytes]  # type: ignore
 
 
+def stream_xor(
+    key: Buffer,
+    nonce: Buffer,
+    data: Buffer,
+    *,
+    into: Buffer | None = None,
+) -> bytearray | memoryview:
+    """XOR data with the keystream from stream(); encrypts and decrypts.
+
+    Unauthenticated: only use when the data is authenticated some other
+    way. Never reuse a nonce with the same key.
+
+    Args:
+        key: Secret key (generate with random_key()).
+        nonce: Public nonce (generate with random_nonce()).
+        data: Input bytes to XOR.
+        into: Buffer to write the output into (default: bytearray created).
+
+    Returns:
+        Output as bytearray if into not provided, memoryview of into otherwise.
+
+    Raises:
+        TypeError: If lengths are invalid.
+    """
+    key = memoryview(key)
+    nonce = memoryview(nonce)
+    data = memoryview(data)
+    if into is not None:
+        into = memoryview(into)
+    if key.nbytes != KEYBYTES:
+        raise TypeError(f"key length must be {KEYBYTES}")
+    if nonce.nbytes != NONCEBYTES:
+        raise TypeError(f"nonce length must be {NONCEBYTES}")
+    if into is None:
+        out = bytearray(data.nbytes)
+    else:
+        if into.nbytes < data.nbytes:
+            raise TypeError("into length must be at least data.nbytes")
+        out = into
+    _lib.aegis128l_stream_xor(
+        ffi.from_buffer(out),
+        _ptr(data),
+        data.nbytes,
+        _ptr(nonce),
+        _ptr(key),
+    )
+    return out if into is None else memoryview(out)[: data.nbytes]  # type: ignore
+
+
 def encrypt_unauthenticated(
     key: Buffer,
     nonce: Buffer,
@@ -369,6 +437,10 @@ def encrypt_unauthenticated(
 ) -> bytearray | memoryview:
     """Encrypt message without authentication (for testing/debugging).
 
+    .. deprecated::
+        Use stream_xor() in new code. Its output is different, so keep
+        this only for data in the old format.
+
     Args:
         key: Secret key (generate with random_key()).
         nonce: Public nonce (generate with random_nonce()).
@@ -381,6 +453,12 @@ def encrypt_unauthenticated(
     Raises:
         TypeError: If lengths are invalid.
     """
+    warnings.warn(
+        "encrypt_unauthenticated is deprecated; use stream_xor "
+        "(its output differs, so keep this only for existing data)",
+        DeprecationWarning,
+        stacklevel=2,
+    )
     key = memoryview(key)
     nonce = memoryview(nonce)
     message = memoryview(message)
@@ -415,6 +493,10 @@ def decrypt_unauthenticated(
 ) -> bytearray | memoryview:
     """Decrypt ciphertext without authentication (for testing/debugging).
 
+    .. deprecated::
+        Use stream_xor() in new code. Its output is different, so keep
+        this only for data in the old format.
+
     Args:
         key: Secret key (same key used during encryption).
         nonce: Public nonce (same nonce used during encryption).
@@ -427,6 +509,12 @@ def decrypt_unauthenticated(
     Raises:
         TypeError: If lengths are invalid.
     """
+    warnings.warn(
+        "decrypt_unauthenticated is deprecated; use stream_xor "
+        "(its output differs, so keep this only for existing data)",
+        DeprecationWarning,
+        stacklevel=2,
+    )
     key = memoryview(key)
     nonce = memoryview(nonce)
     ct = memoryview(ct)
@@ -708,24 +796,17 @@ class Encryptor:
             raise TypeError(
                 "into length must be >= expected output size for this update"
             )
-        written = ffi.new("size_t *")
         rc = _lib.aegis128l_state_encrypt_update(
             self._state.ptr,
             ffi.from_buffer(out_mv),
-            out_mv.nbytes,
-            written,
             _ptr(message),
             message.nbytes,
         )
         if rc != 0:
             err_num = ffi.errno
             err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
-            raise RuntimeError(
-                f"state encrypt update failed: {err_name} written {written[0]}"
-            )
-        w = int(written[0])
-        assert w == expected_out
-        return out if into is None else memoryview(out)[:w]  # type: ignore
+            raise RuntimeError(f"state encrypt update failed: {err_name}")
+        return out if into is None else memoryview(out)[:expected_out]  # type: ignore
 
     def final(self, into: Buffer | None = None) -> bytearray | memoryview:
         """Finalize encryption and return the authentication tag.
@@ -746,24 +827,17 @@ class Encryptor:
         if into is not None:
             into = memoryview(into)
         out = into if into is not None else bytearray(maclen)
-        written = ffi.new("size_t *")
         rc = _lib.aegis128l_state_encrypt_final(
             self._state.ptr,
             ffi.from_buffer(out),
-            memoryview(out).nbytes,
-            written,
             maclen,
         )
         if rc != 0:
             err_num = ffi.errno
             err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
             raise RuntimeError(f"state encrypt final failed: {err_name}")
-        w = int(written[0])
-        if into is None:
-            # Only the tag bytes are returned when we allocate the buffer
-            assert w == maclen
         self._state = None
-        return out if into is None else memoryview(out)[:w]  # type: ignore
+        return out if into is None else memoryview(out)[:maclen]  # type: ignore
 
 
 class Decryptor:
@@ -837,12 +911,9 @@ class Decryptor:
         out_mv = memoryview(out)
         if out_mv.nbytes < expected_out:
             raise TypeError("into length must be >= required capacity for this update")
-        written = ffi.new("size_t *")
-        rc = _lib.aegis128l_state_decrypt_detached_update(
+        rc = _lib.aegis128l_state_decrypt_update(
             self._state.ptr,
             ffi.from_buffer(out_mv),
-            out_mv.nbytes,
-            written,
             _ptr(ct),
             ct.nbytes,
         )
@@ -850,11 +921,7 @@ class Decryptor:
             err_num = ffi.errno
             err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
             raise RuntimeError(f"state decrypt update failed: {err_name}")
-        w = int(written[0])
-        assert w == expected_out, (
-            f"got {w}, expected {expected_out}, ct.nbytes={ct.nbytes}"
-        )
-        return out if into is None else memoryview(out)[:w]  # type: ignore
+        return out if into is None else memoryview(out)[:expected_out]  # type: ignore
 
     def final(self, mac: Buffer) -> None:
         """Finalize decryption by verifying the MAC tag.
@@ -873,9 +940,7 @@ class Decryptor:
         mac = memoryview(mac)
         if mac.nbytes != maclen:
             raise TypeError(f"mac length must be {maclen}")
-        rc = _lib.aegis128l_state_decrypt_detached_final(
-            self._state.ptr, ffi.NULL, 0, ffi.NULL, _ptr(mac), maclen
-        )
+        rc = _lib.aegis128l_state_decrypt_final(self._state.ptr, _ptr(mac), maclen)
         if rc != 0:
             raise ValueError("authentication failed")
         self._state = None
@@ -891,6 +956,220 @@ def new_mac_state():
     return new_aligned_struct("aegis128l_mac_state", ALIGNMENT)
 
 
+RAF_ALG_ID = 1  #: Algorithm identifier in RAF file headers
+
+
+def _raise_errno(operation: str) -> None:
+    err_num = ffi.errno
+    err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
+    raise RuntimeError(f"{operation} failed: {err_name}")
+
+
+def new_raf_ctx():
+    """Allocate and return a new aegis128l_raf_ctx* with proper alignment."""
+    return new_aligned_struct("aegis128l_raf_ctx", ALIGNMENT)
+
+
+def raf_scratch_size(chunk_size: int) -> int:
+    """Return the scratch buffer size required for the given chunk_size."""
+    return _lib.aegis128l_raf_scratch_size(chunk_size)
+
+
+def raf_create(key: Buffer, ctx, io, rng, config) -> None:
+    """Create a new encrypted file, writing the header through io.
+
+    Args:
+        key: Master key (KEYBYTES bytes).
+        ctx: Context from new_raf_ctx().
+        io: aegis_raf_io* cdata (see aeg.raf).
+        rng: aegis_raf_rng* cdata (see aeg.random).
+        config: aegis_raf_config* cdata (see aeg.raf).
+
+    Raises:
+        TypeError: If key length is invalid.
+        FileExistsError: If the file exists and AEGIS_RAF_TRUNCATE was not set.
+        RuntimeError: If creation fails.
+    """
+    key = memoryview(key)
+    if key.nbytes != KEYBYTES:
+        raise TypeError(f"key length must be {KEYBYTES}")
+    rc = _lib.aegis128l_raf_create(ctx.ptr, io, rng, config, _ptr(key))
+    if rc != 0:
+        if ffi.errno == errno.EEXIST:
+            raise FileExistsError("file exists (use truncate to overwrite)")
+        _raise_errno("raf create")
+
+
+def raf_open(key: Buffer, ctx, io, rng, config) -> None:
+    """Open an existing encrypted file, verifying the header MAC.
+
+    Args:
+        key: Master key (KEYBYTES bytes).
+        ctx: Context from new_raf_ctx().
+        io: aegis_raf_io* cdata (see aeg.raf).
+        rng: aegis_raf_rng* cdata (see aeg.random).
+        config: aegis_raf_config* cdata (see aeg.raf).
+
+    Raises:
+        TypeError: If key length is invalid.
+        FileNotFoundError: If the file does not exist.
+        ValueError: If the header is invalid or authentication fails (wrong key).
+        RuntimeError: If opening fails.
+    """
+    key = memoryview(key)
+    if key.nbytes != KEYBYTES:
+        raise TypeError(f"key length must be {KEYBYTES}")
+    rc = _lib.aegis128l_raf_open(ctx.ptr, io, rng, config, _ptr(key))
+    if rc != 0:
+        err_num = ffi.errno
+        if err_num == errno.ENOENT:
+            raise FileNotFoundError("file not found")
+        if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+            raise ValueError("authentication failed")
+        _raise_errno("raf open")
+
+
+def raf_read(
+    ctx, length: int, offset: int, *, into: Buffer | None = None
+) -> bytearray | memoryview:
+    """Read and decrypt up to length bytes at offset.
+
+    Returns:
+        The decrypted bytes, trimmed to the number actually read
+        (fewer at EOF), as bytearray if into not provided,
+        memoryview of into otherwise.
+
+    Raises:
+        TypeError: If into is smaller than length.
+        ValueError: If authentication fails (corruption or tampering).
+        RuntimeError: If the read fails.
+    """
+    if into is None:
+        out = bytearray(length)
+    else:
+        into = memoryview(into)
+        if into.nbytes < length:
+            raise TypeError("into length must be at least length")
+        out = into
+    bytes_read = ffi.new("size_t*")
+    rc = _lib.aegis128l_raf_read(
+        ctx.ptr, ffi.from_buffer(out), bytes_read, length, offset
+    )
+    if rc != 0:
+        err_num = ffi.errno
+        if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+            raise ValueError("authentication failed")
+        _raise_errno("raf read")
+    n = bytes_read[0]
+    return out[:n] if into is None else memoryview(out)[:n]  # type: ignore
+
+
+def raf_write(ctx, data: Buffer, offset: int) -> int:
+    """Encrypt and write data at offset, extending the file as needed.
+
+    Returns:
+        Number of bytes written.
+
+    Raises:
+        RuntimeError: If the write fails (errno name included).
+    """
+    data = memoryview(data)
+    bytes_written = ffi.new("size_t*")
+    rc = _lib.aegis128l_raf_write(
+        ctx.ptr, bytes_written, _ptr(data), data.nbytes, offset
+    )
+    if rc != 0:
+        _raise_errno("raf write")
+    return bytes_written[0]
+
+
+def raf_truncate(ctx, size: int) -> None:
+    """Resize the file to size bytes (zero-filled when growing).
+
+    Raises:
+        RuntimeError: If the truncate fails.
+    """
+    rc = _lib.aegis128l_raf_truncate(ctx.ptr, size)
+    if rc != 0:
+        _raise_errno("raf truncate")
+
+
+def raf_size(ctx) -> int:
+    """Return the logical plaintext file size."""
+    size = ffi.new("uint64_t*")
+    rc = _lib.aegis128l_raf_get_size(ctx.ptr, size)
+    if rc != 0:
+        _raise_errno("raf get_size")
+    return size[0]
+
+
+def raf_sync(ctx) -> None:
+    """Flush writes to the backing store.
+
+    Raises:
+        RuntimeError: If the sync fails.
+    """
+    rc = _lib.aegis128l_raf_sync(ctx.ptr)
+    if rc != 0:
+        _raise_errno("raf sync")
+
+
+def raf_close(ctx) -> None:
+    """Close the context, syncing and zeroizing key material."""
+    _lib.aegis128l_raf_close(ctx.ptr)
+
+
+def raf_merkle_rebuild(ctx) -> None:
+    """Rebuild the Merkle tree by reading and re-hashing every chunk.
+
+    Raises:
+        RuntimeError: If Merkle is not enabled or rebuilding fails.
+        ValueError: If a chunk fails authentication.
+    """
+    rc = _lib.aegis128l_raf_merkle_rebuild(ctx.ptr)
+    if rc != 0:
+        err_num = ffi.errno
+        if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+            raise ValueError("authentication failed")
+        _raise_errno("raf merkle_rebuild")
+
+
+def raf_merkle_verify(ctx) -> int | None:
+    """Verify every chunk's hash against the Merkle tree.
+
+    Returns:
+        None if all chunks match, otherwise the index of the first
+        corrupted chunk.
+
+    Raises:
+        RuntimeError: If Merkle is not enabled or verification fails.
+    """
+    corrupted = ffi.new("uint64_t*")
+    rc = _lib.aegis128l_raf_merkle_verify(ctx.ptr, corrupted)
+    if rc == 0:
+        return None
+    err_num = ffi.errno
+    if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+        return corrupted[0]
+    _raise_errno("raf merkle_verify")
+
+
+def raf_merkle_commitment(ctx, hash_len: int) -> bytes:
+    """Compute the Merkle root commitment hash.
+
+    Returns:
+        The commitment hash as bytes of length hash_len.
+
+    Raises:
+        RuntimeError: If Merkle is not enabled or the call fails.
+    """
+    out = ffi.new(f"uint8_t[{hash_len}]")
+    rc = _lib.aegis128l_raf_merkle_commitment(ctx.ptr, out, hash_len)
+    if rc != 0:
+        _raise_errno("raf merkle_commitment")
+    return bytes(ffi.buffer(out, hash_len))
+
+
 __all__ = [
     # constants
     "NAME",
@@ -900,7 +1179,9 @@ __all__ = [
     "MACBYTES_LONG",
     "ALIGNMENT",
     "RATE",
+    "RAF_ALG_ID",
     # utility functions
+    "random",
     "random_key",
     "random_nonce",
     "nonce_increment",
@@ -911,6 +1192,7 @@ __all__ = [
     "encrypt",
     "decrypt",
     "stream",
+    "stream_xor",
     "encrypt_unauthenticated",
     "decrypt_unauthenticated",
     "mac",
@@ -918,4 +1200,18 @@ __all__ = [
     "Encryptor",
     "Decryptor",
     "Mac",
+    # random-access encrypted files (thin wrappers; see aeg.raf for the high-level API)
+    "new_raf_ctx",
+    "raf_scratch_size",
+    "raf_create",
+    "raf_open",
+    "raf_read",
+    "raf_write",
+    "raf_truncate",
+    "raf_size",
+    "raf_sync",
+    "raf_close",
+    "raf_merkle_rebuild",
+    "raf_merkle_verify",
+    "raf_merkle_commitment",
 ]

src/aeg/aegis128x2.py

@@ -3,11 +3,14 @@
 # DO NOT EDIT OTHER ALGORITHM FILES MANUALLY!
 
 import errno
-import secrets
+import sys
+import threading
+import warnings
 from typing import Literal
 
 from ._loader import ffi
 from ._loader import lib as _lib
+from .random import Random
 from .util import Buffer, new_aligned_struct, nonce_increment, wipe
 
 NAME = "AEGIS-128X2"  #: Algorithm display name
@@ -19,25 +22,41 @@ ALIGNMENT = 64  #: Required alignment for internal structures
 RATE = 64  #: Byte chunk size in internal processing
 
 
+_random_tls = threading.local()
+
+
+def random() -> Random:
+    """Return this module's Random instance (per-thread singleton).
+
+    Each thread gets its own generator on first use, seeded once from the
+    OS, then deterministic: successive calls never repeat an output block.
+    No locking is involved. See aeg.random.Random for the API.
+    """
+    rng = getattr(_random_tls, "rng", None)
+    if rng is None:
+        rng = _random_tls.rng = Random(sys.modules[__name__])
+    return rng
+
+
 def random_key() -> bytearray:
     """
-    Generate a secret key using cryptographically secure random bytes.
+    Generate a secret key using the module's CSPRNG (see random()).
 
     It is recommended to wipe() the key after no longer needed.
     """
-    return bytearray(secrets.token_bytes(KEYBYTES))
+    return random().bytes(KEYBYTES)
 
 
 def random_nonce() -> bytearray:
     """
-    Generate a public nonce using cryptographically secure random bytes.
+    Generate a public nonce using the module's CSPRNG (see random()).
 
     Nonces (a number used once) are public data that may be sent together
     with the ciphertext, but they need to be unique for each use.
 
     See also: nonce_increment() can be used to derive sequential nonces.
     """
-    return bytearray(secrets.token_bytes(NONCEBYTES))
+    return random().bytes(NONCEBYTES)
 
 
 def _ptr(buf):
@@ -360,6 +379,55 @@ def stream(
     return out if into is None else memoryview(out)[: length or memoryview(out).nbytes]  # type: ignore
 
 
+def stream_xor(
+    key: Buffer,
+    nonce: Buffer,
+    data: Buffer,
+    *,
+    into: Buffer | None = None,
+) -> bytearray | memoryview:
+    """XOR data with the keystream from stream(); encrypts and decrypts.
+
+    Unauthenticated: only use when the data is authenticated some other
+    way. Never reuse a nonce with the same key.
+
+    Args:
+        key: Secret key (generate with random_key()).
+        nonce: Public nonce (generate with random_nonce()).
+        data: Input bytes to XOR.
+        into: Buffer to write the output into (default: bytearray created).
+
+    Returns:
+        Output as bytearray if into not provided, memoryview of into otherwise.
+
+    Raises:
+        TypeError: If lengths are invalid.
+    """
+    key = memoryview(key)
+    nonce = memoryview(nonce)
+    data = memoryview(data)
+    if into is not None:
+        into = memoryview(into)
+    if key.nbytes != KEYBYTES:
+        raise TypeError(f"key length must be {KEYBYTES}")
+    if nonce.nbytes != NONCEBYTES:
+        raise TypeError(f"nonce length must be {NONCEBYTES}")
+    if into is None:
+        out = bytearray(data.nbytes)
+    else:
+        if into.nbytes < data.nbytes:
+            raise TypeError("into length must be at least data.nbytes")
+        out = into
+    _lib.aegis128x2_stream_xor(
+        ffi.from_buffer(out),
+        _ptr(data),
+        data.nbytes,
+        _ptr(nonce),
+        _ptr(key),
+    )
+    return out if into is None else memoryview(out)[: data.nbytes]  # type: ignore
+
+
 def encrypt_unauthenticated(
     key: Buffer,
     nonce: Buffer,
@@ -369,6 +437,10 @@ def encrypt_unauthenticated(
 ) -> bytearray | memoryview:
     """Encrypt message without authentication (for testing/debugging).
 
+    .. deprecated::
+        Use stream_xor() in new code. Its output is different, so keep
+        this only for data in the old format.
+
     Args:
         key: Secret key (generate with random_key()).
         nonce: Public nonce (generate with random_nonce()).
@@ -381,6 +453,12 @@ def encrypt_unauthenticated(
     Raises:
         TypeError: If lengths are invalid.
     """
+    warnings.warn(
+        "encrypt_unauthenticated is deprecated; use stream_xor "
+        "(its output differs, so keep this only for existing data)",
+        DeprecationWarning,
+        stacklevel=2,
+    )
     key = memoryview(key)
     nonce = memoryview(nonce)
     message = memoryview(message)
@@ -415,6 +493,10 @@ def decrypt_unauthenticated(
 ) -> bytearray | memoryview:
     """Decrypt ciphertext without authentication (for testing/debugging).
 
+    .. deprecated::
+        Use stream_xor() in new code. Its output is different, so keep
+        this only for data in the old format.
+
     Args:
         key: Secret key (same key used during encryption).
         nonce: Public nonce (same nonce used during encryption).
@@ -427,6 +509,12 @@ def decrypt_unauthenticated(
     Raises:
         TypeError: If lengths are invalid.
     """
+    warnings.warn(
+        "decrypt_unauthenticated is deprecated; use stream_xor "
+        "(its output differs, so keep this only for existing data)",
+        DeprecationWarning,
+        stacklevel=2,
+    )
     key = memoryview(key)
     nonce = memoryview(nonce)
     ct = memoryview(ct)
@@ -708,24 +796,17 @@ class Encryptor:
             raise TypeError(
                 "into length must be >= expected output size for this update"
             )
-        written = ffi.new("size_t *")
         rc = _lib.aegis128x2_state_encrypt_update(
             self._state.ptr,
             ffi.from_buffer(out_mv),
-            out_mv.nbytes,
-            written,
             _ptr(message),
             message.nbytes,
         )
         if rc != 0:
             err_num = ffi.errno
             err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
-            raise RuntimeError(
-                f"state encrypt update failed: {err_name} written {written[0]}"
-            )
-        w = int(written[0])
-        assert w == expected_out
-        return out if into is None else memoryview(out)[:w]  # type: ignore
+            raise RuntimeError(f"state encrypt update failed: {err_name}")
+        return out if into is None else memoryview(out)[:expected_out]  # type: ignore
 
     def final(self, into: Buffer | None = None) -> bytearray | memoryview:
         """Finalize encryption and return the authentication tag.
@@ -746,24 +827,17 @@ class Encryptor:
         if into is not None:
             into = memoryview(into)
         out = into if into is not None else bytearray(maclen)
-        written = ffi.new("size_t *")
         rc = _lib.aegis128x2_state_encrypt_final(
             self._state.ptr,
             ffi.from_buffer(out),
-            memoryview(out).nbytes,
-            written,
             maclen,
         )
         if rc != 0:
             err_num = ffi.errno
             err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
             raise RuntimeError(f"state encrypt final failed: {err_name}")
-        w = int(written[0])
-        if into is None:
-            # Only the tag bytes are returned when we allocate the buffer
-            assert w == maclen
         self._state = None
-        return out if into is None else memoryview(out)[:w]  # type: ignore
+        return out if into is None else memoryview(out)[:maclen]  # type: ignore
 
 
 class Decryptor:
@@ -837,12 +911,9 @@ class Decryptor:
         out_mv = memoryview(out)
         if out_mv.nbytes < expected_out:
             raise TypeError("into length must be >= required capacity for this update")
-        written = ffi.new("size_t *")
-        rc = _lib.aegis128x2_state_decrypt_detached_update(
+        rc = _lib.aegis128x2_state_decrypt_update(
             self._state.ptr,
             ffi.from_buffer(out_mv),
-            out_mv.nbytes,
-            written,
             _ptr(ct),
             ct.nbytes,
         )
@@ -850,11 +921,7 @@ class Decryptor:
             err_num = ffi.errno
             err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
             raise RuntimeError(f"state decrypt update failed: {err_name}")
-        w = int(written[0])
-        assert w == expected_out, (
-            f"got {w}, expected {expected_out}, ct.nbytes={ct.nbytes}"
-        )
-        return out if into is None else memoryview(out)[:w]  # type: ignore
+        return out if into is None else memoryview(out)[:expected_out]  # type: ignore
 
     def final(self, mac: Buffer) -> None:
         """Finalize decryption by verifying the MAC tag.
@@ -873,9 +940,7 @@ class Decryptor:
         mac = memoryview(mac)
         if mac.nbytes != maclen:
             raise TypeError(f"mac length must be {maclen}")
-        rc = _lib.aegis128x2_state_decrypt_detached_final(
-            self._state.ptr, ffi.NULL, 0, ffi.NULL, _ptr(mac), maclen
-        )
+        rc = _lib.aegis128x2_state_decrypt_final(self._state.ptr, _ptr(mac), maclen)
         if rc != 0:
             raise ValueError("authentication failed")
         self._state = None
@@ -891,6 +956,220 @@ def new_mac_state():
     return new_aligned_struct("aegis128x2_mac_state", ALIGNMENT)
 
 
+RAF_ALG_ID = 2  #: Algorithm identifier in RAF file headers
+
+
+def _raise_errno(operation: str) -> None:
+    err_num = ffi.errno
+    err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
+    raise RuntimeError(f"{operation} failed: {err_name}")
+
+
+def new_raf_ctx():
+    """Allocate and return a new aegis128x2_raf_ctx* with proper alignment."""
+    return new_aligned_struct("aegis128x2_raf_ctx", ALIGNMENT)
+
+
+def raf_scratch_size(chunk_size: int) -> int:
+    """Return the scratch buffer size required for the given chunk_size."""
+    return _lib.aegis128x2_raf_scratch_size(chunk_size)
+
+
+def raf_create(key: Buffer, ctx, io, rng, config) -> None:
+    """Create a new encrypted file, writing the header through io.
+
+    Args:
+        key: Master key (KEYBYTES bytes).
+        ctx: Context from new_raf_ctx().
+        io: aegis_raf_io* cdata (see aeg.raf).
+        rng: aegis_raf_rng* cdata (see aeg.random).
+        config: aegis_raf_config* cdata (see aeg.raf).
+
+    Raises:
+        TypeError: If key length is invalid.
+        FileExistsError: If the file exists and AEGIS_RAF_TRUNCATE was not set.
+        RuntimeError: If creation fails.
+    """
+    key = memoryview(key)
+    if key.nbytes != KEYBYTES:
+        raise TypeError(f"key length must be {KEYBYTES}")
+    rc = _lib.aegis128x2_raf_create(ctx.ptr, io, rng, config, _ptr(key))
+    if rc != 0:
+        if ffi.errno == errno.EEXIST:
+            raise FileExistsError("file exists (use truncate to overwrite)")
+        _raise_errno("raf create")
+
+
+def raf_open(key: Buffer, ctx, io, rng, config) -> None:
+    """Open an existing encrypted file, verifying the header MAC.
+
+    Args:
+        key: Master key (KEYBYTES bytes).
+        ctx: Context from new_raf_ctx().
+        io: aegis_raf_io* cdata (see aeg.raf).
+        rng: aegis_raf_rng* cdata (see aeg.random).
+        config: aegis_raf_config* cdata (see aeg.raf).
+
+    Raises:
+        TypeError: If key length is invalid.
+        FileNotFoundError: If the file does not exist.
+        ValueError: If the header is invalid or authentication fails (wrong key).
+        RuntimeError: If opening fails.
+    """
+    key = memoryview(key)
+    if key.nbytes != KEYBYTES:
+        raise TypeError(f"key length must be {KEYBYTES}")
+    rc = _lib.aegis128x2_raf_open(ctx.ptr, io, rng, config, _ptr(key))
+    if rc != 0:
+        err_num = ffi.errno
+        if err_num == errno.ENOENT:
+            raise FileNotFoundError("file not found")
+        if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+            raise ValueError("authentication failed")
+        _raise_errno("raf open")
+
+
+def raf_read(
+    ctx, length: int, offset: int, *, into: Buffer | None = None
+) -> bytearray | memoryview:
+    """Read and decrypt up to length bytes at offset.
+
+    Returns:
+        The decrypted bytes, trimmed to the number actually read
+        (fewer at EOF), as bytearray if into not provided,
+        memoryview of into otherwise.
+
+    Raises:
+        TypeError: If into is smaller than length.
+        ValueError: If authentication fails (corruption or tampering).
+        RuntimeError: If the read fails.
+    """
+    if into is None:
+        out = bytearray(length)
+    else:
+        into = memoryview(into)
+        if into.nbytes < length:
+            raise TypeError("into length must be at least length")
+        out = into
+    bytes_read = ffi.new("size_t*")
+    rc = _lib.aegis128x2_raf_read(
+        ctx.ptr, ffi.from_buffer(out), bytes_read, length, offset
+    )
+    if rc != 0:
+        err_num = ffi.errno
+        if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+            raise ValueError("authentication failed")
+        _raise_errno("raf read")
+    n = bytes_read[0]
+    return out[:n] if into is None else memoryview(out)[:n]  # type: ignore
+
+
+def raf_write(ctx, data: Buffer, offset: int) -> int:
+    """Encrypt and write data at offset, extending the file as needed.
+
+    Returns:
+        Number of bytes written.
+
+    Raises:
+        RuntimeError: If the write fails (errno name included).
+    """
+    data = memoryview(data)
+    bytes_written = ffi.new("size_t*")
+    rc = _lib.aegis128x2_raf_write(
+        ctx.ptr, bytes_written, _ptr(data), data.nbytes, offset
+    )
+    if rc != 0:
+        _raise_errno("raf write")
+    return bytes_written[0]
+
+
+def raf_truncate(ctx, size: int) -> None:
+    """Resize the file to size bytes (zero-filled when growing).
+
+    Raises:
+        RuntimeError: If the truncate fails.
+    """
+    rc = _lib.aegis128x2_raf_truncate(ctx.ptr, size)
+    if rc != 0:
+        _raise_errno("raf truncate")
+
+
+def raf_size(ctx) -> int:
+    """Return the logical plaintext file size."""
+    size = ffi.new("uint64_t*")
+    rc = _lib.aegis128x2_raf_get_size(ctx.ptr, size)
+    if rc != 0:
+        _raise_errno("raf get_size")
+    return size[0]
+
+
+def raf_sync(ctx) -> None:
+    """Flush writes to the backing store.
+
+    Raises:
+        RuntimeError: If the sync fails.
+    """
+    rc = _lib.aegis128x2_raf_sync(ctx.ptr)
+    if rc != 0:
+        _raise_errno("raf sync")
+
+
+def raf_close(ctx) -> None:
+    """Close the context, syncing and zeroizing key material."""
+    _lib.aegis128x2_raf_close(ctx.ptr)
+
+
+def raf_merkle_rebuild(ctx) -> None:
+    """Rebuild the Merkle tree by reading and re-hashing every chunk.
+
+    Raises:
+        RuntimeError: If Merkle is not enabled or rebuilding fails.
+        ValueError: If a chunk fails authentication.
+    """
+    rc = _lib.aegis128x2_raf_merkle_rebuild(ctx.ptr)
+    if rc != 0:
+        err_num = ffi.errno
+        if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+            raise ValueError("authentication failed")
+        _raise_errno("raf merkle_rebuild")
+
+
+def raf_merkle_verify(ctx) -> int | None:
+    """Verify every chunk's hash against the Merkle tree.
+
+    Returns:
+        None if all chunks match, otherwise the index of the first
+        corrupted chunk.
+
+    Raises:
+        RuntimeError: If Merkle is not enabled or verification fails.
+    """
+    corrupted = ffi.new("uint64_t*")
+    rc = _lib.aegis128x2_raf_merkle_verify(ctx.ptr, corrupted)
+    if rc == 0:
+        return None
+    err_num = ffi.errno
+    if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+        return corrupted[0]
+    _raise_errno("raf merkle_verify")
+
+
+def raf_merkle_commitment(ctx, hash_len: int) -> bytes:
+    """Compute the Merkle root commitment hash.
+
+    Returns:
+        The commitment hash as bytes of length hash_len.
+
+    Raises:
+        RuntimeError: If Merkle is not enabled or the call fails.
+    """
+    out = ffi.new(f"uint8_t[{hash_len}]")
+    rc = _lib.aegis128x2_raf_merkle_commitment(ctx.ptr, out, hash_len)
+    if rc != 0:
+        _raise_errno("raf merkle_commitment")
+    return bytes(ffi.buffer(out, hash_len))
+
+
 __all__ = [
     # constants
     "NAME",
@@ -900,7 +1179,9 @@ __all__ = [
     "MACBYTES_LONG",
     "ALIGNMENT",
     "RATE",
+    "RAF_ALG_ID",
     # utility functions
+    "random",
     "random_key",
     "random_nonce",
     "nonce_increment",
@@ -911,6 +1192,7 @@ __all__ = [
     "encrypt",
     "decrypt",
     "stream",
+    "stream_xor",
     "encrypt_unauthenticated",
     "decrypt_unauthenticated",
     "mac",
@@ -918,4 +1200,18 @@ __all__ = [
     "Encryptor",
     "Decryptor",
     "Mac",
+    # random-access encrypted files (thin wrappers; see aeg.raf for the high-level API)
+    "new_raf_ctx",
+    "raf_scratch_size",
+    "raf_create",
+    "raf_open",
+    "raf_read",
+    "raf_write",
+    "raf_truncate",
+    "raf_size",
+    "raf_sync",
+    "raf_close",
+    "raf_merkle_rebuild",
+    "raf_merkle_verify",
+    "raf_merkle_commitment",
 ]

src/aeg/aegis128x4.py

@@ -3,11 +3,14 @@
 # DO NOT EDIT OTHER ALGORITHM FILES MANUALLY!
 
 import errno
-import secrets
+import sys
+import threading
+import warnings
 from typing import Literal
 
 from ._loader import ffi
 from ._loader import lib as _lib
+from .random import Random
 from .util import Buffer, new_aligned_struct, nonce_increment, wipe
 
 NAME = "AEGIS-128X4"  #: Algorithm display name
@@ -19,25 +22,41 @@ ALIGNMENT = 64  #: Required alignment for internal structures
 RATE = 64  #: Byte chunk size in internal processing
 
 
+_random_tls = threading.local()
+
+
+def random() -> Random:
+    """Return this module's Random instance (per-thread singleton).
+
+    Each thread gets its own generator on first use, seeded once from the
+    OS, then deterministic: successive calls never repeat an output block.
+    No locking is involved. See aeg.random.Random for the API.
+    """
+    rng = getattr(_random_tls, "rng", None)
+    if rng is None:
+        rng = _random_tls.rng = Random(sys.modules[__name__])
+    return rng
+
+
 def random_key() -> bytearray:
     """
-    Generate a secret key using cryptographically secure random bytes.
+    Generate a secret key using the module's CSPRNG (see random()).
 
     It is recommended to wipe() the key after no longer needed.
     """
-    return bytearray(secrets.token_bytes(KEYBYTES))
+    return random().bytes(KEYBYTES)
 
 
 def random_nonce() -> bytearray:
     """
-    Generate a public nonce using cryptographically secure random bytes.
+    Generate a public nonce using the module's CSPRNG (see random()).
 
     Nonces (a number used once) are public data that may be sent together
     with the ciphertext, but they need to be unique for each use.
 
     See also: nonce_increment() can be used to derive sequential nonces.
     """
-    return bytearray(secrets.token_bytes(NONCEBYTES))
+    return random().bytes(NONCEBYTES)
 
 
 def _ptr(buf):
@@ -360,6 +379,55 @@ def stream(
     return out if into is None else memoryview(out)[: length or memoryview(out).nbytes]  # type: ignore
 
 
+def stream_xor(
+    key: Buffer,
+    nonce: Buffer,
+    data: Buffer,
+    *,
+    into: Buffer | None = None,
+) -> bytearray | memoryview:
+    """XOR data with the keystream from stream(); encrypts and decrypts.
+
+    Unauthenticated: only use when the data is authenticated some other
+    way. Never reuse a nonce with the same key.
+
+    Args:
+        key: Secret key (generate with random_key()).
+        nonce: Public nonce (generate with random_nonce()).
+        data: Input bytes to XOR.
+        into: Buffer to write the output into (default: bytearray created).
+
+    Returns:
+        Output as bytearray if into not provided, memoryview of into otherwise.
+
+    Raises:
+        TypeError: If lengths are invalid.
+    """
+    key = memoryview(key)
+    nonce = memoryview(nonce)
+    data = memoryview(data)
+    if into is not None:
+        into = memoryview(into)
+    if key.nbytes != KEYBYTES:
+        raise TypeError(f"key length must be {KEYBYTES}")
+    if nonce.nbytes != NONCEBYTES:
+        raise TypeError(f"nonce length must be {NONCEBYTES}")
+    if into is None:
+        out = bytearray(data.nbytes)
+    else:
+        if into.nbytes < data.nbytes:
+            raise TypeError("into length must be at least data.nbytes")
+        out = into
+    _lib.aegis128x4_stream_xor(
+        ffi.from_buffer(out),
+        _ptr(data),
+        data.nbytes,
+        _ptr(nonce),
+        _ptr(key),
+    )
+    return out if into is None else memoryview(out)[: data.nbytes]  # type: ignore
+
+
 def encrypt_unauthenticated(
     key: Buffer,
     nonce: Buffer,
@@ -369,6 +437,10 @@ def encrypt_unauthenticated(
 ) -> bytearray | memoryview:
     """Encrypt message without authentication (for testing/debugging).
 
+    .. deprecated::
+        Use stream_xor() in new code. Its output is different, so keep
+        this only for data in the old format.
+
     Args:
         key: Secret key (generate with random_key()).
         nonce: Public nonce (generate with random_nonce()).
@@ -381,6 +453,12 @@ def encrypt_unauthenticated(
     Raises:
         TypeError: If lengths are invalid.
     """
+    warnings.warn(
+        "encrypt_unauthenticated is deprecated; use stream_xor "
+        "(its output differs, so keep this only for existing data)",
+        DeprecationWarning,
+        stacklevel=2,
+    )
     key = memoryview(key)
     nonce = memoryview(nonce)
     message = memoryview(message)
@@ -415,6 +493,10 @@ def decrypt_unauthenticated(
 ) -> bytearray | memoryview:
     """Decrypt ciphertext without authentication (for testing/debugging).
 
+    .. deprecated::
+        Use stream_xor() in new code. Its output is different, so keep
+        this only for data in the old format.
+
     Args:
         key: Secret key (same key used during encryption).
         nonce: Public nonce (same nonce used during encryption).
@@ -427,6 +509,12 @@ def decrypt_unauthenticated(
     Raises:
         TypeError: If lengths are invalid.
     """
+    warnings.warn(
+        "decrypt_unauthenticated is deprecated; use stream_xor "
+        "(its output differs, so keep this only for existing data)",
+        DeprecationWarning,
+        stacklevel=2,
+    )
     key = memoryview(key)
     nonce = memoryview(nonce)
     ct = memoryview(ct)
@@ -708,24 +796,17 @@ class Encryptor:
             raise TypeError(
                 "into length must be >= expected output size for this update"
             )
-        written = ffi.new("size_t *")
         rc = _lib.aegis128x4_state_encrypt_update(
             self._state.ptr,
             ffi.from_buffer(out_mv),
-            out_mv.nbytes,
-            written,
             _ptr(message),
             message.nbytes,
         )
         if rc != 0:
             err_num = ffi.errno
             err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
-            raise RuntimeError(
-                f"state encrypt update failed: {err_name} written {written[0]}"
-            )
-        w = int(written[0])
-        assert w == expected_out
-        return out if into is None else memoryview(out)[:w]  # type: ignore
+            raise RuntimeError(f"state encrypt update failed: {err_name}")
+        return out if into is None else memoryview(out)[:expected_out]  # type: ignore
 
     def final(self, into: Buffer | None = None) -> bytearray | memoryview:
         """Finalize encryption and return the authentication tag.
@@ -746,24 +827,17 @@ class Encryptor:
         if into is not None:
             into = memoryview(into)
         out = into if into is not None else bytearray(maclen)
-        written = ffi.new("size_t *")
         rc = _lib.aegis128x4_state_encrypt_final(
             self._state.ptr,
             ffi.from_buffer(out),
-            memoryview(out).nbytes,
-            written,
             maclen,
         )
         if rc != 0:
             err_num = ffi.errno
             err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
             raise RuntimeError(f"state encrypt final failed: {err_name}")
-        w = int(written[0])
-        if into is None:
-            # Only the tag bytes are returned when we allocate the buffer
-            assert w == maclen
         self._state = None
-        return out if into is None else memoryview(out)[:w]  # type: ignore
+        return out if into is None else memoryview(out)[:maclen]  # type: ignore
 
 
 class Decryptor:
@@ -837,12 +911,9 @@ class Decryptor:
         out_mv = memoryview(out)
         if out_mv.nbytes < expected_out:
             raise TypeError("into length must be >= required capacity for this update")
-        written = ffi.new("size_t *")
-        rc = _lib.aegis128x4_state_decrypt_detached_update(
+        rc = _lib.aegis128x4_state_decrypt_update(
             self._state.ptr,
             ffi.from_buffer(out_mv),
-            out_mv.nbytes,
-            written,
             _ptr(ct),
             ct.nbytes,
         )
@@ -850,11 +921,7 @@ class Decryptor:
             err_num = ffi.errno
             err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
             raise RuntimeError(f"state decrypt update failed: {err_name}")
-        w = int(written[0])
-        assert w == expected_out, (
-            f"got {w}, expected {expected_out}, ct.nbytes={ct.nbytes}"
-        )
-        return out if into is None else memoryview(out)[:w]  # type: ignore
+        return out if into is None else memoryview(out)[:expected_out]  # type: ignore
 
     def final(self, mac: Buffer) -> None:
         """Finalize decryption by verifying the MAC tag.
@@ -873,9 +940,7 @@ class Decryptor:
         mac = memoryview(mac)
         if mac.nbytes != maclen:
             raise TypeError(f"mac length must be {maclen}")
-        rc = _lib.aegis128x4_state_decrypt_detached_final(
-            self._state.ptr, ffi.NULL, 0, ffi.NULL, _ptr(mac), maclen
-        )
+        rc = _lib.aegis128x4_state_decrypt_final(self._state.ptr, _ptr(mac), maclen)
         if rc != 0:
             raise ValueError("authentication failed")
         self._state = None
@@ -891,6 +956,220 @@ def new_mac_state():
     return new_aligned_struct("aegis128x4_mac_state", ALIGNMENT)
 
 
+RAF_ALG_ID = 3  #: Algorithm identifier in RAF file headers
+
+
+def _raise_errno(operation: str) -> None:
+    err_num = ffi.errno
+    err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
+    raise RuntimeError(f"{operation} failed: {err_name}")
+
+
+def new_raf_ctx():
+    """Allocate and return a new aegis128x4_raf_ctx* with proper alignment."""
+    return new_aligned_struct("aegis128x4_raf_ctx", ALIGNMENT)
+
+
+def raf_scratch_size(chunk_size: int) -> int:
+    """Return the scratch buffer size required for the given chunk_size."""
+    return _lib.aegis128x4_raf_scratch_size(chunk_size)
+
+
+def raf_create(key: Buffer, ctx, io, rng, config) -> None:
+    """Create a new encrypted file, writing the header through io.
+
+    Args:
+        key: Master key (KEYBYTES bytes).
+        ctx: Context from new_raf_ctx().
+        io: aegis_raf_io* cdata (see aeg.raf).
+        rng: aegis_raf_rng* cdata (see aeg.random).
+        config: aegis_raf_config* cdata (see aeg.raf).
+
+    Raises:
+        TypeError: If key length is invalid.
+        FileExistsError: If the file exists and AEGIS_RAF_TRUNCATE was not set.
+        RuntimeError: If creation fails.
+    """
+    key = memoryview(key)
+    if key.nbytes != KEYBYTES:
+        raise TypeError(f"key length must be {KEYBYTES}")
+    rc = _lib.aegis128x4_raf_create(ctx.ptr, io, rng, config, _ptr(key))
+    if rc != 0:
+        if ffi.errno == errno.EEXIST:
+            raise FileExistsError("file exists (use truncate to overwrite)")
+        _raise_errno("raf create")
+
+
+def raf_open(key: Buffer, ctx, io, rng, config) -> None:
+    """Open an existing encrypted file, verifying the header MAC.
+
+    Args:
+        key: Master key (KEYBYTES bytes).
+        ctx: Context from new_raf_ctx().
+        io: aegis_raf_io* cdata (see aeg.raf).
+        rng: aegis_raf_rng* cdata (see aeg.random).
+        config: aegis_raf_config* cdata (see aeg.raf).
+
+    Raises:
+        TypeError: If key length is invalid.
+        FileNotFoundError: If the file does not exist.
+        ValueError: If the header is invalid or authentication fails (wrong key).
+        RuntimeError: If opening fails.
+    """
+    key = memoryview(key)
+    if key.nbytes != KEYBYTES:
+        raise TypeError(f"key length must be {KEYBYTES}")
+    rc = _lib.aegis128x4_raf_open(ctx.ptr, io, rng, config, _ptr(key))
+    if rc != 0:
+        err_num = ffi.errno
+        if err_num == errno.ENOENT:
+            raise FileNotFoundError("file not found")
+        if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+            raise ValueError("authentication failed")
+        _raise_errno("raf open")
+
+
+def raf_read(
+    ctx, length: int, offset: int, *, into: Buffer | None = None
+) -> bytearray | memoryview:
+    """Read and decrypt up to length bytes at offset.
+
+    Returns:
+        The decrypted bytes, trimmed to the number actually read
+        (fewer at EOF), as bytearray if into not provided,
+        memoryview of into otherwise.
+
+    Raises:
+        TypeError: If into is smaller than length.
+        ValueError: If authentication fails (corruption or tampering).
+        RuntimeError: If the read fails.
+    """
+    if into is None:
+        out = bytearray(length)
+    else:
+        into = memoryview(into)
+        if into.nbytes < length:
+            raise TypeError("into length must be at least length")
+        out = into
+    bytes_read = ffi.new("size_t*")
+    rc = _lib.aegis128x4_raf_read(
+        ctx.ptr, ffi.from_buffer(out), bytes_read, length, offset
+    )
+    if rc != 0:
+        err_num = ffi.errno
+        if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+            raise ValueError("authentication failed")
+        _raise_errno("raf read")
+    n = bytes_read[0]
+    return out[:n] if into is None else memoryview(out)[:n]  # type: ignore
+
+
+def raf_write(ctx, data: Buffer, offset: int) -> int:
+    """Encrypt and write data at offset, extending the file as needed.
+
+    Returns:
+        Number of bytes written.
+
+    Raises:
+        RuntimeError: If the write fails (errno name included).
+    """
+    data = memoryview(data)
+    bytes_written = ffi.new("size_t*")
+    rc = _lib.aegis128x4_raf_write(
+        ctx.ptr, bytes_written, _ptr(data), data.nbytes, offset
+    )
+    if rc != 0:
+        _raise_errno("raf write")
+    return bytes_written[0]
+
+
+def raf_truncate(ctx, size: int) -> None:
+    """Resize the file to size bytes (zero-filled when growing).
+
+    Raises:
+        RuntimeError: If the truncate fails.
+    """
+    rc = _lib.aegis128x4_raf_truncate(ctx.ptr, size)
+    if rc != 0:
+        _raise_errno("raf truncate")
+
+
+def raf_size(ctx) -> int:
+    """Return the logical plaintext file size."""
+    size = ffi.new("uint64_t*")
+    rc = _lib.aegis128x4_raf_get_size(ctx.ptr, size)
+    if rc != 0:
+        _raise_errno("raf get_size")
+    return size[0]
+
+
+def raf_sync(ctx) -> None:
+    """Flush writes to the backing store.
+
+    Raises:
+        RuntimeError: If the sync fails.
+    """
+    rc = _lib.aegis128x4_raf_sync(ctx.ptr)
+    if rc != 0:
+        _raise_errno("raf sync")
+
+
+def raf_close(ctx) -> None:
+    """Close the context, syncing and zeroizing key material."""
+    _lib.aegis128x4_raf_close(ctx.ptr)
+
+
+def raf_merkle_rebuild(ctx) -> None:
+    """Rebuild the Merkle tree by reading and re-hashing every chunk.
+
+    Raises:
+        RuntimeError: If Merkle is not enabled or rebuilding fails.
+        ValueError: If a chunk fails authentication.
+    """
+    rc = _lib.aegis128x4_raf_merkle_rebuild(ctx.ptr)
+    if rc != 0:
+        err_num = ffi.errno
+        if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+            raise ValueError("authentication failed")
+        _raise_errno("raf merkle_rebuild")
+
+
+def raf_merkle_verify(ctx) -> int | None:
+    """Verify every chunk's hash against the Merkle tree.
+
+    Returns:
+        None if all chunks match, otherwise the index of the first
+        corrupted chunk.
+
+    Raises:
+        RuntimeError: If Merkle is not enabled or verification fails.
+    """
+    corrupted = ffi.new("uint64_t*")
+    rc = _lib.aegis128x4_raf_merkle_verify(ctx.ptr, corrupted)
+    if rc == 0:
+        return None
+    err_num = ffi.errno
+    if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+        return corrupted[0]
+    _raise_errno("raf merkle_verify")
+
+
+def raf_merkle_commitment(ctx, hash_len: int) -> bytes:
+    """Compute the Merkle root commitment hash.
+
+    Returns:
+        The commitment hash as bytes of length hash_len.
+
+    Raises:
+        RuntimeError: If Merkle is not enabled or the call fails.
+    """
+    out = ffi.new(f"uint8_t[{hash_len}]")
+    rc = _lib.aegis128x4_raf_merkle_commitment(ctx.ptr, out, hash_len)
+    if rc != 0:
+        _raise_errno("raf merkle_commitment")
+    return bytes(ffi.buffer(out, hash_len))
+
+
 __all__ = [
     # constants
     "NAME",
@@ -900,7 +1179,9 @@ __all__ = [
     "MACBYTES_LONG",
     "ALIGNMENT",
     "RATE",
+    "RAF_ALG_ID",
     # utility functions
+    "random",
     "random_key",
     "random_nonce",
     "nonce_increment",
@@ -911,6 +1192,7 @@ __all__ = [
     "encrypt",
     "decrypt",
     "stream",
+    "stream_xor",
     "encrypt_unauthenticated",
     "decrypt_unauthenticated",
     "mac",
@@ -918,4 +1200,18 @@ __all__ = [
     "Encryptor",
     "Decryptor",
     "Mac",
+    # random-access encrypted files (thin wrappers; see aeg.raf for the high-level API)
+    "new_raf_ctx",
+    "raf_scratch_size",
+    "raf_create",
+    "raf_open",
+    "raf_read",
+    "raf_write",
+    "raf_truncate",
+    "raf_size",
+    "raf_sync",
+    "raf_close",
+    "raf_merkle_rebuild",
+    "raf_merkle_verify",
+    "raf_merkle_commitment",
 ]

src/aeg/aegis256.py

@@ -3,11 +3,14 @@
 # DO NOT EDIT OTHER ALGORITHM FILES MANUALLY!
 
 import errno
-import secrets
+import sys
+import threading
+import warnings
 from typing import Literal
 
 from ._loader import ffi
 from ._loader import lib as _lib
+from .random import Random
 from .util import Buffer, new_aligned_struct, nonce_increment, wipe
 
 NAME = "AEGIS-256"  #: Algorithm display name
@@ -19,25 +22,41 @@ ALIGNMENT = 64  #: Required alignment for internal structures
 RATE = 64  #: Byte chunk size in internal processing
 
 
+_random_tls = threading.local()
+
+
+def random() -> Random:
+    """Return this module's Random instance (per-thread singleton).
+
+    Each thread gets its own generator on first use, seeded once from the
+    OS, then deterministic: successive calls never repeat an output block.
+    No locking is involved. See aeg.random.Random for the API.
+    """
+    rng = getattr(_random_tls, "rng", None)
+    if rng is None:
+        rng = _random_tls.rng = Random(sys.modules[__name__])
+    return rng
+
+
 def random_key() -> bytearray:
     """
-    Generate a secret key using cryptographically secure random bytes.
+    Generate a secret key using the module's CSPRNG (see random()).
 
     It is recommended to wipe() the key after no longer needed.
     """
-    return bytearray(secrets.token_bytes(KEYBYTES))
+    return random().bytes(KEYBYTES)
 
 
 def random_nonce() -> bytearray:
     """
-    Generate a public nonce using cryptographically secure random bytes.
+    Generate a public nonce using the module's CSPRNG (see random()).
 
     Nonces (a number used once) are public data that may be sent together
     with the ciphertext, but they need to be unique for each use.
 
     See also: nonce_increment() can be used to derive sequential nonces.
     """
-    return bytearray(secrets.token_bytes(NONCEBYTES))
+    return random().bytes(NONCEBYTES)
 
 
 def _ptr(buf):
@@ -360,6 +379,55 @@ def stream(
     return out if into is None else memoryview(out)[: length or memoryview(out).nbytes]  # type: ignore
 
 
+def stream_xor(
+    key: Buffer,
+    nonce: Buffer,
+    data: Buffer,
+    *,
+    into: Buffer | None = None,
+) -> bytearray | memoryview:
+    """XOR data with the keystream from stream(); encrypts and decrypts.
+
+    Unauthenticated: only use when the data is authenticated some other
+    way. Never reuse a nonce with the same key.
+
+    Args:
+        key: Secret key (generate with random_key()).
+        nonce: Public nonce (generate with random_nonce()).
+        data: Input bytes to XOR.
+        into: Buffer to write the output into (default: bytearray created).
+
+    Returns:
+        Output as bytearray if into not provided, memoryview of into otherwise.
+
+    Raises:
+        TypeError: If lengths are invalid.
+    """
+    key = memoryview(key)
+    nonce = memoryview(nonce)
+    data = memoryview(data)
+    if into is not None:
+        into = memoryview(into)
+    if key.nbytes != KEYBYTES:
+        raise TypeError(f"key length must be {KEYBYTES}")
+    if nonce.nbytes != NONCEBYTES:
+        raise TypeError(f"nonce length must be {NONCEBYTES}")
+    if into is None:
+        out = bytearray(data.nbytes)
+    else:
+        if into.nbytes < data.nbytes:
+            raise TypeError("into length must be at least data.nbytes")
+        out = into
+    _lib.aegis256_stream_xor(
+        ffi.from_buffer(out),
+        _ptr(data),
+        data.nbytes,
+        _ptr(nonce),
+        _ptr(key),
+    )
+    return out if into is None else memoryview(out)[: data.nbytes]  # type: ignore
+
+
 def encrypt_unauthenticated(
     key: Buffer,
     nonce: Buffer,
@@ -369,6 +437,10 @@ def encrypt_unauthenticated(
 ) -> bytearray | memoryview:
     """Encrypt message without authentication (for testing/debugging).
 
+    .. deprecated::
+        Use stream_xor() in new code. Its output is different, so keep
+        this only for data in the old format.
+
     Args:
         key: Secret key (generate with random_key()).
         nonce: Public nonce (generate with random_nonce()).
@@ -381,6 +453,12 @@ def encrypt_unauthenticated(
     Raises:
         TypeError: If lengths are invalid.
     """
+    warnings.warn(
+        "encrypt_unauthenticated is deprecated; use stream_xor "
+        "(its output differs, so keep this only for existing data)",
+        DeprecationWarning,
+        stacklevel=2,
+    )
     key = memoryview(key)
     nonce = memoryview(nonce)
     message = memoryview(message)
@@ -415,6 +493,10 @@ def decrypt_unauthenticated(
 ) -> bytearray | memoryview:
     """Decrypt ciphertext without authentication (for testing/debugging).
 
+    .. deprecated::
+        Use stream_xor() in new code. Its output is different, so keep
+        this only for data in the old format.
+
     Args:
         key: Secret key (same key used during encryption).
         nonce: Public nonce (same nonce used during encryption).
@@ -427,6 +509,12 @@ def decrypt_unauthenticated(
     Raises:
         TypeError: If lengths are invalid.
     """
+    warnings.warn(
+        "decrypt_unauthenticated is deprecated; use stream_xor "
+        "(its output differs, so keep this only for existing data)",
+        DeprecationWarning,
+        stacklevel=2,
+    )
     key = memoryview(key)
     nonce = memoryview(nonce)
     ct = memoryview(ct)
@@ -708,24 +796,17 @@ class Encryptor:
             raise TypeError(
                 "into length must be >= expected output size for this update"
             )
-        written = ffi.new("size_t *")
         rc = _lib.aegis256_state_encrypt_update(
             self._state.ptr,
             ffi.from_buffer(out_mv),
-            out_mv.nbytes,
-            written,
             _ptr(message),
             message.nbytes,
         )
         if rc != 0:
             err_num = ffi.errno
             err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
-            raise RuntimeError(
-                f"state encrypt update failed: {err_name} written {written[0]}"
-            )
-        w = int(written[0])
-        assert w == expected_out
-        return out if into is None else memoryview(out)[:w]  # type: ignore
+            raise RuntimeError(f"state encrypt update failed: {err_name}")
+        return out if into is None else memoryview(out)[:expected_out]  # type: ignore
 
     def final(self, into: Buffer | None = None) -> bytearray | memoryview:
         """Finalize encryption and return the authentication tag.
@@ -746,24 +827,17 @@ class Encryptor:
         if into is not None:
             into = memoryview(into)
         out = into if into is not None else bytearray(maclen)
-        written = ffi.new("size_t *")
         rc = _lib.aegis256_state_encrypt_final(
             self._state.ptr,
             ffi.from_buffer(out),
-            memoryview(out).nbytes,
-            written,
             maclen,
         )
         if rc != 0:
             err_num = ffi.errno
             err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
             raise RuntimeError(f"state encrypt final failed: {err_name}")
-        w = int(written[0])
-        if into is None:
-            # Only the tag bytes are returned when we allocate the buffer
-            assert w == maclen
         self._state = None
-        return out if into is None else memoryview(out)[:w]  # type: ignore
+        return out if into is None else memoryview(out)[:maclen]  # type: ignore
 
 
 class Decryptor:
@@ -837,12 +911,9 @@ class Decryptor:
         out_mv = memoryview(out)
         if out_mv.nbytes < expected_out:
             raise TypeError("into length must be >= required capacity for this update")
-        written = ffi.new("size_t *")
-        rc = _lib.aegis256_state_decrypt_detached_update(
+        rc = _lib.aegis256_state_decrypt_update(
             self._state.ptr,
             ffi.from_buffer(out_mv),
-            out_mv.nbytes,
-            written,
             _ptr(ct),
             ct.nbytes,
         )
@@ -850,11 +921,7 @@ class Decryptor:
             err_num = ffi.errno
             err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
             raise RuntimeError(f"state decrypt update failed: {err_name}")
-        w = int(written[0])
-        assert w == expected_out, (
-            f"got {w}, expected {expected_out}, ct.nbytes={ct.nbytes}"
-        )
-        return out if into is None else memoryview(out)[:w]  # type: ignore
+        return out if into is None else memoryview(out)[:expected_out]  # type: ignore
 
     def final(self, mac: Buffer) -> None:
         """Finalize decryption by verifying the MAC tag.
@@ -873,9 +940,7 @@ class Decryptor:
         mac = memoryview(mac)
         if mac.nbytes != maclen:
             raise TypeError(f"mac length must be {maclen}")
-        rc = _lib.aegis256_state_decrypt_detached_final(
-            self._state.ptr, ffi.NULL, 0, ffi.NULL, _ptr(mac), maclen
-        )
+        rc = _lib.aegis256_state_decrypt_final(self._state.ptr, _ptr(mac), maclen)
         if rc != 0:
             raise ValueError("authentication failed")
         self._state = None
@@ -891,6 +956,220 @@ def new_mac_state():
     return new_aligned_struct("aegis256_mac_state", ALIGNMENT)
 
 
+RAF_ALG_ID = 4  #: Algorithm identifier in RAF file headers
+
+
+def _raise_errno(operation: str) -> None:
+    err_num = ffi.errno
+    err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
+    raise RuntimeError(f"{operation} failed: {err_name}")
+
+
+def new_raf_ctx():
+    """Allocate and return a new aegis256_raf_ctx* with proper alignment."""
+    return new_aligned_struct("aegis256_raf_ctx", ALIGNMENT)
+
+
+def raf_scratch_size(chunk_size: int) -> int:
+    """Return the scratch buffer size required for the given chunk_size."""
+    return _lib.aegis256_raf_scratch_size(chunk_size)
+
+
+def raf_create(key: Buffer, ctx, io, rng, config) -> None:
+    """Create a new encrypted file, writing the header through io.
+
+    Args:
+        key: Master key (KEYBYTES bytes).
+        ctx: Context from new_raf_ctx().
+        io: aegis_raf_io* cdata (see aeg.raf).
+        rng: aegis_raf_rng* cdata (see aeg.random).
+        config: aegis_raf_config* cdata (see aeg.raf).
+
+    Raises:
+        TypeError: If key length is invalid.
+        FileExistsError: If the file exists and AEGIS_RAF_TRUNCATE was not set.
+        RuntimeError: If creation fails.
+    """
+    key = memoryview(key)
+    if key.nbytes != KEYBYTES:
+        raise TypeError(f"key length must be {KEYBYTES}")
+    rc = _lib.aegis256_raf_create(ctx.ptr, io, rng, config, _ptr(key))
+    if rc != 0:
+        if ffi.errno == errno.EEXIST:
+            raise FileExistsError("file exists (use truncate to overwrite)")
+        _raise_errno("raf create")
+
+
+def raf_open(key: Buffer, ctx, io, rng, config) -> None:
+    """Open an existing encrypted file, verifying the header MAC.
+
+    Args:
+        key: Master key (KEYBYTES bytes).
+        ctx: Context from new_raf_ctx().
+        io: aegis_raf_io* cdata (see aeg.raf).
+        rng: aegis_raf_rng* cdata (see aeg.random).
+        config: aegis_raf_config* cdata (see aeg.raf).
+
+    Raises:
+        TypeError: If key length is invalid.
+        FileNotFoundError: If the file does not exist.
+        ValueError: If the header is invalid or authentication fails (wrong key).
+        RuntimeError: If opening fails.
+    """
+    key = memoryview(key)
+    if key.nbytes != KEYBYTES:
+        raise TypeError(f"key length must be {KEYBYTES}")
+    rc = _lib.aegis256_raf_open(ctx.ptr, io, rng, config, _ptr(key))
+    if rc != 0:
+        err_num = ffi.errno
+        if err_num == errno.ENOENT:
+            raise FileNotFoundError("file not found")
+        if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+            raise ValueError("authentication failed")
+        _raise_errno("raf open")
+
+
+def raf_read(
+    ctx, length: int, offset: int, *, into: Buffer | None = None
+) -> bytearray | memoryview:
+    """Read and decrypt up to length bytes at offset.
+
+    Returns:
+        The decrypted bytes, trimmed to the number actually read
+        (fewer at EOF), as bytearray if into not provided,
+        memoryview of into otherwise.
+
+    Raises:
+        TypeError: If into is smaller than length.
+        ValueError: If authentication fails (corruption or tampering).
+        RuntimeError: If the read fails.
+    """
+    if into is None:
+        out = bytearray(length)
+    else:
+        into = memoryview(into)
+        if into.nbytes < length:
+            raise TypeError("into length must be at least length")
+        out = into
+    bytes_read = ffi.new("size_t*")
+    rc = _lib.aegis256_raf_read(
+        ctx.ptr, ffi.from_buffer(out), bytes_read, length, offset
+    )
+    if rc != 0:
+        err_num = ffi.errno
+        if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+            raise ValueError("authentication failed")
+        _raise_errno("raf read")
+    n = bytes_read[0]
+    return out[:n] if into is None else memoryview(out)[:n]  # type: ignore
+
+
+def raf_write(ctx, data: Buffer, offset: int) -> int:
+    """Encrypt and write data at offset, extending the file as needed.
+
+    Returns:
+        Number of bytes written.
+
+    Raises:
+        RuntimeError: If the write fails (errno name included).
+    """
+    data = memoryview(data)
+    bytes_written = ffi.new("size_t*")
+    rc = _lib.aegis256_raf_write(
+        ctx.ptr, bytes_written, _ptr(data), data.nbytes, offset
+    )
+    if rc != 0:
+        _raise_errno("raf write")
+    return bytes_written[0]
+
+
+def raf_truncate(ctx, size: int) -> None:
+    """Resize the file to size bytes (zero-filled when growing).
+
+    Raises:
+        RuntimeError: If the truncate fails.
+    """
+    rc = _lib.aegis256_raf_truncate(ctx.ptr, size)
+    if rc != 0:
+        _raise_errno("raf truncate")
+
+
+def raf_size(ctx) -> int:
+    """Return the logical plaintext file size."""
+    size = ffi.new("uint64_t*")
+    rc = _lib.aegis256_raf_get_size(ctx.ptr, size)
+    if rc != 0:
+        _raise_errno("raf get_size")
+    return size[0]
+
+
+def raf_sync(ctx) -> None:
+    """Flush writes to the backing store.
+
+    Raises:
+        RuntimeError: If the sync fails.
+    """
+    rc = _lib.aegis256_raf_sync(ctx.ptr)
+    if rc != 0:
+        _raise_errno("raf sync")
+
+
+def raf_close(ctx) -> None:
+    """Close the context, syncing and zeroizing key material."""
+    _lib.aegis256_raf_close(ctx.ptr)
+
+
+def raf_merkle_rebuild(ctx) -> None:
+    """Rebuild the Merkle tree by reading and re-hashing every chunk.
+
+    Raises:
+        RuntimeError: If Merkle is not enabled or rebuilding fails.
+        ValueError: If a chunk fails authentication.
+    """
+    rc = _lib.aegis256_raf_merkle_rebuild(ctx.ptr)
+    if rc != 0:
+        err_num = ffi.errno
+        if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+            raise ValueError("authentication failed")
+        _raise_errno("raf merkle_rebuild")
+
+
+def raf_merkle_verify(ctx) -> int | None:
+    """Verify every chunk's hash against the Merkle tree.
+
+    Returns:
+        None if all chunks match, otherwise the index of the first
+        corrupted chunk.
+
+    Raises:
+        RuntimeError: If Merkle is not enabled or verification fails.
+    """
+    corrupted = ffi.new("uint64_t*")
+    rc = _lib.aegis256_raf_merkle_verify(ctx.ptr, corrupted)
+    if rc == 0:
+        return None
+    err_num = ffi.errno
+    if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+        return corrupted[0]
+    _raise_errno("raf merkle_verify")
+
+
+def raf_merkle_commitment(ctx, hash_len: int) -> bytes:
+    """Compute the Merkle root commitment hash.
+
+    Returns:
+        The commitment hash as bytes of length hash_len.
+
+    Raises:
+        RuntimeError: If Merkle is not enabled or the call fails.
+    """
+    out = ffi.new(f"uint8_t[{hash_len}]")
+    rc = _lib.aegis256_raf_merkle_commitment(ctx.ptr, out, hash_len)
+    if rc != 0:
+        _raise_errno("raf merkle_commitment")
+    return bytes(ffi.buffer(out, hash_len))
+
+
 __all__ = [
     # constants
     "NAME",
@@ -900,7 +1179,9 @@ __all__ = [
     "MACBYTES_LONG",
     "ALIGNMENT",
     "RATE",
+    "RAF_ALG_ID",
     # utility functions
+    "random",
     "random_key",
     "random_nonce",
     "nonce_increment",
@@ -911,6 +1192,7 @@ __all__ = [
     "encrypt",
     "decrypt",
     "stream",
+    "stream_xor",
     "encrypt_unauthenticated",
     "decrypt_unauthenticated",
     "mac",
@@ -918,4 +1200,18 @@ __all__ = [
     "Encryptor",
     "Decryptor",
     "Mac",
+    # random-access encrypted files (thin wrappers; see aeg.raf for the high-level API)
+    "new_raf_ctx",
+    "raf_scratch_size",
+    "raf_create",
+    "raf_open",
+    "raf_read",
+    "raf_write",
+    "raf_truncate",
+    "raf_size",
+    "raf_sync",
+    "raf_close",
+    "raf_merkle_rebuild",
+    "raf_merkle_verify",
+    "raf_merkle_commitment",
 ]

src/aeg/aegis256x2.py

@@ -3,11 +3,14 @@
 # DO NOT EDIT OTHER ALGORITHM FILES MANUALLY!
 
 import errno
-import secrets
+import sys
+import threading
+import warnings
 from typing import Literal
 
 from ._loader import ffi
 from ._loader import lib as _lib
+from .random import Random
 from .util import Buffer, new_aligned_struct, nonce_increment, wipe
 
 NAME = "AEGIS-256X2"  #: Algorithm display name
@@ -19,25 +22,41 @@ ALIGNMENT = 64  #: Required alignment for internal structures
 RATE = 64  #: Byte chunk size in internal processing
 
 
+_random_tls = threading.local()
+
+
+def random() -> Random:
+    """Return this module's Random instance (per-thread singleton).
+
+    Each thread gets its own generator on first use, seeded once from the
+    OS, then deterministic: successive calls never repeat an output block.
+    No locking is involved. See aeg.random.Random for the API.
+    """
+    rng = getattr(_random_tls, "rng", None)
+    if rng is None:
+        rng = _random_tls.rng = Random(sys.modules[__name__])
+    return rng
+
+
 def random_key() -> bytearray:
     """
-    Generate a secret key using cryptographically secure random bytes.
+    Generate a secret key using the module's CSPRNG (see random()).
 
     It is recommended to wipe() the key after no longer needed.
     """
-    return bytearray(secrets.token_bytes(KEYBYTES))
+    return random().bytes(KEYBYTES)
 
 
 def random_nonce() -> bytearray:
     """
-    Generate a public nonce using cryptographically secure random bytes.
+    Generate a public nonce using the module's CSPRNG (see random()).
 
     Nonces (a number used once) are public data that may be sent together
     with the ciphertext, but they need to be unique for each use.
 
     See also: nonce_increment() can be used to derive sequential nonces.
     """
-    return bytearray(secrets.token_bytes(NONCEBYTES))
+    return random().bytes(NONCEBYTES)
 
 
 def _ptr(buf):
@@ -360,6 +379,55 @@ def stream(
     return out if into is None else memoryview(out)[: length or memoryview(out).nbytes]  # type: ignore
 
 
+def stream_xor(
+    key: Buffer,
+    nonce: Buffer,
+    data: Buffer,
+    *,
+    into: Buffer | None = None,
+) -> bytearray | memoryview:
+    """XOR data with the keystream from stream(); encrypts and decrypts.
+
+    Unauthenticated: only use when the data is authenticated some other
+    way. Never reuse a nonce with the same key.
+
+    Args:
+        key: Secret key (generate with random_key()).
+        nonce: Public nonce (generate with random_nonce()).
+        data: Input bytes to XOR.
+        into: Buffer to write the output into (default: bytearray created).
+
+    Returns:
+        Output as bytearray if into not provided, memoryview of into otherwise.
+
+    Raises:
+        TypeError: If lengths are invalid.
+    """
+    key = memoryview(key)
+    nonce = memoryview(nonce)
+    data = memoryview(data)
+    if into is not None:
+        into = memoryview(into)
+    if key.nbytes != KEYBYTES:
+        raise TypeError(f"key length must be {KEYBYTES}")
+    if nonce.nbytes != NONCEBYTES:
+        raise TypeError(f"nonce length must be {NONCEBYTES}")
+    if into is None:
+        out = bytearray(data.nbytes)
+    else:
+        if into.nbytes < data.nbytes:
+            raise TypeError("into length must be at least data.nbytes")
+        out = into
+    _lib.aegis256x2_stream_xor(
+        ffi.from_buffer(out),
+        _ptr(data),
+        data.nbytes,
+        _ptr(nonce),
+        _ptr(key),
+    )
+    return out if into is None else memoryview(out)[: data.nbytes]  # type: ignore
+
+
 def encrypt_unauthenticated(
     key: Buffer,
     nonce: Buffer,
@@ -369,6 +437,10 @@ def encrypt_unauthenticated(
 ) -> bytearray | memoryview:
     """Encrypt message without authentication (for testing/debugging).
 
+    .. deprecated::
+        Use stream_xor() in new code. Its output is different, so keep
+        this only for data in the old format.
+
     Args:
         key: Secret key (generate with random_key()).
         nonce: Public nonce (generate with random_nonce()).
@@ -381,6 +453,12 @@ def encrypt_unauthenticated(
     Raises:
         TypeError: If lengths are invalid.
     """
+    warnings.warn(
+        "encrypt_unauthenticated is deprecated; use stream_xor "
+        "(its output differs, so keep this only for existing data)",
+        DeprecationWarning,
+        stacklevel=2,
+    )
     key = memoryview(key)
     nonce = memoryview(nonce)
     message = memoryview(message)
@@ -415,6 +493,10 @@ def decrypt_unauthenticated(
 ) -> bytearray | memoryview:
     """Decrypt ciphertext without authentication (for testing/debugging).
 
+    .. deprecated::
+        Use stream_xor() in new code. Its output is different, so keep
+        this only for data in the old format.
+
     Args:
         key: Secret key (same key used during encryption).
         nonce: Public nonce (same nonce used during encryption).
@@ -427,6 +509,12 @@ def decrypt_unauthenticated(
     Raises:
         TypeError: If lengths are invalid.
     """
+    warnings.warn(
+        "decrypt_unauthenticated is deprecated; use stream_xor "
+        "(its output differs, so keep this only for existing data)",
+        DeprecationWarning,
+        stacklevel=2,
+    )
     key = memoryview(key)
     nonce = memoryview(nonce)
     ct = memoryview(ct)
@@ -708,24 +796,17 @@ class Encryptor:
             raise TypeError(
                 "into length must be >= expected output size for this update"
             )
-        written = ffi.new("size_t *")
         rc = _lib.aegis256x2_state_encrypt_update(
             self._state.ptr,
             ffi.from_buffer(out_mv),
-            out_mv.nbytes,
-            written,
             _ptr(message),
             message.nbytes,
         )
         if rc != 0:
             err_num = ffi.errno
             err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
-            raise RuntimeError(
-                f"state encrypt update failed: {err_name} written {written[0]}"
-            )
-        w = int(written[0])
-        assert w == expected_out
-        return out if into is None else memoryview(out)[:w]  # type: ignore
+            raise RuntimeError(f"state encrypt update failed: {err_name}")
+        return out if into is None else memoryview(out)[:expected_out]  # type: ignore
 
     def final(self, into: Buffer | None = None) -> bytearray | memoryview:
         """Finalize encryption and return the authentication tag.
@@ -746,24 +827,17 @@ class Encryptor:
         if into is not None:
             into = memoryview(into)
         out = into if into is not None else bytearray(maclen)
-        written = ffi.new("size_t *")
         rc = _lib.aegis256x2_state_encrypt_final(
             self._state.ptr,
             ffi.from_buffer(out),
-            memoryview(out).nbytes,
-            written,
             maclen,
         )
         if rc != 0:
             err_num = ffi.errno
             err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
             raise RuntimeError(f"state encrypt final failed: {err_name}")
-        w = int(written[0])
-        if into is None:
-            # Only the tag bytes are returned when we allocate the buffer
-            assert w == maclen
         self._state = None
-        return out if into is None else memoryview(out)[:w]  # type: ignore
+        return out if into is None else memoryview(out)[:maclen]  # type: ignore
 
 
 class Decryptor:
@@ -837,12 +911,9 @@ class Decryptor:
         out_mv = memoryview(out)
         if out_mv.nbytes < expected_out:
             raise TypeError("into length must be >= required capacity for this update")
-        written = ffi.new("size_t *")
-        rc = _lib.aegis256x2_state_decrypt_detached_update(
+        rc = _lib.aegis256x2_state_decrypt_update(
             self._state.ptr,
             ffi.from_buffer(out_mv),
-            out_mv.nbytes,
-            written,
             _ptr(ct),
             ct.nbytes,
         )
@@ -850,11 +921,7 @@ class Decryptor:
             err_num = ffi.errno
             err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
             raise RuntimeError(f"state decrypt update failed: {err_name}")
-        w = int(written[0])
-        assert w == expected_out, (
-            f"got {w}, expected {expected_out}, ct.nbytes={ct.nbytes}"
-        )
-        return out if into is None else memoryview(out)[:w]  # type: ignore
+        return out if into is None else memoryview(out)[:expected_out]  # type: ignore
 
     def final(self, mac: Buffer) -> None:
         """Finalize decryption by verifying the MAC tag.
@@ -873,9 +940,7 @@ class Decryptor:
         mac = memoryview(mac)
         if mac.nbytes != maclen:
             raise TypeError(f"mac length must be {maclen}")
-        rc = _lib.aegis256x2_state_decrypt_detached_final(
-            self._state.ptr, ffi.NULL, 0, ffi.NULL, _ptr(mac), maclen
-        )
+        rc = _lib.aegis256x2_state_decrypt_final(self._state.ptr, _ptr(mac), maclen)
         if rc != 0:
             raise ValueError("authentication failed")
         self._state = None
@@ -891,6 +956,220 @@ def new_mac_state():
     return new_aligned_struct("aegis256x2_mac_state", ALIGNMENT)
 
 
+RAF_ALG_ID = 5  #: Algorithm identifier in RAF file headers
+
+
+def _raise_errno(operation: str) -> None:
+    err_num = ffi.errno
+    err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
+    raise RuntimeError(f"{operation} failed: {err_name}")
+
+
+def new_raf_ctx():
+    """Allocate and return a new aegis256x2_raf_ctx* with proper alignment."""
+    return new_aligned_struct("aegis256x2_raf_ctx", ALIGNMENT)
+
+
+def raf_scratch_size(chunk_size: int) -> int:
+    """Return the scratch buffer size required for the given chunk_size."""
+    return _lib.aegis256x2_raf_scratch_size(chunk_size)
+
+
+def raf_create(key: Buffer, ctx, io, rng, config) -> None:
+    """Create a new encrypted file, writing the header through io.
+
+    Args:
+        key: Master key (KEYBYTES bytes).
+        ctx: Context from new_raf_ctx().
+        io: aegis_raf_io* cdata (see aeg.raf).
+        rng: aegis_raf_rng* cdata (see aeg.random).
+        config: aegis_raf_config* cdata (see aeg.raf).
+
+    Raises:
+        TypeError: If key length is invalid.
+        FileExistsError: If the file exists and AEGIS_RAF_TRUNCATE was not set.
+        RuntimeError: If creation fails.
+    """
+    key = memoryview(key)
+    if key.nbytes != KEYBYTES:
+        raise TypeError(f"key length must be {KEYBYTES}")
+    rc = _lib.aegis256x2_raf_create(ctx.ptr, io, rng, config, _ptr(key))
+    if rc != 0:
+        if ffi.errno == errno.EEXIST:
+            raise FileExistsError("file exists (use truncate to overwrite)")
+        _raise_errno("raf create")
+
+
+def raf_open(key: Buffer, ctx, io, rng, config) -> None:
+    """Open an existing encrypted file, verifying the header MAC.
+
+    Args:
+        key: Master key (KEYBYTES bytes).
+        ctx: Context from new_raf_ctx().
+        io: aegis_raf_io* cdata (see aeg.raf).
+        rng: aegis_raf_rng* cdata (see aeg.random).
+        config: aegis_raf_config* cdata (see aeg.raf).
+
+    Raises:
+        TypeError: If key length is invalid.
+        FileNotFoundError: If the file does not exist.
+        ValueError: If the header is invalid or authentication fails (wrong key).
+        RuntimeError: If opening fails.
+    """
+    key = memoryview(key)
+    if key.nbytes != KEYBYTES:
+        raise TypeError(f"key length must be {KEYBYTES}")
+    rc = _lib.aegis256x2_raf_open(ctx.ptr, io, rng, config, _ptr(key))
+    if rc != 0:
+        err_num = ffi.errno
+        if err_num == errno.ENOENT:
+            raise FileNotFoundError("file not found")
+        if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+            raise ValueError("authentication failed")
+        _raise_errno("raf open")
+
+
+def raf_read(
+    ctx, length: int, offset: int, *, into: Buffer | None = None
+) -> bytearray | memoryview:
+    """Read and decrypt up to length bytes at offset.
+
+    Returns:
+        The decrypted bytes, trimmed to the number actually read
+        (fewer at EOF), as bytearray if into not provided,
+        memoryview of into otherwise.
+
+    Raises:
+        TypeError: If into is smaller than length.
+        ValueError: If authentication fails (corruption or tampering).
+        RuntimeError: If the read fails.
+    """
+    if into is None:
+        out = bytearray(length)
+    else:
+        into = memoryview(into)
+        if into.nbytes < length:
+            raise TypeError("into length must be at least length")
+        out = into
+    bytes_read = ffi.new("size_t*")
+    rc = _lib.aegis256x2_raf_read(
+        ctx.ptr, ffi.from_buffer(out), bytes_read, length, offset
+    )
+    if rc != 0:
+        err_num = ffi.errno
+        if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+            raise ValueError("authentication failed")
+        _raise_errno("raf read")
+    n = bytes_read[0]
+    return out[:n] if into is None else memoryview(out)[:n]  # type: ignore
+
+
+def raf_write(ctx, data: Buffer, offset: int) -> int:
+    """Encrypt and write data at offset, extending the file as needed.
+
+    Returns:
+        Number of bytes written.
+
+    Raises:
+        RuntimeError: If the write fails (errno name included).
+    """
+    data = memoryview(data)
+    bytes_written = ffi.new("size_t*")
+    rc = _lib.aegis256x2_raf_write(
+        ctx.ptr, bytes_written, _ptr(data), data.nbytes, offset
+    )
+    if rc != 0:
+        _raise_errno("raf write")
+    return bytes_written[0]
+
+
+def raf_truncate(ctx, size: int) -> None:
+    """Resize the file to size bytes (zero-filled when growing).
+
+    Raises:
+        RuntimeError: If the truncate fails.
+    """
+    rc = _lib.aegis256x2_raf_truncate(ctx.ptr, size)
+    if rc != 0:
+        _raise_errno("raf truncate")
+
+
+def raf_size(ctx) -> int:
+    """Return the logical plaintext file size."""
+    size = ffi.new("uint64_t*")
+    rc = _lib.aegis256x2_raf_get_size(ctx.ptr, size)
+    if rc != 0:
+        _raise_errno("raf get_size")
+    return size[0]
+
+
+def raf_sync(ctx) -> None:
+    """Flush writes to the backing store.
+
+    Raises:
+        RuntimeError: If the sync fails.
+    """
+    rc = _lib.aegis256x2_raf_sync(ctx.ptr)
+    if rc != 0:
+        _raise_errno("raf sync")
+
+
+def raf_close(ctx) -> None:
+    """Close the context, syncing and zeroizing key material."""
+    _lib.aegis256x2_raf_close(ctx.ptr)
+
+
+def raf_merkle_rebuild(ctx) -> None:
+    """Rebuild the Merkle tree by reading and re-hashing every chunk.
+
+    Raises:
+        RuntimeError: If Merkle is not enabled or rebuilding fails.
+        ValueError: If a chunk fails authentication.
+    """
+    rc = _lib.aegis256x2_raf_merkle_rebuild(ctx.ptr)
+    if rc != 0:
+        err_num = ffi.errno
+        if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+            raise ValueError("authentication failed")
+        _raise_errno("raf merkle_rebuild")
+
+
+def raf_merkle_verify(ctx) -> int | None:
+    """Verify every chunk's hash against the Merkle tree.
+
+    Returns:
+        None if all chunks match, otherwise the index of the first
+        corrupted chunk.
+
+    Raises:
+        RuntimeError: If Merkle is not enabled or verification fails.
+    """
+    corrupted = ffi.new("uint64_t*")
+    rc = _lib.aegis256x2_raf_merkle_verify(ctx.ptr, corrupted)
+    if rc == 0:
+        return None
+    err_num = ffi.errno
+    if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+        return corrupted[0]
+    _raise_errno("raf merkle_verify")
+
+
+def raf_merkle_commitment(ctx, hash_len: int) -> bytes:
+    """Compute the Merkle root commitment hash.
+
+    Returns:
+        The commitment hash as bytes of length hash_len.
+
+    Raises:
+        RuntimeError: If Merkle is not enabled or the call fails.
+    """
+    out = ffi.new(f"uint8_t[{hash_len}]")
+    rc = _lib.aegis256x2_raf_merkle_commitment(ctx.ptr, out, hash_len)
+    if rc != 0:
+        _raise_errno("raf merkle_commitment")
+    return bytes(ffi.buffer(out, hash_len))
+
+
 __all__ = [
     # constants
     "NAME",
@@ -900,7 +1179,9 @@ __all__ = [
     "MACBYTES_LONG",
     "ALIGNMENT",
     "RATE",
+    "RAF_ALG_ID",
     # utility functions
+    "random",
     "random_key",
     "random_nonce",
     "nonce_increment",
@@ -911,6 +1192,7 @@ __all__ = [
     "encrypt",
     "decrypt",
     "stream",
+    "stream_xor",
     "encrypt_unauthenticated",
     "decrypt_unauthenticated",
     "mac",
@@ -918,4 +1200,18 @@ __all__ = [
     "Encryptor",
     "Decryptor",
     "Mac",
+    # random-access encrypted files (thin wrappers; see aeg.raf for the high-level API)
+    "new_raf_ctx",
+    "raf_scratch_size",
+    "raf_create",
+    "raf_open",
+    "raf_read",
+    "raf_write",
+    "raf_truncate",
+    "raf_size",
+    "raf_sync",
+    "raf_close",
+    "raf_merkle_rebuild",
+    "raf_merkle_verify",
+    "raf_merkle_commitment",
 ]

src/aeg/aegis256x4.py

@@ -3,11 +3,14 @@
 # DO NOT EDIT OTHER ALGORITHM FILES MANUALLY!
 
 import errno
-import secrets
+import sys
+import threading
+import warnings
 from typing import Literal
 
 from ._loader import ffi
 from ._loader import lib as _lib
+from .random import Random
 from .util import Buffer, new_aligned_struct, nonce_increment, wipe
 
 NAME = "AEGIS-256X4"  #: Algorithm display name
@@ -19,25 +22,41 @@ ALIGNMENT = 64  #: Required alignment for internal structures
 RATE = 64  #: Byte chunk size in internal processing
 
 
+_random_tls = threading.local()
+
+
+def random() -> Random:
+    """Return this module's Random instance (per-thread singleton).
+
+    Each thread gets its own generator on first use, seeded once from the
+    OS, then deterministic: successive calls never repeat an output block.
+    No locking is involved. See aeg.random.Random for the API.
+    """
+    rng = getattr(_random_tls, "rng", None)
+    if rng is None:
+        rng = _random_tls.rng = Random(sys.modules[__name__])
+    return rng
+
+
 def random_key() -> bytearray:
     """
-    Generate a secret key using cryptographically secure random bytes.
+    Generate a secret key using the module's CSPRNG (see random()).
 
     It is recommended to wipe() the key after no longer needed.
     """
-    return bytearray(secrets.token_bytes(KEYBYTES))
+    return random().bytes(KEYBYTES)
 
 
 def random_nonce() -> bytearray:
     """
-    Generate a public nonce using cryptographically secure random bytes.
+    Generate a public nonce using the module's CSPRNG (see random()).
 
     Nonces (a number used once) are public data that may be sent together
     with the ciphertext, but they need to be unique for each use.
 
     See also: nonce_increment() can be used to derive sequential nonces.
     """
-    return bytearray(secrets.token_bytes(NONCEBYTES))
+    return random().bytes(NONCEBYTES)
 
 
 def _ptr(buf):
@@ -360,6 +379,55 @@ def stream(
     return out if into is None else memoryview(out)[: length or memoryview(out).nbytes]  # type: ignore
 
 
+def stream_xor(
+    key: Buffer,
+    nonce: Buffer,
+    data: Buffer,
+    *,
+    into: Buffer | None = None,
+) -> bytearray | memoryview:
+    """XOR data with the keystream from stream(); encrypts and decrypts.
+
+    Unauthenticated: only use when the data is authenticated some other
+    way. Never reuse a nonce with the same key.
+
+    Args:
+        key: Secret key (generate with random_key()).
+        nonce: Public nonce (generate with random_nonce()).
+        data: Input bytes to XOR.
+        into: Buffer to write the output into (default: bytearray created).
+
+    Returns:
+        Output as bytearray if into not provided, memoryview of into otherwise.
+
+    Raises:
+        TypeError: If lengths are invalid.
+    """
+    key = memoryview(key)
+    nonce = memoryview(nonce)
+    data = memoryview(data)
+    if into is not None:
+        into = memoryview(into)
+    if key.nbytes != KEYBYTES:
+        raise TypeError(f"key length must be {KEYBYTES}")
+    if nonce.nbytes != NONCEBYTES:
+        raise TypeError(f"nonce length must be {NONCEBYTES}")
+    if into is None:
+        out = bytearray(data.nbytes)
+    else:
+        if into.nbytes < data.nbytes:
+            raise TypeError("into length must be at least data.nbytes")
+        out = into
+    _lib.aegis256x4_stream_xor(
+        ffi.from_buffer(out),
+        _ptr(data),
+        data.nbytes,
+        _ptr(nonce),
+        _ptr(key),
+    )
+    return out if into is None else memoryview(out)[: data.nbytes]  # type: ignore
+
+
 def encrypt_unauthenticated(
     key: Buffer,
     nonce: Buffer,
@@ -369,6 +437,10 @@ def encrypt_unauthenticated(
 ) -> bytearray | memoryview:
     """Encrypt message without authentication (for testing/debugging).
 
+    .. deprecated::
+        Use stream_xor() in new code. Its output is different, so keep
+        this only for data in the old format.
+
     Args:
         key: Secret key (generate with random_key()).
         nonce: Public nonce (generate with random_nonce()).
@@ -381,6 +453,12 @@ def encrypt_unauthenticated(
     Raises:
         TypeError: If lengths are invalid.
     """
+    warnings.warn(
+        "encrypt_unauthenticated is deprecated; use stream_xor "
+        "(its output differs, so keep this only for existing data)",
+        DeprecationWarning,
+        stacklevel=2,
+    )
     key = memoryview(key)
     nonce = memoryview(nonce)
     message = memoryview(message)
@@ -415,6 +493,10 @@ def decrypt_unauthenticated(
 ) -> bytearray | memoryview:
     """Decrypt ciphertext without authentication (for testing/debugging).
 
+    .. deprecated::
+        Use stream_xor() in new code. Its output is different, so keep
+        this only for data in the old format.
+
     Args:
         key: Secret key (same key used during encryption).
         nonce: Public nonce (same nonce used during encryption).
@@ -427,6 +509,12 @@ def decrypt_unauthenticated(
     Raises:
         TypeError: If lengths are invalid.
     """
+    warnings.warn(
+        "decrypt_unauthenticated is deprecated; use stream_xor "
+        "(its output differs, so keep this only for existing data)",
+        DeprecationWarning,
+        stacklevel=2,
+    )
     key = memoryview(key)
     nonce = memoryview(nonce)
     ct = memoryview(ct)
@@ -708,24 +796,17 @@ class Encryptor:
             raise TypeError(
                 "into length must be >= expected output size for this update"
             )
-        written = ffi.new("size_t *")
         rc = _lib.aegis256x4_state_encrypt_update(
             self._state.ptr,
             ffi.from_buffer(out_mv),
-            out_mv.nbytes,
-            written,
             _ptr(message),
             message.nbytes,
         )
         if rc != 0:
             err_num = ffi.errno
             err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
-            raise RuntimeError(
-                f"state encrypt update failed: {err_name} written {written[0]}"
-            )
-        w = int(written[0])
-        assert w == expected_out
-        return out if into is None else memoryview(out)[:w]  # type: ignore
+            raise RuntimeError(f"state encrypt update failed: {err_name}")
+        return out if into is None else memoryview(out)[:expected_out]  # type: ignore
 
     def final(self, into: Buffer | None = None) -> bytearray | memoryview:
         """Finalize encryption and return the authentication tag.
@@ -746,24 +827,17 @@ class Encryptor:
         if into is not None:
             into = memoryview(into)
         out = into if into is not None else bytearray(maclen)
-        written = ffi.new("size_t *")
         rc = _lib.aegis256x4_state_encrypt_final(
             self._state.ptr,
             ffi.from_buffer(out),
-            memoryview(out).nbytes,
-            written,
             maclen,
         )
         if rc != 0:
             err_num = ffi.errno
             err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
             raise RuntimeError(f"state encrypt final failed: {err_name}")
-        w = int(written[0])
-        if into is None:
-            # Only the tag bytes are returned when we allocate the buffer
-            assert w == maclen
         self._state = None
-        return out if into is None else memoryview(out)[:w]  # type: ignore
+        return out if into is None else memoryview(out)[:maclen]  # type: ignore
 
 
 class Decryptor:
@@ -837,12 +911,9 @@ class Decryptor:
         out_mv = memoryview(out)
         if out_mv.nbytes < expected_out:
             raise TypeError("into length must be >= required capacity for this update")
-        written = ffi.new("size_t *")
-        rc = _lib.aegis256x4_state_decrypt_detached_update(
+        rc = _lib.aegis256x4_state_decrypt_update(
             self._state.ptr,
             ffi.from_buffer(out_mv),
-            out_mv.nbytes,
-            written,
             _ptr(ct),
             ct.nbytes,
         )
@@ -850,11 +921,7 @@ class Decryptor:
             err_num = ffi.errno
             err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
             raise RuntimeError(f"state decrypt update failed: {err_name}")
-        w = int(written[0])
-        assert w == expected_out, (
-            f"got {w}, expected {expected_out}, ct.nbytes={ct.nbytes}"
-        )
-        return out if into is None else memoryview(out)[:w]  # type: ignore
+        return out if into is None else memoryview(out)[:expected_out]  # type: ignore
 
     def final(self, mac: Buffer) -> None:
         """Finalize decryption by verifying the MAC tag.
@@ -873,9 +940,7 @@ class Decryptor:
         mac = memoryview(mac)
         if mac.nbytes != maclen:
             raise TypeError(f"mac length must be {maclen}")
-        rc = _lib.aegis256x4_state_decrypt_detached_final(
-            self._state.ptr, ffi.NULL, 0, ffi.NULL, _ptr(mac), maclen
-        )
+        rc = _lib.aegis256x4_state_decrypt_final(self._state.ptr, _ptr(mac), maclen)
         if rc != 0:
             raise ValueError("authentication failed")
         self._state = None
@@ -891,6 +956,220 @@ def new_mac_state():
     return new_aligned_struct("aegis256x4_mac_state", ALIGNMENT)
 
 
+RAF_ALG_ID = 6  #: Algorithm identifier in RAF file headers
+
+
+def _raise_errno(operation: str) -> None:
+    err_num = ffi.errno
+    err_name = errno.errorcode.get(err_num, f"errno_{err_num}")
+    raise RuntimeError(f"{operation} failed: {err_name}")
+
+
+def new_raf_ctx():
+    """Allocate and return a new aegis256x4_raf_ctx* with proper alignment."""
+    return new_aligned_struct("aegis256x4_raf_ctx", ALIGNMENT)
+
+
+def raf_scratch_size(chunk_size: int) -> int:
+    """Return the scratch buffer size required for the given chunk_size."""
+    return _lib.aegis256x4_raf_scratch_size(chunk_size)
+
+
+def raf_create(key: Buffer, ctx, io, rng, config) -> None:
+    """Create a new encrypted file, writing the header through io.
+
+    Args:
+        key: Master key (KEYBYTES bytes).
+        ctx: Context from new_raf_ctx().
+        io: aegis_raf_io* cdata (see aeg.raf).
+        rng: aegis_raf_rng* cdata (see aeg.random).
+        config: aegis_raf_config* cdata (see aeg.raf).
+
+    Raises:
+        TypeError: If key length is invalid.
+        FileExistsError: If the file exists and AEGIS_RAF_TRUNCATE was not set.
+        RuntimeError: If creation fails.
+    """
+    key = memoryview(key)
+    if key.nbytes != KEYBYTES:
+        raise TypeError(f"key length must be {KEYBYTES}")
+    rc = _lib.aegis256x4_raf_create(ctx.ptr, io, rng, config, _ptr(key))
+    if rc != 0:
+        if ffi.errno == errno.EEXIST:
+            raise FileExistsError("file exists (use truncate to overwrite)")
+        _raise_errno("raf create")
+
+
+def raf_open(key: Buffer, ctx, io, rng, config) -> None:
+    """Open an existing encrypted file, verifying the header MAC.
+
+    Args:
+        key: Master key (KEYBYTES bytes).
+        ctx: Context from new_raf_ctx().
+        io: aegis_raf_io* cdata (see aeg.raf).
+        rng: aegis_raf_rng* cdata (see aeg.random).
+        config: aegis_raf_config* cdata (see aeg.raf).
+
+    Raises:
+        TypeError: If key length is invalid.
+        FileNotFoundError: If the file does not exist.
+        ValueError: If the header is invalid or authentication fails (wrong key).
+        RuntimeError: If opening fails.
+    """
+    key = memoryview(key)
+    if key.nbytes != KEYBYTES:
+        raise TypeError(f"key length must be {KEYBYTES}")
+    rc = _lib.aegis256x4_raf_open(ctx.ptr, io, rng, config, _ptr(key))
+    if rc != 0:
+        err_num = ffi.errno
+        if err_num == errno.ENOENT:
+            raise FileNotFoundError("file not found")
+        if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+            raise ValueError("authentication failed")
+        _raise_errno("raf open")
+
+
+def raf_read(
+    ctx, length: int, offset: int, *, into: Buffer | None = None
+) -> bytearray | memoryview:
+    """Read and decrypt up to length bytes at offset.
+
+    Returns:
+        The decrypted bytes, trimmed to the number actually read
+        (fewer at EOF), as bytearray if into not provided,
+        memoryview of into otherwise.
+
+    Raises:
+        TypeError: If into is smaller than length.
+        ValueError: If authentication fails (corruption or tampering).
+        RuntimeError: If the read fails.
+    """
+    if into is None:
+        out = bytearray(length)
+    else:
+        into = memoryview(into)
+        if into.nbytes < length:
+            raise TypeError("into length must be at least length")
+        out = into
+    bytes_read = ffi.new("size_t*")
+    rc = _lib.aegis256x4_raf_read(
+        ctx.ptr, ffi.from_buffer(out), bytes_read, length, offset
+    )
+    if rc != 0:
+        err_num = ffi.errno
+        if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+            raise ValueError("authentication failed")
+        _raise_errno("raf read")
+    n = bytes_read[0]
+    return out[:n] if into is None else memoryview(out)[:n]  # type: ignore
+
+
+def raf_write(ctx, data: Buffer, offset: int) -> int:
+    """Encrypt and write data at offset, extending the file as needed.
+
+    Returns:
+        Number of bytes written.
+
+    Raises:
+        RuntimeError: If the write fails (errno name included).
+    """
+    data = memoryview(data)
+    bytes_written = ffi.new("size_t*")
+    rc = _lib.aegis256x4_raf_write(
+        ctx.ptr, bytes_written, _ptr(data), data.nbytes, offset
+    )
+    if rc != 0:
+        _raise_errno("raf write")
+    return bytes_written[0]
+
+
+def raf_truncate(ctx, size: int) -> None:
+    """Resize the file to size bytes (zero-filled when growing).
+
+    Raises:
+        RuntimeError: If the truncate fails.
+    """
+    rc = _lib.aegis256x4_raf_truncate(ctx.ptr, size)
+    if rc != 0:
+        _raise_errno("raf truncate")
+
+
+def raf_size(ctx) -> int:
+    """Return the logical plaintext file size."""
+    size = ffi.new("uint64_t*")
+    rc = _lib.aegis256x4_raf_get_size(ctx.ptr, size)
+    if rc != 0:
+        _raise_errno("raf get_size")
+    return size[0]
+
+
+def raf_sync(ctx) -> None:
+    """Flush writes to the backing store.
+
+    Raises:
+        RuntimeError: If the sync fails.
+    """
+    rc = _lib.aegis256x4_raf_sync(ctx.ptr)
+    if rc != 0:
+        _raise_errno("raf sync")
+
+
+def raf_close(ctx) -> None:
+    """Close the context, syncing and zeroizing key material."""
+    _lib.aegis256x4_raf_close(ctx.ptr)
+
+
+def raf_merkle_rebuild(ctx) -> None:
+    """Rebuild the Merkle tree by reading and re-hashing every chunk.
+
+    Raises:
+        RuntimeError: If Merkle is not enabled or rebuilding fails.
+        ValueError: If a chunk fails authentication.
+    """
+    rc = _lib.aegis256x4_raf_merkle_rebuild(ctx.ptr)
+    if rc != 0:
+        err_num = ffi.errno
+        if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+            raise ValueError("authentication failed")
+        _raise_errno("raf merkle_rebuild")
+
+
+def raf_merkle_verify(ctx) -> int | None:
+    """Verify every chunk's hash against the Merkle tree.
+
+    Returns:
+        None if all chunks match, otherwise the index of the first
+        corrupted chunk.
+
+    Raises:
+        RuntimeError: If Merkle is not enabled or verification fails.
+    """
+    corrupted = ffi.new("uint64_t*")
+    rc = _lib.aegis256x4_raf_merkle_verify(ctx.ptr, corrupted)
+    if rc == 0:
+        return None
+    err_num = ffi.errno
+    if err_num in (errno.EINVAL, getattr(errno, "EBADMSG", -1)):
+        return corrupted[0]
+    _raise_errno("raf merkle_verify")
+
+
+def raf_merkle_commitment(ctx, hash_len: int) -> bytes:
+    """Compute the Merkle root commitment hash.
+
+    Returns:
+        The commitment hash as bytes of length hash_len.
+
+    Raises:
+        RuntimeError: If Merkle is not enabled or the call fails.
+    """
+    out = ffi.new(f"uint8_t[{hash_len}]")
+    rc = _lib.aegis256x4_raf_merkle_commitment(ctx.ptr, out, hash_len)
+    if rc != 0:
+        _raise_errno("raf merkle_commitment")
+    return bytes(ffi.buffer(out, hash_len))
+
+
 __all__ = [
     # constants
     "NAME",
@@ -900,7 +1179,9 @@ __all__ = [
     "MACBYTES_LONG",
     "ALIGNMENT",
     "RATE",
+    "RAF_ALG_ID",
     # utility functions
+    "random",
     "random_key",
     "random_nonce",
     "nonce_increment",
@@ -911,6 +1192,7 @@ __all__ = [
     "encrypt",
     "decrypt",
     "stream",
+    "stream_xor",
     "encrypt_unauthenticated",
     "decrypt_unauthenticated",
     "mac",
@@ -918,4 +1200,18 @@ __all__ = [
     "Encryptor",
     "Decryptor",
     "Mac",
+    # random-access encrypted files (thin wrappers; see aeg.raf for the high-level API)
+    "new_raf_ctx",
+    "raf_scratch_size",
+    "raf_create",
+    "raf_open",
+    "raf_read",
+    "raf_write",
+    "raf_truncate",
+    "raf_size",
+    "raf_sync",
+    "raf_close",
+    "raf_merkle_rebuild",
+    "raf_merkle_verify",
+    "raf_merkle_commitment",
 ]

src/aeg/aegis_cdef.h

@@ -1,16 +1,13 @@
 /* This file is generated with tools/generate.py. Do not edit. */
 
-typedef unsigned char uint8_t;
-typedef unsigned long size_t;
-
 /* aegis.h */
 int aegis_init(void);
-int aegis_verify_16(const uint8_t *x, const uint8_t *y) ;
-int aegis_verify_32(const uint8_t *x, const uint8_t *y) ;
+int aegis_verify_16(const uint8_t *x, const uint8_t *y);
+int aegis_verify_32(const uint8_t *x, const uint8_t *y);
 
 /* aegis128l.h */
-typedef struct aegis128l_state { ...; } aegis128l_state;
-typedef struct aegis128l_mac_state { ...; } aegis128l_mac_state;
+typedef struct aegis128l_state { uint8_t opaque[256]; } aegis128l_state;
+typedef struct aegis128l_mac_state { uint8_t opaque[384]; } aegis128l_mac_state;
 size_t aegis128l_keybytes(void);
 size_t aegis128l_npubbytes(void);
 size_t aegis128l_abytes_min(void);
@@ -33,7 +30,7 @@ int aegis128l_decrypt_detached(uint8_t *m,
                                const uint8_t *ad,
                                size_t adlen,
                                const uint8_t *npub,
-                               const uint8_t *k) ;
+                               const uint8_t *k);
 int aegis128l_encrypt(uint8_t *c,
                       size_t maclen,
                       const uint8_t *m,
@@ -49,42 +46,22 @@ int aegis128l_decrypt(uint8_t *m,
                       const uint8_t *ad,
                       size_t adlen,
                       const uint8_t *npub,
-                      const uint8_t *k) ;
+                      const uint8_t *k);
 void aegis128l_state_init(aegis128l_state *st_,
                           const uint8_t *ad,
                           size_t adlen,
                           const uint8_t *npub,
                           const uint8_t *k);
-int aegis128l_state_encrypt_update(aegis128l_state *st_,
-                                   uint8_t *c,
-                                   size_t clen_max,
-                                   size_t *written,
-                                   const uint8_t *m,
-                                   size_t mlen);
-int aegis128l_state_encrypt_detached_final(aegis128l_state *st_,
-                                           uint8_t *c,
-                                           size_t clen_max,
-                                           size_t *written,
-                                           uint8_t *mac,
-                                           size_t maclen);
-int aegis128l_state_encrypt_final(aegis128l_state *st_,
-                                  uint8_t *c,
-                                  size_t clen_max,
-                                  size_t *written,
-                                  size_t maclen);
-int aegis128l_state_decrypt_detached_update(aegis128l_state *st_,
-                                            uint8_t *m,
-                                            size_t mlen_max,
-                                            size_t *written,
-                                            const uint8_t *c,
-                                            size_t clen) ;
-int aegis128l_state_decrypt_detached_final(aegis128l_state *st_,
-                                           uint8_t *m,
-                                           size_t mlen_max,
-                                           size_t *written,
-                                           const uint8_t *mac,
-                                           size_t maclen) ;
+int aegis128l_state_encrypt_update(aegis128l_state *st_, uint8_t *c, const uint8_t *m, size_t mlen);
+int aegis128l_state_encrypt_final(aegis128l_state *st_, uint8_t *mac, size_t maclen);
+int aegis128l_state_decrypt_update(aegis128l_state *st_, uint8_t *m, const uint8_t *c, size_t clen);
+int aegis128l_state_decrypt_final(aegis128l_state *st_, const uint8_t *mac, size_t maclen);
 void aegis128l_stream(uint8_t *out, size_t len, const uint8_t *npub, const uint8_t *k);
+void aegis128l_stream_xor(uint8_t *out,
+                          const uint8_t *in,
+                          size_t len,
+                          const uint8_t *npub,
+                          const uint8_t *k);
 void aegis128l_encrypt_unauthenticated(uint8_t *c,
                                        const uint8_t *m,
                                        size_t mlen,
@@ -103,8 +80,8 @@ void aegis128l_mac_reset(aegis128l_mac_state *st_);
 void aegis128l_mac_state_clone(aegis128l_mac_state *dst, const aegis128l_mac_state *src);
 
 /* aegis128x2.h */
-typedef struct aegis128x2_state { ...; } aegis128x2_state;
-typedef struct aegis128x2_mac_state { ...; } aegis128x2_mac_state;
+typedef struct aegis128x2_state { uint8_t opaque[448]; } aegis128x2_state;
+typedef struct aegis128x2_mac_state { uint8_t opaque[704]; } aegis128x2_mac_state;
 size_t aegis128x2_keybytes(void);
 size_t aegis128x2_npubbytes(void);
 size_t aegis128x2_abytes_min(void);
@@ -127,7 +104,7 @@ int aegis128x2_decrypt_detached(uint8_t *m,
                                 const uint8_t *ad,
                                 size_t adlen,
                                 const uint8_t *npub,
-                                const uint8_t *k) ;
+                                const uint8_t *k);
 int aegis128x2_encrypt(uint8_t *c,
                        size_t maclen,
                        const uint8_t *m,
@@ -143,7 +120,7 @@ int aegis128x2_decrypt(uint8_t *m,
                        const uint8_t *ad,
                        size_t adlen,
                        const uint8_t *npub,
-                       const uint8_t *k) ;
+                       const uint8_t *k);
 void aegis128x2_state_init(aegis128x2_state *st_,
                            const uint8_t *ad,
                            size_t adlen,
@@ -151,34 +128,20 @@ void aegis128x2_state_init(aegis128x2_state *st_,
                            const uint8_t *k);
 int aegis128x2_state_encrypt_update(aegis128x2_state *st_,
                                     uint8_t *c,
-                                    size_t clen_max,
-                                    size_t *written,
                                     const uint8_t *m,
                                     size_t mlen);
-int aegis128x2_state_encrypt_detached_final(aegis128x2_state *st_,
-                                            uint8_t *c,
-                                            size_t clen_max,
-                                            size_t *written,
-                                            uint8_t *mac,
-                                            size_t maclen);
-int aegis128x2_state_encrypt_final(aegis128x2_state *st_,
-                                   uint8_t *c,
-                                   size_t clen_max,
-                                   size_t *written,
-                                   size_t maclen);
-int aegis128x2_state_decrypt_detached_update(aegis128x2_state *st_,
-                                             uint8_t *m,
-                                             size_t mlen_max,
-                                             size_t *written,
-                                             const uint8_t *c,
-                                             size_t clen) ;
-int aegis128x2_state_decrypt_detached_final(aegis128x2_state *st_,
-                                            uint8_t *m,
-                                            size_t mlen_max,
-                                            size_t *written,
-                                            const uint8_t *mac,
-                                            size_t maclen) ;
+int aegis128x2_state_encrypt_final(aegis128x2_state *st_, uint8_t *mac, size_t maclen);
+int aegis128x2_state_decrypt_update(aegis128x2_state *st_,
+                                    uint8_t *m,
+                                    const uint8_t *c,
+                                    size_t clen);
+int aegis128x2_state_decrypt_final(aegis128x2_state *st_, const uint8_t *mac, size_t maclen);
 void aegis128x2_stream(uint8_t *out, size_t len, const uint8_t *npub, const uint8_t *k);
+void aegis128x2_stream_xor(uint8_t *out,
+                           const uint8_t *in,
+                           size_t len,
+                           const uint8_t *npub,
+                           const uint8_t *k);
 void aegis128x2_encrypt_unauthenticated(uint8_t *c,
                                         const uint8_t *m,
                                         size_t mlen,
@@ -197,8 +160,8 @@ void aegis128x2_mac_reset(aegis128x2_mac_state *st_);
 void aegis128x2_mac_state_clone(aegis128x2_mac_state *dst, const aegis128x2_mac_state *src);
 
 /* aegis128x4.h */
-typedef struct aegis128x4_state { ...; } aegis128x4_state;
-typedef struct aegis128x4_mac_state { ...; } aegis128x4_mac_state;
+typedef struct aegis128x4_state { uint8_t opaque[832]; } aegis128x4_state;
+typedef struct aegis128x4_mac_state { uint8_t opaque[1344]; } aegis128x4_mac_state;
 size_t aegis128x4_keybytes(void);
 size_t aegis128x4_npubbytes(void);
 size_t aegis128x4_abytes_min(void);
@@ -221,7 +184,7 @@ int aegis128x4_decrypt_detached(uint8_t *m,
                                 const uint8_t *ad,
                                 size_t adlen,
                                 const uint8_t *npub,
-                                const uint8_t *k) ;
+                                const uint8_t *k);
 int aegis128x4_encrypt(uint8_t *c,
                        size_t maclen,
                        const uint8_t *m,
@@ -237,7 +200,7 @@ int aegis128x4_decrypt(uint8_t *m,
                        const uint8_t *ad,
                        size_t adlen,
                        const uint8_t *npub,
-                       const uint8_t *k) ;
+                       const uint8_t *k);
 void aegis128x4_state_init(aegis128x4_state *st_,
                            const uint8_t *ad,
                            size_t adlen,
@@ -245,34 +208,20 @@ void aegis128x4_state_init(aegis128x4_state *st_,
                            const uint8_t *k);
 int aegis128x4_state_encrypt_update(aegis128x4_state *st_,
                                     uint8_t *c,
-                                    size_t clen_max,
-                                    size_t *written,
                                     const uint8_t *m,
                                     size_t mlen);
-int aegis128x4_state_encrypt_detached_final(aegis128x4_state *st_,
-                                            uint8_t *c,
-                                            size_t clen_max,
-                                            size_t *written,
-                                            uint8_t *mac,
-                                            size_t maclen);
-int aegis128x4_state_encrypt_final(aegis128x4_state *st_,
-                                   uint8_t *c,
-                                   size_t clen_max,
-                                   size_t *written,
-                                   size_t maclen);
-int aegis128x4_state_decrypt_detached_update(aegis128x4_state *st_,
-                                             uint8_t *m,
-                                             size_t mlen_max,
-                                             size_t *written,
-                                             const uint8_t *c,
-                                             size_t clen) ;
-int aegis128x4_state_decrypt_detached_final(aegis128x4_state *st_,
-                                            uint8_t *m,
-                                            size_t mlen_max,
-                                            size_t *written,
-                                            const uint8_t *mac,
-                                            size_t maclen) ;
+int aegis128x4_state_encrypt_final(aegis128x4_state *st_, uint8_t *mac, size_t maclen);
+int aegis128x4_state_decrypt_update(aegis128x4_state *st_,
+                                    uint8_t *m,
+                                    const uint8_t *c,
+                                    size_t clen);
+int aegis128x4_state_decrypt_final(aegis128x4_state *st_, const uint8_t *mac, size_t maclen);
 void aegis128x4_stream(uint8_t *out, size_t len, const uint8_t *npub, const uint8_t *k);
+void aegis128x4_stream_xor(uint8_t *out,
+                           const uint8_t *in,
+                           size_t len,
+                           const uint8_t *npub,
+                           const uint8_t *k);
 void aegis128x4_encrypt_unauthenticated(uint8_t *c,
                                         const uint8_t *m,
                                         size_t mlen,
@@ -291,8 +240,8 @@ void aegis128x4_mac_reset(aegis128x4_mac_state *st_);
 void aegis128x4_mac_state_clone(aegis128x4_mac_state *dst, const aegis128x4_mac_state *src);
 
 /* aegis256.h */
-typedef struct aegis256_state { ...; } aegis256_state;
-typedef struct aegis256_mac_state { ...; } aegis256_mac_state;
+typedef struct aegis256_state { uint8_t opaque[192]; } aegis256_state;
+typedef struct aegis256_mac_state { uint8_t opaque[288]; } aegis256_mac_state;
 size_t aegis256_keybytes(void);
 size_t aegis256_npubbytes(void);
 size_t aegis256_abytes_min(void);
@@ -315,7 +264,7 @@ int aegis256_decrypt_detached(uint8_t *m,
                               const uint8_t *ad,
                               size_t adlen,
                               const uint8_t *npub,
-                              const uint8_t *k) ;
+                              const uint8_t *k);
 int aegis256_encrypt(uint8_t *c,
                      size_t maclen,
                      const uint8_t *m,
@@ -331,42 +280,22 @@ int aegis256_decrypt(uint8_t *m,
                      const uint8_t *ad,
                      size_t adlen,
                      const uint8_t *npub,
-                     const uint8_t *k) ;
+                     const uint8_t *k);
 void aegis256_state_init(aegis256_state *st_,
                          const uint8_t *ad,
                          size_t adlen,
                          const uint8_t *npub,
                          const uint8_t *k);
-int aegis256_state_encrypt_update(aegis256_state *st_,
-                                  uint8_t *c,
-                                  size_t clen_max,
-                                  size_t *written,
-                                  const uint8_t *m,
-                                  size_t mlen);
-int aegis256_state_encrypt_detached_final(aegis256_state *st_,
-                                          uint8_t *c,
-                                          size_t clen_max,
-                                          size_t *written,
-                                          uint8_t *mac,
-                                          size_t maclen);
-int aegis256_state_encrypt_final(aegis256_state *st_,
-                                 uint8_t *c,
-                                 size_t clen_max,
-                                 size_t *written,
-                                 size_t maclen);
-int aegis256_state_decrypt_detached_update(aegis256_state *st_,
-                                           uint8_t *m,
-                                           size_t mlen_max,
-                                           size_t *written,
-                                           const uint8_t *c,
-                                           size_t clen) ;
-int aegis256_state_decrypt_detached_final(aegis256_state *st_,
-                                          uint8_t *m,
-                                          size_t mlen_max,
-                                          size_t *written,
-                                          const uint8_t *mac,
-                                          size_t maclen) ;
+int aegis256_state_encrypt_update(aegis256_state *st_, uint8_t *c, const uint8_t *m, size_t mlen);
+int aegis256_state_encrypt_final(aegis256_state *st_, uint8_t *mac, size_t maclen);
+int aegis256_state_decrypt_update(aegis256_state *st_, uint8_t *m, const uint8_t *c, size_t clen);
+int aegis256_state_decrypt_final(aegis256_state *st_, const uint8_t *mac, size_t maclen);
 void aegis256_stream(uint8_t *out, size_t len, const uint8_t *npub, const uint8_t *k);
+void aegis256_stream_xor(uint8_t *out,
+                         const uint8_t *in,
+                         size_t len,
+                         const uint8_t *npub,
+                         const uint8_t *k);
 void aegis256_encrypt_unauthenticated(uint8_t *c,
                                       const uint8_t *m,
                                       size_t mlen,
@@ -385,8 +314,8 @@ void aegis256_mac_reset(aegis256_mac_state *st_);
 void aegis256_mac_state_clone(aegis256_mac_state *dst, const aegis256_mac_state *src);
 
 /* aegis256x2.h */
-typedef struct aegis256x2_state { ...; } aegis256x2_state;
-typedef struct aegis256x2_mac_state { ...; } aegis256x2_mac_state;
+typedef struct aegis256x2_state { uint8_t opaque[320]; } aegis256x2_state;
+typedef struct aegis256x2_mac_state { uint8_t opaque[512]; } aegis256x2_mac_state;
 size_t aegis256x2_keybytes(void);
 size_t aegis256x2_npubbytes(void);
 size_t aegis256x2_abytes_min(void);
@@ -409,7 +338,7 @@ int aegis256x2_decrypt_detached(uint8_t *m,
                                 const uint8_t *ad,
                                 size_t adlen,
                                 const uint8_t *npub,
-                                const uint8_t *k) ;
+                                const uint8_t *k);
 int aegis256x2_encrypt(uint8_t *c,
                        size_t maclen,
                        const uint8_t *m,
@@ -425,7 +354,7 @@ int aegis256x2_decrypt(uint8_t *m,
                        const uint8_t *ad,
                        size_t adlen,
                        const uint8_t *npub,
-                       const uint8_t *k) ;
+                       const uint8_t *k);
 void aegis256x2_state_init(aegis256x2_state *st_,
                            const uint8_t *ad,
                            size_t adlen,
@@ -433,34 +362,20 @@ void aegis256x2_state_init(aegis256x2_state *st_,
                            const uint8_t *k);
 int aegis256x2_state_encrypt_update(aegis256x2_state *st_,
                                     uint8_t *c,
-                                    size_t clen_max,
-                                    size_t *written,
                                     const uint8_t *m,
                                     size_t mlen);
-int aegis256x2_state_encrypt_detached_final(aegis256x2_state *st_,
-                                            uint8_t *c,
-                                            size_t clen_max,
-                                            size_t *written,
-                                            uint8_t *mac,
-                                            size_t maclen);
-int aegis256x2_state_encrypt_final(aegis256x2_state *st_,
-                                   uint8_t *c,
-                                   size_t clen_max,
-                                   size_t *written,
-                                   size_t maclen);
-int aegis256x2_state_decrypt_detached_update(aegis256x2_state *st_,
-                                             uint8_t *m,
-                                             size_t mlen_max,
-                                             size_t *written,
-                                             const uint8_t *c,
-                                             size_t clen) ;
-int aegis256x2_state_decrypt_detached_final(aegis256x2_state *st_,
-                                            uint8_t *m,
-                                            size_t mlen_max,
-                                            size_t *written,
-                                            const uint8_t *mac,
-                                            size_t maclen) ;
+int aegis256x2_state_encrypt_final(aegis256x2_state *st_, uint8_t *mac, size_t maclen);
+int aegis256x2_state_decrypt_update(aegis256x2_state *st_,
+                                    uint8_t *m,
+                                    const uint8_t *c,
+                                    size_t clen);
+int aegis256x2_state_decrypt_final(aegis256x2_state *st_, const uint8_t *mac, size_t maclen);
 void aegis256x2_stream(uint8_t *out, size_t len, const uint8_t *npub, const uint8_t *k);
+void aegis256x2_stream_xor(uint8_t *out,
+                           const uint8_t *in,
+                           size_t len,
+                           const uint8_t *npub,
+                           const uint8_t *k);
 void aegis256x2_encrypt_unauthenticated(uint8_t *c,
                                         const uint8_t *m,
                                         size_t mlen,
@@ -479,8 +394,8 @@ void aegis256x2_mac_reset(aegis256x2_mac_state *st_);
 void aegis256x2_mac_state_clone(aegis256x2_mac_state *dst, const aegis256x2_mac_state *src);
 
 /* aegis256x4.h */
-typedef struct aegis256x4_state { ...; } aegis256x4_state;
-typedef struct aegis256x4_mac_state { ...; } aegis256x4_mac_state;
+typedef struct aegis256x4_state { uint8_t opaque[576]; } aegis256x4_state;
+typedef struct aegis256x4_mac_state { uint8_t opaque[960]; } aegis256x4_mac_state;
 size_t aegis256x4_keybytes(void);
 size_t aegis256x4_npubbytes(void);
 size_t aegis256x4_abytes_min(void);
@@ -503,7 +418,7 @@ int aegis256x4_decrypt_detached(uint8_t *m,
                                 const uint8_t *ad,
                                 size_t adlen,
                                 const uint8_t *npub,
-                                const uint8_t *k) ;
+                                const uint8_t *k);
 int aegis256x4_encrypt(uint8_t *c,
                        size_t maclen,
                        const uint8_t *m,
@@ -519,7 +434,7 @@ int aegis256x4_decrypt(uint8_t *m,
                        const uint8_t *ad,
                        size_t adlen,
                        const uint8_t *npub,
-                       const uint8_t *k) ;
+                       const uint8_t *k);
 void aegis256x4_state_init(aegis256x4_state *st_,
                            const uint8_t *ad,
                            size_t adlen,
@@ -527,34 +442,20 @@ void aegis256x4_state_init(aegis256x4_state *st_,
                            const uint8_t *k);
 int aegis256x4_state_encrypt_update(aegis256x4_state *st_,
                                     uint8_t *c,
-                                    size_t clen_max,
-                                    size_t *written,
                                     const uint8_t *m,
                                     size_t mlen);
-int aegis256x4_state_encrypt_detached_final(aegis256x4_state *st_,
-                                            uint8_t *c,
-                                            size_t clen_max,
-                                            size_t *written,
-                                            uint8_t *mac,
-                                            size_t maclen);
-int aegis256x4_state_encrypt_final(aegis256x4_state *st_,
-                                   uint8_t *c,
-                                   size_t clen_max,
-                                   size_t *written,
-                                   size_t maclen);
-int aegis256x4_state_decrypt_detached_update(aegis256x4_state *st_,
-                                             uint8_t *m,
-                                             size_t mlen_max,
-                                             size_t *written,
-                                             const uint8_t *c,
-                                             size_t clen) ;
-int aegis256x4_state_decrypt_detached_final(aegis256x4_state *st_,
-                                            uint8_t *m,
-                                            size_t mlen_max,
-                                            size_t *written,
-                                            const uint8_t *mac,
-                                            size_t maclen) ;
+int aegis256x4_state_encrypt_final(aegis256x4_state *st_, uint8_t *mac, size_t maclen);
+int aegis256x4_state_decrypt_update(aegis256x4_state *st_,
+                                    uint8_t *m,
+                                    const uint8_t *c,
+                                    size_t clen);
+int aegis256x4_state_decrypt_final(aegis256x4_state *st_, const uint8_t *mac, size_t maclen);
 void aegis256x4_stream(uint8_t *out, size_t len, const uint8_t *npub, const uint8_t *k);
+void aegis256x4_stream_xor(uint8_t *out,
+                           const uint8_t *in,
+                           size_t len,
+                           const uint8_t *npub,
+                           const uint8_t *k);
 void aegis256x4_encrypt_unauthenticated(uint8_t *c,
                                         const uint8_t *m,
                                         size_t mlen,
@@ -571,3 +472,232 @@ int aegis256x4_mac_final(aegis256x4_mac_state *st_, uint8_t *mac, size_t maclen)
 int aegis256x4_mac_verify(aegis256x4_mac_state *st_, const uint8_t *mac, size_t maclen);
 void aegis256x4_mac_reset(aegis256x4_mac_state *st_);
 void aegis256x4_mac_state_clone(aegis256x4_mac_state *dst, const aegis256x4_mac_state *src);
+
+/* aegis_raf.h */
+typedef struct aegis_raf_scratch { uint8_t *buf; size_t len; } aegis_raf_scratch;
+typedef struct aegis_raf_io { void *user; int (*read_at)(void *user,
+                                               uint8_t *buf,
+                                               size_t len,
+                                               uint64_t off); int (*write_at)(void *user,
+                                               const uint8_t *buf,
+                                               size_t len,
+                                               uint64_t off); int (*get_size)(void *user,
+                                               uint64_t *size); int (*set_size)(void *user,
+                                               uint64_t size); int (*sync)(void *user); } aegis_raf_io;
+typedef struct aegis_raf_rng { void *user; int (*random)(void *user, uint8_t *out, size_t len); } aegis_raf_rng;
+typedef struct aegis_raf_merkle_config { int (*hash_leaf)(void *user,
+                                              uint8_t *out,
+                                              size_t out_len,
+                                              const uint8_t *chunk,
+                                              size_t chunk_len,
+                                              uint64_t chunk_idx); int (*hash_parent)(void *user,
+                                              uint8_t *out,
+                                              size_t out_len,
+                                              const uint8_t *left,
+                                              const uint8_t *right,
+                                              uint32_t level,
+                                              uint64_t node_idx); int (*hash_empty)(void *user,
+                                              uint8_t *out,
+                                              size_t out_len,
+                                              uint32_t level,
+                                              uint64_t node_idx); int (*hash_commitment)(void *user,
+                                              uint8_t *out,
+                                              size_t out_len,
+                                              const uint8_t *structural_root,
+                                              const uint8_t *ctx,
+                                              size_t ctx_len,
+                                              uint64_t file_size); void *user; uint8_t *buf; size_t len; uint64_t max_chunks; uint32_t hash_len; } aegis_raf_merkle_config;
+typedef struct aegis_raf_config { const aegis_raf_scratch *scratch; const aegis_raf_merkle_config *merkle; uint32_t chunk_size; uint8_t flags; } aegis_raf_config;
+typedef struct aegis_raf_info { uint64_t file_size; uint32_t chunk_size; uint8_t alg_id; } aegis_raf_info;
+typedef struct aegis128l_raf_ctx { uint8_t opaque[512]; } aegis128l_raf_ctx;
+typedef struct aegis128x2_raf_ctx { uint8_t opaque[512]; } aegis128x2_raf_ctx;
+typedef struct aegis128x4_raf_ctx { uint8_t opaque[512]; } aegis128x4_raf_ctx;
+typedef struct aegis256_raf_ctx { uint8_t opaque[512]; } aegis256_raf_ctx;
+typedef struct aegis256x2_raf_ctx { uint8_t opaque[512]; } aegis256x2_raf_ctx;
+typedef struct aegis256x4_raf_ctx { uint8_t opaque[512]; } aegis256x4_raf_ctx;
+size_t aegis_raf_chunk_min(void);
+size_t aegis_raf_chunk_max(void);
+size_t aegis_raf_header_size(void);
+size_t aegis_raf_scratch_align(void);
+int aegis_raf_probe(const aegis_raf_io *io, aegis_raf_info *info);
+size_t aegis128l_raf_scratch_size(uint32_t chunk_size);
+int aegis128l_raf_scratch_validate(const aegis_raf_scratch *scratch, uint32_t chunk_size);
+size_t aegis128x2_raf_scratch_size(uint32_t chunk_size);
+int aegis128x2_raf_scratch_validate(const aegis_raf_scratch *scratch, uint32_t chunk_size);
+size_t aegis128x4_raf_scratch_size(uint32_t chunk_size);
+int aegis128x4_raf_scratch_validate(const aegis_raf_scratch *scratch, uint32_t chunk_size);
+size_t aegis256_raf_scratch_size(uint32_t chunk_size);
+int aegis256_raf_scratch_validate(const aegis_raf_scratch *scratch, uint32_t chunk_size);
+size_t aegis256x2_raf_scratch_size(uint32_t chunk_size);
+int aegis256x2_raf_scratch_validate(const aegis_raf_scratch *scratch, uint32_t chunk_size);
+size_t aegis256x4_raf_scratch_size(uint32_t chunk_size);
+int aegis256x4_raf_scratch_validate(const aegis_raf_scratch *scratch, uint32_t chunk_size);
+size_t aegis_raf_merkle_buffer_size(const aegis_raf_merkle_config *cfg);
+int aegis_raf_derive_master_key(uint8_t *out,
+                                size_t out_len,
+                                const uint8_t *master_key,
+                                size_t master_key_len,
+                                const uint8_t *context,
+                                size_t context_len);
+int aegis128l_raf_create(aegis128l_raf_ctx *ctx,
+                         const aegis_raf_io *io,
+                         const aegis_raf_rng *rng,
+                         const aegis_raf_config *cfg,
+                         const uint8_t *master_key);
+int aegis128l_raf_open(aegis128l_raf_ctx *ctx,
+                       const aegis_raf_io *io,
+                       const aegis_raf_rng *rng,
+                       const aegis_raf_config *cfg,
+                       const uint8_t *master_key);
+int aegis128l_raf_read(aegis128l_raf_ctx *ctx,
+                       uint8_t *out,
+                       size_t *bytes_read,
+                       size_t len,
+                       uint64_t offset);
+int aegis128l_raf_write(aegis128l_raf_ctx *ctx,
+                        size_t *bytes_written,
+                        const uint8_t *in,
+                        size_t len,
+                        uint64_t offset);
+int aegis128l_raf_truncate(aegis128l_raf_ctx *ctx, uint64_t size);
+int aegis128l_raf_get_size(const aegis128l_raf_ctx *ctx, uint64_t *size);
+int aegis128l_raf_sync(aegis128l_raf_ctx *ctx);
+void aegis128l_raf_close(aegis128l_raf_ctx *ctx);
+int aegis128l_raf_merkle_rebuild(aegis128l_raf_ctx *ctx);
+int aegis128l_raf_merkle_verify(aegis128l_raf_ctx *ctx, uint64_t *corrupted_chunk);
+int aegis128l_raf_merkle_commitment(const aegis128l_raf_ctx *ctx, uint8_t *out, size_t out_len);
+int aegis128x2_raf_create(aegis128x2_raf_ctx *ctx,
+                          const aegis_raf_io *io,
+                          const aegis_raf_rng *rng,
+                          const aegis_raf_config *cfg,
+                          const uint8_t *master_key);
+int aegis128x2_raf_open(aegis128x2_raf_ctx *ctx,
+                        const aegis_raf_io *io,
+                        const aegis_raf_rng *rng,
+                        const aegis_raf_config *cfg,
+                        const uint8_t *master_key);
+int aegis128x2_raf_read(aegis128x2_raf_ctx *ctx,
+                        uint8_t *out,
+                        size_t *bytes_read,
+                        size_t len,
+                        uint64_t offset);
+int aegis128x2_raf_write(aegis128x2_raf_ctx *ctx,
+                         size_t *bytes_written,
+                         const uint8_t *in,
+                         size_t len,
+                         uint64_t offset);
+int aegis128x2_raf_truncate(aegis128x2_raf_ctx *ctx, uint64_t size);
+int aegis128x2_raf_get_size(const aegis128x2_raf_ctx *ctx, uint64_t *size);
+int aegis128x2_raf_sync(aegis128x2_raf_ctx *ctx);
+void aegis128x2_raf_close(aegis128x2_raf_ctx *ctx);
+int aegis128x2_raf_merkle_rebuild(aegis128x2_raf_ctx *ctx);
+int aegis128x2_raf_merkle_verify(aegis128x2_raf_ctx *ctx, uint64_t *corrupted_chunk);
+int aegis128x2_raf_merkle_commitment(const aegis128x2_raf_ctx *ctx, uint8_t *out, size_t out_len);
+int aegis128x4_raf_create(aegis128x4_raf_ctx *ctx,
+                          const aegis_raf_io *io,
+                          const aegis_raf_rng *rng,
+                          const aegis_raf_config *cfg,
+                          const uint8_t *master_key);
+int aegis128x4_raf_open(aegis128x4_raf_ctx *ctx,
+                        const aegis_raf_io *io,
+                        const aegis_raf_rng *rng,
+                        const aegis_raf_config *cfg,
+                        const uint8_t *master_key);
+int aegis128x4_raf_read(aegis128x4_raf_ctx *ctx,
+                        uint8_t *out,
+                        size_t *bytes_read,
+                        size_t len,
+                        uint64_t offset);
+int aegis128x4_raf_write(aegis128x4_raf_ctx *ctx,
+                         size_t *bytes_written,
+                         const uint8_t *in,
+                         size_t len,
+                         uint64_t offset);
+int aegis128x4_raf_truncate(aegis128x4_raf_ctx *ctx, uint64_t size);
+int aegis128x4_raf_get_size(const aegis128x4_raf_ctx *ctx, uint64_t *size);
+int aegis128x4_raf_sync(aegis128x4_raf_ctx *ctx);
+void aegis128x4_raf_close(aegis128x4_raf_ctx *ctx);
+int aegis128x4_raf_merkle_rebuild(aegis128x4_raf_ctx *ctx);
+int aegis128x4_raf_merkle_verify(aegis128x4_raf_ctx *ctx, uint64_t *corrupted_chunk);
+int aegis128x4_raf_merkle_commitment(const aegis128x4_raf_ctx *ctx, uint8_t *out, size_t out_len);
+int aegis256_raf_create(aegis256_raf_ctx *ctx,
+                        const aegis_raf_io *io,
+                        const aegis_raf_rng *rng,
+                        const aegis_raf_config *cfg,
+                        const uint8_t *master_key);
+int aegis256_raf_open(aegis256_raf_ctx *ctx,
+                      const aegis_raf_io *io,
+                      const aegis_raf_rng *rng,
+                      const aegis_raf_config *cfg,
+                      const uint8_t *master_key);
+int aegis256_raf_read(aegis256_raf_ctx *ctx,
+                      uint8_t *out,
+                      size_t *bytes_read,
+                      size_t len,
+                      uint64_t offset);
+int aegis256_raf_write(aegis256_raf_ctx *ctx,
+                       size_t *bytes_written,
+                       const uint8_t *in,
+                       size_t len,
+                       uint64_t offset);
+int aegis256_raf_truncate(aegis256_raf_ctx *ctx, uint64_t size);
+int aegis256_raf_get_size(const aegis256_raf_ctx *ctx, uint64_t *size);
+int aegis256_raf_sync(aegis256_raf_ctx *ctx);
+void aegis256_raf_close(aegis256_raf_ctx *ctx);
+int aegis256_raf_merkle_rebuild(aegis256_raf_ctx *ctx);
+int aegis256_raf_merkle_verify(aegis256_raf_ctx *ctx, uint64_t *corrupted_chunk);
+int aegis256_raf_merkle_commitment(const aegis256_raf_ctx *ctx, uint8_t *out, size_t out_len);
+int aegis256x2_raf_create(aegis256x2_raf_ctx *ctx,
+                          const aegis_raf_io *io,
+                          const aegis_raf_rng *rng,
+                          const aegis_raf_config *cfg,
+                          const uint8_t *master_key);
+int aegis256x2_raf_open(aegis256x2_raf_ctx *ctx,
+                        const aegis_raf_io *io,
+                        const aegis_raf_rng *rng,
+                        const aegis_raf_config *cfg,
+                        const uint8_t *master_key);
+int aegis256x2_raf_read(aegis256x2_raf_ctx *ctx,
+                        uint8_t *out,
+                        size_t *bytes_read,
+                        size_t len,
+                        uint64_t offset);
+int aegis256x2_raf_write(aegis256x2_raf_ctx *ctx,
+                         size_t *bytes_written,
+                         const uint8_t *in,
+                         size_t len,
+                         uint64_t offset);
+int aegis256x2_raf_truncate(aegis256x2_raf_ctx *ctx, uint64_t size);
+int aegis256x2_raf_get_size(const aegis256x2_raf_ctx *ctx, uint64_t *size);
+int aegis256x2_raf_sync(aegis256x2_raf_ctx *ctx);
+void aegis256x2_raf_close(aegis256x2_raf_ctx *ctx);
+int aegis256x2_raf_merkle_rebuild(aegis256x2_raf_ctx *ctx);
+int aegis256x2_raf_merkle_verify(aegis256x2_raf_ctx *ctx, uint64_t *corrupted_chunk);
+int aegis256x2_raf_merkle_commitment(const aegis256x2_raf_ctx *ctx, uint8_t *out, size_t out_len);
+int aegis256x4_raf_create(aegis256x4_raf_ctx *ctx,
+                          const aegis_raf_io *io,
+                          const aegis_raf_rng *rng,
+                          const aegis_raf_config *cfg,
+                          const uint8_t *master_key);
+int aegis256x4_raf_open(aegis256x4_raf_ctx *ctx,
+                        const aegis_raf_io *io,
+                        const aegis_raf_rng *rng,
+                        const aegis_raf_config *cfg,
+                        const uint8_t *master_key);
+int aegis256x4_raf_read(aegis256x4_raf_ctx *ctx,
+                        uint8_t *out,
+                        size_t *bytes_read,
+                        size_t len,
+                        uint64_t offset);
+int aegis256x4_raf_write(aegis256x4_raf_ctx *ctx,
+                         size_t *bytes_written,
+                         const uint8_t *in,
+                         size_t len,
+                         uint64_t offset);
+int aegis256x4_raf_truncate(aegis256x4_raf_ctx *ctx, uint64_t size);
+int aegis256x4_raf_get_size(const aegis256x4_raf_ctx *ctx, uint64_t *size);
+int aegis256x4_raf_sync(aegis256x4_raf_ctx *ctx);
+void aegis256x4_raf_close(aegis256x4_raf_ctx *ctx);
+int aegis256x4_raf_merkle_rebuild(aegis256x4_raf_ctx *ctx);
+int aegis256x4_raf_merkle_verify(aegis256x4_raf_ctx *ctx, uint64_t *corrupted_chunk);
+int aegis256x4_raf_merkle_commitment(const aegis256x4_raf_ctx *ctx, uint8_t *out, size_t out_len);

src/aeg/benchmark.py

@@ -1,4 +1,4 @@
-#!/usr/bin/env python3
+#!/usr/bin/env -S uv run
 """
 Python benchmark matching src/test/benchmark.zig for all supported Aegis algorithms.
 
@@ -7,8 +7,19 @@ It performs two benchmarks with the same parameters as the Zig version:
 - AEGIS MAC (clone state pattern)
 
 Output format and throughput units mirror the Zig benchmark (Mb/s).
+
+Runnable directly (./src/aeg/benchmark.py) or as python -m aeg.benchmark.
 """
 
+# Direct execution puts this package's directory on sys.path, where our
+# random.py would shadow the stdlib random module — drop it before imports.
+import sys
+from pathlib import Path
+
+_pkg_dir = str(Path(__file__).resolve().parent)
+if sys.path and Path(sys.path[0]).resolve() == Path(_pkg_dir):
+    sys.path.pop(0)
+
 import secrets
 import time
 

src/aeg/raf.py

@@ -0,0 +1,984 @@
+"""RAF (random-access encrypted file) high-level API.
+
+Provides pread/pwrite-style access to encrypted files. Files are divided into
+fixed-size chunks, each independently encrypted with a fresh nonce, enabling
+efficient random access without decrypting the whole file.
+
+Use create() / open() with a cipher module (from aeg.cipher()) and a path,
+an io.BytesIO, or a Storage instance.
+
+Portions of the storage/callback/Merkle/file-like machinery are adapted from
+pyaegis (https://github.com/jedisct1/pyaegis) by Frank Denis, MIT license.
+"""
+
+from __future__ import annotations
+
+import errno as errno_module
+import hashlib
+import io as io_module
+import os
+import threading
+from typing import NamedTuple, Protocol, Self, runtime_checkable
+
+from . import cipher as _cipher_for_name
+from . import random as _random
+from ._loader import ffi, lib
+from ._typing import Cipher
+from .util import Buffer
+
+__all__ = [
+    "CHUNK_MIN",
+    "CHUNK_MAX",
+    "HEADER_SIZE",
+    "SCRATCH_ALIGN",
+    "Storage",
+    "FileStorage",
+    "BytesIOStorage",
+    "StreamStorage",
+    "MerkleHasher",
+    "SHA256MerkleHasher",
+    "RafInfo",
+    "Raf",
+    "create",
+    "open",
+    "probe",
+    "derive_master_key",
+]
+
+CHUNK_MIN = int(lib.aegis_raf_chunk_min())  #: Minimum chunk size in bytes
+CHUNK_MAX = int(lib.aegis_raf_chunk_max())  #: Maximum chunk size in bytes
+HEADER_SIZE = int(lib.aegis_raf_header_size())  #: Encrypted file header size
+SCRATCH_ALIGN = int(lib.aegis_raf_scratch_align())  #: Required scratch buffer alignment
+
+# From aegis_raf.h (preprocessor macros, not available through cdef)
+_FLAG_CREATE = 0x01
+_FLAG_TRUNCATE = 0x02
+_MERKLE_HASH_MIN = 8
+_MERKLE_HASH_MAX = 64
+
+
+@runtime_checkable
+class Storage(Protocol):
+    """Backing storage for RAF files. Operations must complete fully or raise OSError."""
+
+    def read_at(self, buf: bytearray, offset: int) -> None:
+        """Read exactly len(buf) bytes at offset into buf."""
+        ...
+
+    def write_at(self, data: bytes, offset: int) -> None:
+        """Write exactly len(data) bytes at offset."""
+        ...
+
+    def get_size(self) -> int:
+        """Return current storage size in bytes."""
+        ...
+
+    def set_size(self, size: int) -> None:
+        """Resize storage (truncate or extend)."""
+        ...
+
+    def sync(self) -> None:
+        """Flush writes to durable storage. May be a no-op."""
+        ...
+
+
+class FileStorage:
+    """File-based storage.
+
+    Uses os.pread/os.pwrite where available (Unix); otherwise falls back to
+    a lock plus os.lseek/os.read/os.write (e.g. on Windows).
+    """
+
+    def __init__(self, path: str | os.PathLike, mode: str = "r+b"):
+        """Open a file for RAF storage.
+
+        Args:
+            path: Path to the file.
+            mode: "r+b" to open existing, "w+b" or "x+b" to create.
+        """
+        if mode == "r+b":
+            flags = os.O_RDWR
+        elif mode == "w+b":
+            flags = os.O_RDWR | os.O_CREAT | os.O_TRUNC
+        elif mode == "x+b":
+            flags = os.O_RDWR | os.O_CREAT | os.O_EXCL
+        else:
+            raise ValueError(f"Unsupported mode: {mode}")
+        # O_BINARY (Windows only): keep the CRT from translating \r\n in
+        # binary data, which corrupts ciphertext and shifts offsets
+        flags |= getattr(os, "O_BINARY", 0)
+        self._fd = os.open(path, flags, 0o644)
+        self._closed = False
+        self._lock = threading.Lock()
+        self._positional = hasattr(os, "pread") and hasattr(os, "pwrite")
+
+    def read_at(self, buf: bytearray, offset: int) -> None:
+        """Read exactly len(buf) bytes. Raises OSError on short read."""
+        view = memoryview(buf)
+        if self._positional:
+            done = 0
+            while done < len(view):
+                chunk = os.pread(self._fd, len(view) - done, offset + done)
+                if not chunk:
+                    raise OSError(f"Short read: expected {len(view)}, got {done}")
+                view[done : done + len(chunk)] = chunk
+                done += len(chunk)
+        else:
+            with self._lock:
+                os.lseek(self._fd, offset, os.SEEK_SET)
+                done = 0
+                while done < len(view):
+                    chunk = os.read(self._fd, len(view) - done)
+                    if not chunk:
+                        raise OSError(f"Short read: expected {len(view)}, got {done}")
+                    view[done : done + len(chunk)] = chunk
+                    done += len(chunk)
+
+    def write_at(self, data: bytes, offset: int) -> None:
+        """Write exactly len(data) bytes."""
+        view = memoryview(data)
+        if self._positional:
+            done = 0
+            while done < len(view):
+                done += os.pwrite(self._fd, view[done:], offset + done)
+        else:
+            with self._lock:
+                os.lseek(self._fd, offset, os.SEEK_SET)
+                done = 0
+                while done < len(view):
+                    done += os.write(self._fd, view[done:])
+
+    def get_size(self) -> int:
+        """Return current file size."""
+        return os.fstat(self._fd).st_size
+
+    def set_size(self, size: int) -> None:
+        """Truncate or extend the file to the given size."""
+        os.ftruncate(self._fd, size)
+
+    def sync(self) -> None:
+        """Flush file data to disk."""
+        os.fsync(self._fd)
+
+    def close(self) -> None:
+        """Close the underlying file descriptor."""
+        if not self._closed:
+            os.close(self._fd)
+            self._closed = True
+
+    def __enter__(self) -> Self:
+        return self
+
+    def __exit__(self, *args) -> None:
+        self.close()
+
+
+class BytesIOStorage:
+    """In-memory storage backed by a bytearray. Useful for tests."""
+
+    def __init__(self, initial: bytes = b""):
+        self._data = bytearray(initial)
+
+    def read_at(self, buf: bytearray, offset: int) -> None:
+        """Read exactly len(buf) bytes at offset into buf."""
+        end = offset + len(buf)
+        if end > len(self._data):
+            raise OSError(
+                f"Read past EOF: offset={offset}, len={len(buf)}, size={len(self._data)}"
+            )
+        buf[:] = self._data[offset:end]
+
+    def write_at(self, data: bytes, offset: int) -> None:
+        """Write exactly len(data) bytes at offset."""
+        if not data:
+            return
+        end = offset + len(data)
+        if end > len(self._data):
+            self._data.extend(b"\x00" * (end - len(self._data)))
+        self._data[offset:end] = data
+
+    def get_size(self) -> int:
+        """Return current buffer size."""
+        return len(self._data)
+
+    def set_size(self, size: int) -> None:
+        """Resize the buffer (truncate or extend with zeros)."""
+        current = len(self._data)
+        if size < current:
+            del self._data[size:]
+        elif size > current:
+            self._data.extend(b"\x00" * (size - current))
+
+    def sync(self) -> None:
+        """No-op for in-memory storage."""
+
+    def getvalue(self) -> bytes:
+        """Return the entire buffer contents."""
+        return bytes(self._data)
+
+    def __enter__(self) -> Self:
+        return self
+
+    def __exit__(self, *args) -> None:
+        pass
+
+
+class StreamStorage:
+    """Storage adapter for seek-based file-like objects (io.BytesIO, open files).
+
+    Uses a lock plus seek/read/write, mirroring FileStorage's fallback path
+    for platforms without os.pread/os.pwrite. The stream is not closed by
+    Raf; closing it remains the caller's responsibility.
+    """
+
+    def __init__(self, stream):
+        self._stream = stream
+        self._lock = threading.Lock()
+
+    def read_at(self, buf: bytearray, offset: int) -> None:
+        """Read exactly len(buf) bytes at offset into buf."""
+        with self._lock:
+            self._stream.seek(offset)
+            view = memoryview(buf)
+            done = 0
+            while done < len(view):
+                chunk = self._stream.read(len(view) - done)
+                if not chunk:
+                    raise OSError(f"Short read: expected {len(view)}, got {done}")
+                view[done : done + len(chunk)] = chunk
+                done += len(chunk)
+
+    def write_at(self, data: bytes, offset: int) -> None:
+        """Write exactly len(data) bytes at offset."""
+        with self._lock:
+            self._stream.seek(offset)
+            view = memoryview(data)
+            done = 0
+            while done < len(view):
+                done += self._stream.write(view[done:])
+
+    def get_size(self) -> int:
+        """Return current stream size."""
+        with self._lock:
+            pos = self._stream.tell()
+            try:
+                return self._stream.seek(0, os.SEEK_END)
+            finally:
+                self._stream.seek(pos)
+
+    def set_size(self, size: int) -> None:
+        """Resize the stream (truncate or extend with zeros)."""
+        self._stream.truncate(size)
+
+    def sync(self) -> None:
+        """Flush the stream if it supports flushing."""
+        if hasattr(self._stream, "flush"):
+            self._stream.flush()
+
+    def __enter__(self) -> Self:
+        return self
+
+    def __exit__(self, *args) -> None:
+        pass
+
+
+@runtime_checkable
+class MerkleHasher(Protocol):
+    """Protocol for Merkle tree hash functions."""
+
+    @property
+    def hash_len(self) -> int: ...
+
+    def hash_leaf(self, chunk: bytes, chunk_len: int, chunk_idx: int) -> bytes: ...
+    def hash_parent(
+        self, left: bytes, right: bytes, level: int, node_idx: int
+    ) -> bytes: ...
+    def hash_empty(self, level: int, node_idx: int) -> bytes: ...
+    def hash_commitment(
+        self, structural_root: bytes, ctx: bytes, file_size: int
+    ) -> bytes: ...
+
+
+class SHA256MerkleHasher:
+    """Default Merkle hasher using SHA-256 with domain-separated prefixes."""
+
+    hash_len = 32
+
+    def hash_leaf(self, chunk: bytes, chunk_len: int, chunk_idx: int) -> bytes:
+        h = hashlib.sha256()
+        h.update(b"\x00")
+        h.update(chunk_idx.to_bytes(8, "little"))
+        h.update(chunk[:chunk_len])
+        return h.digest()
+
+    def hash_parent(
+        self, left: bytes, right: bytes, level: int, node_idx: int
+    ) -> bytes:
+        h = hashlib.sha256()
+        h.update(b"\x01")
+        h.update(level.to_bytes(4, "little"))
+        h.update(node_idx.to_bytes(8, "little"))
+        h.update(left)
+        h.update(right)
+        return h.digest()
+
+    def hash_empty(self, level: int, node_idx: int) -> bytes:
+        h = hashlib.sha256()
+        h.update(b"\x02")
+        h.update(level.to_bytes(4, "little"))
+        h.update(node_idx.to_bytes(8, "little"))
+        return h.digest()
+
+    def hash_commitment(
+        self, structural_root: bytes, ctx: bytes, file_size: int
+    ) -> bytes:
+        h = hashlib.sha256()
+        h.update(b"\x03")
+        h.update(structural_root)
+        h.update(ctx)
+        h.update(file_size.to_bytes(8, "little"))
+        return h.digest()
+
+
+# CFFI callbacks (module-level so they stay alive)
+
+
+@ffi.callback("int(void*, uint8_t*, size_t, uint64_t)")
+def _read_at_callback(user, buf, length, offset):
+    try:
+        storage = ffi.from_handle(user)
+        temp = bytearray(length)
+        storage.read_at(temp, offset)
+        ffi.buffer(buf, length)[:] = temp
+        return 0
+    except Exception:
+        ffi.errno = errno_module.EIO
+        return -1
+
+
+@ffi.callback("int(void*, const uint8_t*, size_t, uint64_t)")
+def _write_at_callback(user, buf, length, offset):
+    try:
+        storage = ffi.from_handle(user)
+        storage.write_at(bytes(ffi.buffer(buf, length)), offset)
+        return 0
+    except Exception:
+        ffi.errno = errno_module.EIO
+        return -1
+
+
+@ffi.callback("int(void*, uint64_t*)")
+def _get_size_callback(user, size_ptr):
+    try:
+        size_ptr[0] = ffi.from_handle(user).get_size()
+        return 0
+    except Exception:
+        ffi.errno = errno_module.EIO
+        return -1
+
+
+@ffi.callback("int(void*, uint64_t)")
+def _set_size_callback(user, size):
+    try:
+        ffi.from_handle(user).set_size(size)
+        return 0
+    except Exception:
+        ffi.errno = errno_module.EIO
+        return -1
+
+
+@ffi.callback("int(void*)")
+def _sync_callback(user):
+    try:
+        ffi.from_handle(user).sync()
+        return 0
+    except Exception:
+        ffi.errno = errno_module.EIO
+        return -1
+
+
+@ffi.callback("int(void*, uint8_t*, size_t, const uint8_t*, size_t, uint64_t)")
+def _hash_leaf_callback(user, out, out_len, chunk, chunk_len, chunk_idx):
+    try:
+        hasher = ffi.from_handle(user)
+        digest = hasher.hash_leaf(
+            bytes(ffi.buffer(chunk, chunk_len)), chunk_len, chunk_idx
+        )
+        if len(digest) != out_len:
+            ffi.errno = errno_module.EINVAL
+            return -1
+        ffi.memmove(out, digest, out_len)
+        return 0
+    except Exception:
+        ffi.errno = errno_module.EINVAL
+        return -1
+
+
+@ffi.callback(
+    "int(void*, uint8_t*, size_t, const uint8_t*, const uint8_t*, uint32_t, uint64_t)"
+)
+def _hash_parent_callback(user, out, out_len, left, right, level, node_idx):
+    try:
+        hasher = ffi.from_handle(user)
+        digest = hasher.hash_parent(
+            bytes(ffi.buffer(left, out_len)),
+            bytes(ffi.buffer(right, out_len)),
+            level,
+            node_idx,
+        )
+        if len(digest) != out_len:
+            ffi.errno = errno_module.EINVAL
+            return -1
+        ffi.memmove(out, digest, out_len)
+        return 0
+    except Exception:
+        ffi.errno = errno_module.EINVAL
+        return -1
+
+
+@ffi.callback("int(void*, uint8_t*, size_t, uint32_t, uint64_t)")
+def _hash_empty_callback(user, out, out_len, level, node_idx):
+    try:
+        hasher = ffi.from_handle(user)
+        digest = hasher.hash_empty(level, node_idx)
+        if len(digest) != out_len:
+            ffi.errno = errno_module.EINVAL
+            return -1
+        ffi.memmove(out, digest, out_len)
+        return 0
+    except Exception:
+        ffi.errno = errno_module.EINVAL
+        return -1
+
+
+@ffi.callback(
+    "int(void*, uint8_t*, size_t, const uint8_t*, const uint8_t*, size_t, uint64_t)"
+)
+def _hash_commitment_callback(
+    user, out, out_len, structural_root, ctx, ctx_len, file_size
+):
+    try:
+        hasher = ffi.from_handle(user)
+        root_bytes = bytes(ffi.buffer(structural_root, out_len))
+        ctx_bytes = bytes(ffi.buffer(ctx, ctx_len)) if ctx != ffi.NULL else b""
+        digest = hasher.hash_commitment(root_bytes, ctx_bytes, file_size)
+        if len(digest) != out_len:
+            ffi.errno = errno_module.EINVAL
+            return -1
+        ffi.memmove(out, digest, out_len)
+        return 0
+    except Exception:
+        ffi.errno = errno_module.EINVAL
+        return -1
+
+
+def _make_io(storage: Storage):
+    """Build an aegis_raf_io* cdata for a Storage. Returns (io, handle)."""
+    handle = ffi.new_handle(storage)
+    io = ffi.new("aegis_raf_io*")
+    io.user = handle
+    io.read_at = _read_at_callback
+    io.write_at = _write_at_callback
+    io.get_size = _get_size_callback
+    io.set_size = _set_size_callback
+    io.sync = _sync_callback
+    return io, handle
+
+
+def _allocate_scratch(cipher: Cipher, chunk_size: int):
+    """Allocate an aligned aegis_raf_scratch*. Returns (raw, scratch)."""
+    size = cipher.raf_scratch_size(chunk_size)
+    raw = bytearray(size + SCRATCH_ALIGN)
+    raw_ptr = ffi.from_buffer(raw)
+    offset = (-int(ffi.cast("uintptr_t", raw_ptr))) & (SCRATCH_ALIGN - 1)
+    scratch = ffi.new("aegis_raf_scratch*")
+    scratch.buf = ffi.cast("uint8_t*", raw_ptr) + offset
+    scratch.len = size
+    return raw, scratch
+
+
+def derive_master_key(master_key: Buffer, context: Buffer = b"") -> bytes:
+    """Derive a context-bound RAF master key from an application master key.
+
+    Args:
+        master_key: A 16-byte or 32-byte application master key.
+        context: A public identifier for a file or file family
+                 (at most 120 bytes for 16-byte keys, 72 for 32-byte keys).
+
+    Returns:
+        A RAF-scoped key of the same length as master_key.
+
+    Raises:
+        ValueError: If the key or context length is invalid.
+        RuntimeError: If the C KDF rejects validated inputs.
+    """
+    master_key = memoryview(master_key)
+    context = memoryview(context)
+    key_size = master_key.nbytes
+    if key_size not in (16, 32):
+        raise ValueError(f"master_key must be 16 or 32 bytes, got {key_size}")
+    max_context = 120 if key_size == 16 else 72
+    if context.nbytes > max_context:
+        raise ValueError(
+            f"context must be at most {max_context} bytes for a {key_size}-byte key"
+        )
+    out = bytearray(key_size)
+    rc = lib.aegis_raf_derive_master_key(
+        ffi.from_buffer(out),
+        key_size,
+        bytes(master_key),
+        key_size,
+        bytes(context),
+        context.nbytes,
+    )
+    if rc != 0:
+        err_num = ffi.errno
+        err_name = errno_module.errorcode.get(err_num, f"errno_{err_num}")
+        raise RuntimeError(f"derive master key failed: {err_name}")
+    return bytes(out)
+
+
+class RafInfo(NamedTuple):
+    """Unverified file metadata from probe()."""
+
+    alg_id: int
+    chunk_size: int
+    file_size: int
+
+
+def probe(storage: Storage) -> RafInfo:
+    """Probe an encrypted file to determine its parameters.
+
+    WARNING: the returned values are read from the header WITHOUT
+    cryptographic verification. Only use them for scratch sizing and
+    algorithm selection; open() verifies the header MAC with the key.
+
+    Returns:
+        RafInfo(alg_id, chunk_size, file_size).
+
+    Raises:
+        RuntimeError: Invalid header or I/O failure.
+    """
+    io, _handle = _make_io(storage)
+    info = ffi.new("aegis_raf_info*")
+    if lib.aegis_raf_probe(io, info) != 0:
+        err_num = ffi.errno
+        err_name = errno_module.errorcode.get(err_num, f"errno_{err_num}")
+        raise RuntimeError(f"probe failed: {err_name}")
+    return RafInfo(int(info.alg_id), int(info.chunk_size), int(info.file_size))
+
+
+def _as_storage(storage_or_path, mode: str) -> tuple[Storage, bool]:
+    if isinstance(storage_or_path, (str, bytes, os.PathLike)):
+        return FileStorage(storage_or_path, mode), True
+    if isinstance(storage_or_path, io_module.BytesIO):
+        return StreamStorage(storage_or_path), False
+    return storage_or_path, False
+
+
+def _resolve_cipher(cipher) -> Cipher:
+    if isinstance(cipher, str):
+        return _cipher_for_name(cipher)
+    return cipher
+
+
+def create(
+    storage_or_path,
+    key: Buffer,
+    cipher,
+    *,
+    chunk_size: int = 65536,
+    truncate: bool = False,
+    merkle: bool | MerkleHasher = False,
+    merkle_max_chunks: int = 16384,
+    rng: _random.RandomSource | None = None,
+) -> Raf:
+    """Create a new encrypted file.
+
+    Args:
+        storage_or_path: A Storage instance or a path (opened "x+b", or "w+b"
+                         when truncate=True).
+        key: Master key (cipher.KEYBYTES bytes).
+        cipher: A cipher module (from aeg.cipher()) or algorithm name.
+        chunk_size: Plaintext bytes per chunk (CHUNK_MIN..CHUNK_MAX, multiple of 16).
+        truncate: Overwrite an existing file.
+        merkle: Enable Merkle tree. True uses SHA256MerkleHasher, or pass a
+                custom MerkleHasher instance.
+        merkle_max_chunks: Maximum number of chunks the Merkle tree can track.
+        rng: Random source (see aeg.random). Default: a fresh aeg.random.Random
+            seeded for the same cipher module passed as cipher=; any callable(n)
+            returning bytes overrides.
+
+    Raises:
+        TypeError: If the key length is invalid.
+        ValueError: If chunk_size or merkle parameters are invalid.
+        FileExistsError: If the file exists and truncate is False.
+        RuntimeError: If creation fails.
+    """
+    cipher = _resolve_cipher(cipher)
+    if not (CHUNK_MIN <= chunk_size <= CHUNK_MAX):
+        raise ValueError(
+            f"chunk_size must be {CHUNK_MIN}-{CHUNK_MAX}, got {chunk_size}"
+        )
+    if chunk_size % 16 != 0:
+        raise ValueError(f"chunk_size must be a multiple of 16, got {chunk_size}")
+    storage, owns = _as_storage(storage_or_path, "w+b" if truncate else "x+b")
+    return Raf(
+        storage,
+        key,
+        cipher,
+        owns_storage=owns,
+        creating=True,
+        chunk_size=chunk_size,
+        truncate=truncate,
+        merkle=merkle,
+        merkle_max_chunks=merkle_max_chunks,
+        rng=rng,
+    )
+
+
+def open(
+    storage_or_path,
+    key: Buffer,
+    cipher,
+    *,
+    merkle: bool | MerkleHasher = False,
+    merkle_max_chunks: int = 16384,
+    rng: _random.RandomSource | None = None,
+) -> Raf:
+    """Open an existing encrypted file.
+
+    The header is probed to size internal buffers and its alg_id is checked
+    against cipher.RAF_ALG_ID; a mismatch raises ValueError (the algorithm is
+    never switched silently).
+
+    Args:
+        storage_or_path: A Storage instance or a path (opened "r+b").
+        key: Master key (cipher.KEYBYTES bytes).
+        cipher: A cipher module (from aeg.cipher()) or algorithm name.
+        merkle: Enable Merkle tree (call merkle_rebuild() before use).
+        merkle_max_chunks: Maximum number of chunks the Merkle tree can track.
+        rng: Random source (see aeg.random). Default: a fresh aeg.random.Random
+            seeded for the same cipher module passed as cipher=; any callable(n)
+            returning bytes overrides.
+
+    Raises:
+        TypeError: If the key length is invalid.
+        ValueError: If the algorithm doesn't match cipher or authentication fails.
+        FileNotFoundError: If the file does not exist.
+        RuntimeError: If opening fails.
+    """
+    cipher = _resolve_cipher(cipher)
+    storage, owns = _as_storage(storage_or_path, "r+b")
+    info = probe(storage)
+    if info.alg_id != cipher.RAF_ALG_ID:
+        raise ValueError(
+            f"file uses algorithm id {info.alg_id}, "
+            f"expected {cipher.RAF_ALG_ID} ({cipher.NAME})"
+        )
+    return Raf(
+        storage,
+        key,
+        cipher,
+        owns_storage=owns,
+        creating=False,
+        chunk_size=info.chunk_size,
+        truncate=False,
+        merkle=merkle,
+        merkle_max_chunks=merkle_max_chunks,
+        rng=rng,
+    )
+
+
+class Raf:
+    """File-like random-access encrypted file. Use raf.create() / raf.open()."""
+
+    def __init__(
+        self,
+        storage: Storage,
+        key: Buffer,
+        cipher: Cipher,
+        *,
+        owns_storage: bool,
+        creating: bool,
+        chunk_size: int,
+        truncate: bool,
+        merkle: bool | MerkleHasher,
+        merkle_max_chunks: int,
+        rng,
+    ):
+        self._cipher = cipher
+        self._storage = storage
+        self._owns_storage = owns_storage
+        self._closed = False
+        self._position = 0
+
+        self._ctx = cipher.new_raf_ctx()
+        self._io, self._storage_handle = _make_io(storage)
+        if rng is None:
+            rng = _random.Random(cipher)
+        self._rng, self._rng_keepalive = _random.as_raf_rng(rng)
+        self._raw_scratch, self._scratch = _allocate_scratch(cipher, chunk_size)
+
+        self._config = ffi.new("aegis_raf_config*")
+        self._config.scratch = self._scratch
+        self._config.chunk_size = chunk_size
+        flags = 0
+        if creating:
+            flags |= _FLAG_CREATE
+            if truncate:
+                flags |= _FLAG_TRUNCATE
+        self._config.flags = flags
+        self._config.merkle = ffi.NULL
+
+        self._merkle_hasher = None
+        self._merkle_cfg = None
+        self._merkle_buf = None
+        self._merkle_hasher_handle = None
+        if merkle:
+            hasher = SHA256MerkleHasher() if merkle is True else merkle
+            if merkle_max_chunks <= 0:
+                raise ValueError("merkle_max_chunks must be > 0")
+            if not (_MERKLE_HASH_MIN <= hasher.hash_len <= _MERKLE_HASH_MAX):
+                raise ValueError(
+                    f"hash_len must be between {_MERKLE_HASH_MIN} and "
+                    f"{_MERKLE_HASH_MAX}, got {hasher.hash_len}"
+                )
+            mcfg = ffi.new("aegis_raf_merkle_config*")
+            mcfg.hash_len = hasher.hash_len
+            mcfg.max_chunks = merkle_max_chunks
+            buf_size = lib.aegis_raf_merkle_buffer_size(mcfg)
+            if buf_size == 0 or buf_size >= int(ffi.cast("size_t", -1)):
+                raise ValueError("Merkle tree buffer size overflow")
+            self._merkle_buf = bytearray(buf_size)
+            self._merkle_hasher_handle = ffi.new_handle(hasher)
+            mcfg.hash_leaf = _hash_leaf_callback
+            mcfg.hash_parent = _hash_parent_callback
+            mcfg.hash_empty = _hash_empty_callback
+            mcfg.hash_commitment = _hash_commitment_callback
+            mcfg.user = self._merkle_hasher_handle
+            mcfg.buf = ffi.from_buffer(self._merkle_buf)
+            mcfg.len = buf_size
+            self._merkle_hasher = hasher
+            self._merkle_cfg = mcfg
+            self._config.merkle = mcfg
+
+        if creating:
+            cipher.raf_create(key, self._ctx, self._io, self._rng, self._config)
+        else:
+            cipher.raf_open(key, self._ctx, self._io, self._rng, self._config)
+
+    def _check_open(self) -> None:
+        if self._closed:
+            raise ValueError("I/O operation on closed file")
+
+    def read(self, size: int = -1, offset: int | None = None) -> bytes:
+        """Read and decrypt bytes, advancing the current position.
+
+        Args:
+            size: Number of bytes to read (-1 for all remaining).
+            offset: Position to read from (None uses current position).
+
+        Returns:
+            Decrypted bytes (fewer than requested at EOF).
+
+        Raises:
+            ValueError: If authentication fails (corruption or tampering).
+            RuntimeError: If the read fails.
+        """
+        self._check_open()
+        if offset is None:
+            offset = self._position
+        elif offset < 0:
+            raise ValueError(f"offset must be non-negative, got {offset}")
+        if size < 0:
+            size = max(0, self.size - offset)
+        if size == 0:
+            return b""
+        out = self._cipher.raf_read(self._ctx, size, offset)
+        self._position = offset + len(out)
+        return bytes(out)
+
+    def pread(self, size: int, offset: int) -> bytes:
+        """Read at offset without updating the position (like os.pread)."""
+        self._check_open()
+        if offset < 0:
+            raise ValueError(f"offset must be non-negative, got {offset}")
+        if size < 0:
+            raise ValueError(f"size must be non-negative, got {size}")
+        if size == 0:
+            return b""
+        return bytes(self._cipher.raf_read(self._ctx, size, offset))
+
+    def read_into(self, buf: Buffer, offset: int | None = None) -> int:
+        """Read and decrypt into buf, advancing the position.
+
+        Returns:
+            Number of bytes actually read.
+        """
+        self._check_open()
+        buf = memoryview(buf)
+        if offset is None:
+            offset = self._position
+        elif offset < 0:
+            raise ValueError(f"offset must be non-negative, got {offset}")
+        if buf.nbytes == 0:
+            return 0
+        out = self._cipher.raf_read(self._ctx, buf.nbytes, offset, into=buf)
+        n = len(out)
+        self._position = offset + n
+        return n
+
+    def write(self, data: Buffer, offset: int | None = None) -> int:
+        """Encrypt and write bytes, advancing the current position.
+
+        Returns:
+            Number of bytes written (always len(data) on success).
+
+        Raises:
+            ValueError: If the write exceeds merkle_max_chunks.
+            RuntimeError: If the write fails.
+        """
+        self._check_open()
+        data = memoryview(data)
+        if offset is None:
+            offset = self._position
+        elif offset < 0:
+            raise ValueError(f"offset must be non-negative, got {offset}")
+        if data.nbytes == 0:
+            return 0
+        try:
+            n = self._cipher.raf_write(self._ctx, data, offset)
+        except RuntimeError as e:
+            if "EOVERFLOW" in str(e) and self._merkle_cfg is not None:
+                new_end = offset + data.nbytes
+                if new_end <= 0xFFFFFFFFFFFFFFFF:
+                    chunk_size = self._config.chunk_size
+                    new_chunks = (new_end + chunk_size - 1) // chunk_size
+                    if new_chunks > self._merkle_cfg.max_chunks:
+                        raise ValueError(
+                            f"write exceeds merkle_max_chunks "
+                            f"({self._merkle_cfg.max_chunks}); "
+                            "open the file with a larger merkle_max_chunks"
+                        ) from e
+            raise
+        self._position = offset + n
+        return n
+
+    def pwrite(self, data: Buffer, offset: int) -> int:
+        """Write at offset without updating the position (like os.pwrite)."""
+        pos = self._position
+        try:
+            return self.write(data, offset)
+        finally:
+            self._position = pos
+
+    def truncate(self, size: int | None = None) -> int:
+        """Resize the file (default: current position). Returns the new size."""
+        self._check_open()
+        if size is None:
+            size = self._position
+        self._cipher.raf_truncate(self._ctx, size)
+        return size
+
+    def seek(self, offset: int, whence: int = 0) -> int:
+        """Move the file position (whence: 0=absolute, 1=relative, 2=from end)."""
+        self._check_open()
+        if whence == 0:
+            new_pos = offset
+        elif whence == 1:
+            new_pos = self._position + offset
+        elif whence == 2:
+            new_pos = self.size + offset
+        else:
+            raise ValueError(f"invalid whence: {whence}")
+        if new_pos < 0:
+            raise ValueError(f"negative seek position: {new_pos}")
+        self._position = new_pos
+        return new_pos
+
+    def tell(self) -> int:
+        """Return the current position."""
+        return self._position
+
+    @property
+    def size(self) -> int:
+        """Logical plaintext file size."""
+        self._check_open()
+        return self._cipher.raf_size(self._ctx)
+
+    def sync(self) -> None:
+        """Flush writes to backing storage."""
+        self._check_open()
+        self._cipher.raf_sync(self._ctx)
+
+    def close(self) -> None:
+        """Close the file, syncing and zeroizing key material."""
+        if not self._closed:
+            self._closed = True
+            self._cipher.raf_close(self._ctx)
+            if self._owns_storage and hasattr(self._storage, "close"):
+                self._storage.close()
+
+    @property
+    def closed(self) -> bool:
+        """True if the file has been closed."""
+        return self._closed
+
+    def __enter__(self) -> Self:
+        return self
+
+    def __exit__(self, *args) -> None:
+        self.close()
+
+    def _check_merkle_enabled(self) -> None:
+        if self._merkle_cfg is None:
+            raise ValueError("Merkle tree is not enabled on this file")
+
+    def merkle_rebuild(self) -> None:
+        """Rebuild the Merkle tree by reading and re-hashing every chunk.
+
+        Must be called after opening an existing file with merkle enabled.
+        """
+        self._check_open()
+        self._check_merkle_enabled()
+        self._cipher.raf_merkle_rebuild(self._ctx)
+
+    def merkle_verify(self) -> int | None:
+        """Verify every chunk's hash against the Merkle tree.
+
+        Returns:
+            None if all chunks match, otherwise the index of the first
+            corrupted chunk.
+        """
+        self._check_open()
+        self._check_merkle_enabled()
+        return self._cipher.raf_merkle_verify(self._ctx)
+
+    @property
+    def root_hash(self) -> bytes | None:
+        """Current Merkle root commitment hash, or None if Merkle is not enabled."""
+        self._check_open()
+        if self._merkle_cfg is None:
+            return None
+        return self._cipher.raf_merkle_commitment(
+            self._ctx, self._merkle_hasher.hash_len
+        )
+
+    def verify_root(self, expected: Buffer) -> None:
+        """Rebuild the Merkle tree and verify the root matches expected.
+
+        Raises:
+            ValueError: If the root mismatches or expected has the wrong length.
+        """
+        self._check_open()
+        self._check_merkle_enabled()
+        expected = memoryview(expected)
+        if expected.nbytes != self._merkle_hasher.hash_len:
+            raise ValueError(
+                f"expected must be {self._merkle_hasher.hash_len} bytes, "
+                f"got {expected.nbytes}"
+            )
+        self.merkle_rebuild()
+        if self.root_hash != bytes(expected):
+            raise ValueError("authentication failed: merkle root mismatch")

src/aeg/random.py

@@ -0,0 +1,114 @@
+"""Pluggable random sources for aeg, built on AEGIS streams.
+
+``Random`` is seeded once from the OS (secrets), then deterministic: each call
+produces AEGIS keystream under the current key/nonce and increments the nonce
+afterwards (little-endian), so no output block is ever produced twice.
+Modeled on quantumpipe's ChaCha20-based Random class.
+
+Note: within the ``aeg`` package namespace this submodule shadows the stdlib
+``random`` module; import it as ``from aeg import random`` (or ``aeg.random``).
+"""
+
+import errno
+import secrets
+from typing import Protocol
+
+from ._loader import ffi
+from .util import Buffer, nonce_increment
+
+__all__ = ["RandomSource", "Random", "system_random", "as_raf_rng"]
+
+
+class RandomSource(Protocol):
+    """Callable returning n cryptographically secure random bytes."""
+
+    def __call__(self, n: int) -> bytes: ...
+
+
+class Random:
+    """AEGIS stream RNG: seeded once from the OS, then deterministic.
+
+    Each call produces keystream under the current key/nonce and increments
+    the nonce afterwards, so no output block is ever produced twice.
+
+    The primary way to get one is the cipher module's own random() helper,
+    e.g. ``aeg.cipher("AEGIS-128X2").random()``, which returns a per-thread
+    singleton (each thread gets its own OS-seeded generator; no locking).
+    """
+
+    def __init__(self, cipher, *, seed: Buffer | None = None):
+        """Create a Random using the given cipher module.
+
+        Args:
+            cipher: Cipher module (from aeg.cipher()).
+            seed: Optional nonce || key bytes (NONCEBYTES + KEYBYTES long).
+                  Primarily for tests; default seeds from the OS via secrets.
+
+        Raises:
+            TypeError: If seed length is invalid.
+        """
+        self._cipher = cipher
+        if seed is None:
+            seed = secrets.token_bytes(cipher.NONCEBYTES + cipher.KEYBYTES)
+        seed = memoryview(seed)
+        if seed.nbytes != cipher.NONCEBYTES + cipher.KEYBYTES:
+            raise TypeError(
+                f"seed length must be {cipher.NONCEBYTES + cipher.KEYBYTES} "
+                f"(nonce || key) for {cipher.NAME}"
+            )
+        self._nonce = bytearray(seed[: cipher.NONCEBYTES])
+        self._key = seed[cipher.NONCEBYTES :]
+
+    def bytes(self, n: int) -> bytearray:
+        """Return n random bytes as a bytearray."""
+        out = self._cipher.stream(self._key, self._nonce, n)
+        nonce_increment(self._nonce)
+        return out
+
+    def into(self, buf: Buffer) -> None:
+        """Fill buf in place with random bytes."""
+        mv = memoryview(buf).cast("B")
+        self._cipher.stream(self._key, self._nonce, into=mv)
+        nonce_increment(self._nonce)
+
+    def __call__(self, n: int) -> bytearray:
+        """Alias for bytes(n); satisfies the RandomSource protocol."""
+        return self.bytes(n)
+
+
+def system_random(n: int) -> bytes:
+    """Random source backed directly by secrets.token_bytes."""
+    return secrets.token_bytes(n)
+
+
+@ffi.callback("int(void*, uint8_t*, size_t)")
+def _random_callback(user, out, length):
+    try:
+        source = ffi.from_handle(user)
+        data = source(length)
+        if len(data) != length:
+            ffi.errno = errno.EINVAL
+            return -1
+        ffi.memmove(out, data, length)
+        return 0
+    except Exception:
+        ffi.errno = errno.EIO
+        return -1
+
+
+def as_raf_rng(source: RandomSource):
+    """Build an aegis_raf_rng* cdata wrapping a Python random source.
+
+    Args:
+        source: Callable returning n random bytes, e.g. a Random instance
+                or system_random.
+
+    Returns:
+        Tuple of (rng, keepalive). Keep ``keepalive`` alive for as long as
+        the rng may be called by C code.
+    """
+    handle = ffi.new_handle(source)
+    rng = ffi.new("aegis_raf_rng*")
+    rng.user = handle
+    rng.random = _random_callback
+    return rng, (handle, _random_callback)

src/aeg/securebuf.py

@@ -0,0 +1,116 @@
+"""SecureBuffer: locked, core-dump-excluded memory for sensitive data.
+
+Backed by an anonymous mmap that is locked against swapping (mlock /
+VirtualLock) and excluded from core dumps (MADV_DONTDUMP) where available.
+The contents are wiped on close().
+
+SecureBuffer is not used for return values; allocate one yourself and pass
+it as an ``into`` target to keep plaintext/keys out of the regular heap:
+
+    with SecureBuffer(4096) as buf:
+        ciph.decrypt(key, nonce, ct, mac, into=buf)
+
+Adapted from quantumpipe's securebuf module.
+"""
+
+import ctypes
+import mmap
+import os
+from contextlib import suppress
+from typing import Self
+
+from .util import wipe
+
+__all__ = ["SecureBuffer"]
+
+
+class SecureBuffer:
+    """A fixed-size buffer in locked, non-dumped memory, wiped on close."""
+
+    def __init__(self, size: int):
+        if size <= 0:
+            raise ValueError(f"size must be positive, got {size}")
+        self._mmap = mmap.mmap(-1, size, access=mmap.ACCESS_WRITE)
+        try:
+            self._lock()
+            self._dontdump()
+        except BaseException:
+            self._mmap.close()
+            raise
+
+    @property
+    def closed(self) -> bool:
+        """True if the buffer has been closed."""
+        return self._mmap.closed
+
+    def _addr(self) -> int:
+        return ctypes.addressof(ctypes.c_char.from_buffer(self._mmap))
+
+    def _lock(self) -> None:
+        addr, size = self._addr(), len(self._mmap)
+        if os.name == "nt":
+            api = ctypes.WinDLL("kernel32", use_last_error=True)
+            api.VirtualLock.argtypes = (ctypes.c_void_p, ctypes.c_size_t)
+            api.VirtualLock.restype = ctypes.c_bool
+            if not api.VirtualLock(addr, size):
+                raise ctypes.WinError(ctypes.get_last_error())
+        else:
+            libc = ctypes.CDLL(None, use_errno=True)
+            libc.mlock.argtypes = (ctypes.c_void_p, ctypes.c_size_t)
+            libc.mlock.restype = ctypes.c_int
+            if libc.mlock(addr, size):
+                err = ctypes.get_errno()
+                raise OSError(err, os.strerror(err))
+
+    def _unlock(self) -> None:
+        addr, size = self._addr(), len(self._mmap)
+        if os.name == "nt":
+            api = ctypes.WinDLL("kernel32", use_last_error=True)
+            api.VirtualUnlock.argtypes = (ctypes.c_void_p, ctypes.c_size_t)
+            api.VirtualUnlock.restype = ctypes.c_bool
+            if not api.VirtualUnlock(addr, size):
+                raise ctypes.WinError(ctypes.get_last_error())
+        else:
+            libc = ctypes.CDLL(None, use_errno=True)
+            libc.munlock.argtypes = (ctypes.c_void_p, ctypes.c_size_t)
+            libc.munlock.restype = ctypes.c_int
+            if libc.munlock(addr, size):
+                err = ctypes.get_errno()
+                raise OSError(err, os.strerror(err))
+
+    def _dontdump(self) -> None:
+        if hasattr(self._mmap, "madvise") and hasattr(mmap, "MADV_DONTDUMP"):
+            self._mmap.madvise(mmap.MADV_DONTDUMP)
+
+    def close(self) -> None:
+        """Wipe the contents, unlock and release the memory. Idempotent."""
+        if self.closed:
+            return
+        wipe(self._mmap)
+        self._unlock()
+        self._mmap.close()
+
+    def __buffer__(self, flags: int) -> memoryview:
+        return memoryview(self._mmap)
+
+    def __getitem__(self, key):
+        return memoryview(self._mmap)[key]
+
+    def __setitem__(self, key, value):
+        memoryview(self._mmap)[key] = value
+
+    def __bytes__(self):
+        raise TypeError("SecureBuffer cannot be converted to bytes")
+
+    def __len__(self) -> int:
+        return len(self._mmap)
+
+    def __enter__(self) -> Self:
+        return self
+
+    def __exit__(self, *args) -> None:
+        self.close()
+
+    def __del__(self):
+        with suppress(Exception):
+            self.close()

src/aeg/util.py

@@ -4,19 +4,12 @@ Currently provides Python-side aligned allocation helpers that avoid relying
 on libc/posix_memalign. Memory is owned by Python; C code only borrows it.
 """
 
-from typing import Protocol
+from collections.abc import Buffer
 
 from ._loader import ffi
 
 __all__ = ["new_aligned_struct", "aligned_address", "Buffer", "nonce_increment", "wipe"]
 
-try:
-    from collections.abc import Buffer  # type: ignore
-except ImportError:
-    # Fallback for Python < 3.12
-    class Buffer(Protocol):
-        def __buffer__(self, flags: int) -> memoryview: ...
-
 
 def aligned_address(obj) -> int:
     """Return the integer address of the start of a cffi array object."""

tests/test_raf.py

@@ -0,0 +1,377 @@
+import pytest
+
+from aeg import aegis128l, aegis128x2, aegis128x4, aegis256, aegis256x2, aegis256x4, raf
+from aeg import random as aeg_random
+
+CIPHERS = [aegis128l, aegis128x2, aegis128x4, aegis256, aegis256x2, aegis256x4]
+
+CHUNK = 1024
+
+
+def cipher_id(c):
+    return c.NAME
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_roundtrip_bytesio(c):
+    key = c.random_key()
+    data = bytes(range(256)) * 20  # spans multiple chunks
+    st = raf.BytesIOStorage()
+    with raf.create(st, key, c, chunk_size=CHUNK) as f:
+        assert f.write(data) == len(data)
+        assert f.size == len(data)
+    with raf.open(st, key, c) as f:
+        assert f.read() == data
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_roundtrip_io_bytesio(c):
+    """A bare io.BytesIO is auto-wrapped in StreamStorage."""
+    import io
+
+    key = c.random_key()
+    data = bytes(range(256)) * 20
+    bio = io.BytesIO()
+    with raf.create(bio, key, c, chunk_size=CHUNK) as f:
+        f.write(data)
+        f.pwrite(b"XY", 1000)  # unaligned write across a chunk boundary
+        data = data[:1000] + b"XY" + data[1002:]
+        f.truncate(2000)
+    with raf.open(bio, key, c) as f:
+        assert f.size == 2000
+        assert f.pread(6, 999) == data[999:1005]
+        assert f.read() == data[:2000]
+    bio.close()
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_roundtrip_filestorage(c, tmp_path):
+    key = c.random_key()
+    data = b"file-backed" * 500
+    path = tmp_path / "test.raf"
+    with raf.create(path, key, c, chunk_size=CHUNK) as f:
+        f.write(data)
+    with raf.open(path, key, c) as f:
+        assert f.read() == data
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_random_access(c):
+    key = c.random_key()
+    st = raf.BytesIOStorage()
+    data = bytearray(4097)
+    with raf.create(st, key, c, chunk_size=CHUNK) as f:
+        f.write(data)
+        # unaligned writes across chunk boundaries
+        f.pwrite(b"ABCDEF", 1000)  # crosses chunk 0/1 boundary at 1024
+        f.pwrite(b"z", 4096)  # last byte
+        data[1000:1006] = b"ABCDEF"
+        data[4096:4097] = b"z"
+        assert f.pread(6, 1000) == b"ABCDEF"
+        assert f.pread(4097, 0) == bytes(data)
+        assert f.tell() == len(data)  # pwrite/pread don't move the position
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_seek_tell_read_into(c):
+    key = c.random_key()
+    st = raf.BytesIOStorage()
+    data = b"0123456789" * 100
+    with raf.create(st, key, c, chunk_size=CHUNK) as f:
+        f.write(data)
+        assert f.tell() == len(data)
+        assert f.seek(10) == 10
+        assert f.read(5) == data[10:15]
+        assert f.tell() == 15
+        assert f.seek(-5, 1) == 10
+        assert f.seek(0, 2) == len(data)
+        buf = bytearray(20)
+        assert f.read_into(buf, 100) == 20
+        assert bytes(buf) == data[100:120]
+        with pytest.raises(ValueError):
+            f.seek(-1)
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_truncate(c):
+    key = c.random_key()
+    st = raf.BytesIOStorage()
+    with raf.create(st, key, c, chunk_size=CHUNK) as f:
+        f.write(b"x" * 3000)
+        assert f.truncate(1500) == 1500
+        assert f.size == 1500
+        assert f.pread(1500, 0) == b"x" * 1500
+        f.truncate(2500)  # grow, zero-filled
+        assert f.size == 2500
+        assert f.pread(1000, 1500) == b"\x00" * 1000
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_wrong_key(c):
+    key = c.random_key()
+    st = raf.BytesIOStorage()
+    with raf.create(st, key, c, chunk_size=CHUNK) as f:
+        f.write(b"secret")
+    with pytest.raises(ValueError, match="authentication failed"):
+        raf.open(st, c.random_key(), c)
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_tampered_chunk(c):
+    key = c.random_key()
+    st = raf.BytesIOStorage()
+    with raf.create(st, key, c, chunk_size=CHUNK) as f:
+        f.write(b"y" * 2048)
+    # flip a ciphertext byte inside the first chunk record (after the nonce)
+    st._data[raf.HEADER_SIZE + c.NONCEBYTES + 10] ^= 1
+    with raf.open(st, key, c) as f:
+        with pytest.raises(ValueError, match="authentication failed"):
+            f.read()
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_probe_and_cipher_mismatch(c):
+    key = c.random_key()
+    st = raf.BytesIOStorage()
+    with raf.create(st, key, c, chunk_size=CHUNK) as f:
+        f.write(b"data")
+    info = raf.probe(st)
+    assert info.alg_id == c.RAF_ALG_ID
+    assert info.chunk_size == CHUNK
+    assert info.file_size == 4
+    other = aegis256 if c is not aegis256 else aegis128l
+    with pytest.raises(ValueError, match="algorithm id"):
+        raf.open(st, key, other)
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_create_existing(c, tmp_path):
+    key = c.random_key()
+    path = tmp_path / "exists.raf"
+    raf.create(path, key, c, chunk_size=CHUNK).close()
+    with pytest.raises(FileExistsError):
+        raf.create(path, key, c, chunk_size=CHUNK)
+    with raf.create(path, key, c, chunk_size=CHUNK, truncate=True) as f:
+        assert f.size == 0
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_open_missing(c, tmp_path):
+    with pytest.raises(FileNotFoundError):
+        raf.open(tmp_path / "missing.raf", c.random_key(), c)
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_merkle(c):
+    key = c.random_key()
+    st = raf.BytesIOStorage()
+    data = b"m" * 5000
+    with raf.create(st, key, c, chunk_size=CHUNK, merkle=True) as f:
+        f.write(data)
+        root1 = f.root_hash
+        assert root1 is not None and len(root1) == 32
+        assert f.merkle_verify() is None
+        f.pwrite(b"X", 0)
+        root2 = f.root_hash
+        assert root1 != root2
+        assert f.merkle_verify() is None
+    with raf.open(st, key, c, merkle=True) as f:
+        f.verify_root(root2)  # rebuilds and compares
+        assert f.root_hash == root2
+        with pytest.raises(ValueError, match="authentication failed"):
+            f.verify_root(root1)
+        with pytest.raises(ValueError):
+            f.verify_root(b"short")
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_merkle_detects_corruption(c):
+    key = c.random_key()
+    st = raf.BytesIOStorage()
+    with raf.create(st, key, c, chunk_size=CHUNK, merkle=True) as f:
+        f.write(b"z" * 3000)
+        f.merkle_rebuild()
+        record = c.NONCEBYTES + CHUNK + 16
+        # corrupt chunk 2's ciphertext in the backing store
+        st._data[raf.HEADER_SIZE + 2 * record + c.NONCEBYTES + 5] ^= 1
+        assert f.merkle_verify() == 2
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_merkle_overflow(c):
+    key = c.random_key()
+    st = raf.BytesIOStorage()
+    with raf.create(
+        st, key, c, chunk_size=CHUNK, merkle=True, merkle_max_chunks=2
+    ) as f:
+        f.write(b"a" * 2048)
+        with pytest.raises(ValueError, match="merkle_max_chunks"):
+            f.write(b"b")
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_merkle_disabled(c):
+    key = c.random_key()
+    st = raf.BytesIOStorage()
+    with raf.create(st, key, c, chunk_size=CHUNK) as f:
+        assert f.root_hash is None
+        with pytest.raises(ValueError, match="not enabled"):
+            f.merkle_verify()
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_custom_rng(c):
+    counter = 0
+
+    def deterministic(n: int) -> bytes:
+        nonlocal counter
+        out = bytes((counter + i) % 256 for i in range(n))
+        counter += n
+        return out
+
+    key = c.random_key()
+    st = raf.BytesIOStorage()
+    with raf.create(st, key, c, chunk_size=CHUNK, rng=deterministic) as f:
+        f.write(b"rng test")
+    with raf.open(st, key, c) as f:
+        assert f.read() == b"rng test"
+    assert counter > 0
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_default_rng_matches_cipher(c):
+    """raf.create/open without rng use a Random seeded for the same cipher."""
+    key = c.random_key()
+    st = raf.BytesIOStorage()
+    with raf.create(st, key, c, chunk_size=CHUNK) as f:
+        assert f._rng_keepalive[0]  # handle alive
+        f.write(b"default rng")
+    with raf.open(st, key, c) as f:
+        assert f.read() == b"default rng"
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_random_per_cipher(c):
+    """Random can be constructed with each cipher module."""
+    r1 = aeg_random.Random(c)
+    r2 = c.random()
+    out = r1.bytes(100)
+    assert len(out) == 100
+    assert any(out)  # not all zeros
+    assert len(r2.bytes(50)) == 50
+    # Random instance satisfies the RandomSource callable protocol
+    assert callable(r1)
+    rng_cdata, keepalive = aeg_random.as_raf_rng(r1)
+    assert rng_cdata.random is not None and keepalive
+
+
+def test_random_determinism_with_seed():
+    """Two Random instances seeded identically produce identical output."""
+    c = aegis256x4
+    seed = bytes(range(c.NONCEBYTES + c.KEYBYTES))
+    r1 = aeg_random.Random(c, seed=seed)
+    r2 = aeg_random.Random(c, seed=seed)
+    assert r1.bytes(64) == r2.bytes(64)
+    assert r1.bytes(1000) == r2.bytes(1000)
+    with pytest.raises(TypeError, match="seed length"):
+        aeg_random.Random(c, seed=b"short")
+
+
+def test_random_into_equivalence():
+    """into() fills in place and matches bytes() from an identically-seeded RNG."""
+    c = aegis256x4
+    seed = bytes(c.NONCEBYTES + c.KEYBYTES)
+    r1 = aeg_random.Random(c, seed=seed)
+    r2 = aeg_random.Random(c, seed=seed)
+    buf = bytearray(128)
+    r1.into(buf)
+    assert bytes(buf) == bytes(r2.bytes(128))
+    # buffers of other buffer-protocol types work too
+    import array
+
+    arr = array.array("b", bytes(32))
+    r1.into(arr)
+    assert any(arr.tobytes())
+
+
+def test_random_no_repeat():
+    """Successive calls never repeat a block (nonce increments)."""
+    r = aeg_random.Random(aegis128x2)
+    blocks = {bytes(r.bytes(64)) for _ in range(1000)}
+    assert len(blocks) == 1000
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_cipher_module_random_helper(c):
+    """Each cipher module provides a random() helper bound to itself."""
+    r = c.random()
+    assert isinstance(r, aeg_random.Random)
+    assert r._cipher is c
+    assert len(r.bytes(32)) == 32
+    buf = bytearray(48)
+    r.into(buf)
+    assert any(buf)
+
+
+def test_random_module():
+    assert len(aeg_random.system_random(16)) == 16
+    # any callable source can be bridged to the C RNG struct
+    rng, keepalive = aeg_random.as_raf_rng(aeg_random.system_random)
+    assert rng.random is not None and keepalive
+
+    def bad(n: int) -> bytes:
+        return b"too short"
+
+    rng2, _ = aeg_random.as_raf_rng(bad)
+    out = aeg_random.ffi.new("uint8_t[8]")
+    assert rng2.random(rng2.user, out, 8) == -1
+
+
+def test_derive_master_key():
+    mk16 = bytes(range(16))
+    mk32 = bytes(range(32))
+    assert len(raf.derive_master_key(mk16)) == 16
+    assert len(raf.derive_master_key(mk32)) == 32
+    # context-bound: different contexts give different keys
+    assert raf.derive_master_key(mk32, b"a") != raf.derive_master_key(mk32, b"b")
+    # empty context still derives a scoped subkey
+    assert raf.derive_master_key(mk32) != mk32
+    with pytest.raises(ValueError):
+        raf.derive_master_key(b"\x00" * 24)
+    with pytest.raises(ValueError):
+        raf.derive_master_key(mk16, b"c" * 121)
+    with pytest.raises(ValueError):
+        raf.derive_master_key(mk32, b"c" * 73)
+    # derived keys work end-to-end
+    c = aegis256x4
+    key = raf.derive_master_key(mk32, b"my-file")
+    st = raf.BytesIOStorage()
+    with raf.create(st, key, c, chunk_size=CHUNK) as f:
+        f.write(b"derived")
+    with raf.open(st, key, c) as f:
+        assert f.read() == b"derived"
+
+
+def test_invalid_chunk_size():
+    c = aegis256x4
+    with pytest.raises(ValueError, match="chunk_size"):
+        raf.create(raf.BytesIOStorage(), c.random_key(), c, chunk_size=512)
+    with pytest.raises(ValueError, match="multiple of 16"):
+        raf.create(raf.BytesIOStorage(), c.random_key(), c, chunk_size=1032)
+
+
+def test_bad_key_length():
+    c = aegis256x4
+    with pytest.raises(TypeError, match="key length"):
+        raf.create(raf.BytesIOStorage(), b"\x00" * 16, c, chunk_size=CHUNK)
+
+
+def test_closed_file():
+    c = aegis256x4
+    st = raf.BytesIOStorage()
+    f = raf.create(st, c.random_key(), c, chunk_size=CHUNK)
+    f.close()
+    assert f.closed
+    with pytest.raises(ValueError, match="closed"):
+        f.read()
+    f.close()  # idempotent

tests/test_raises.py

@@ -252,6 +252,7 @@ class TestDecryptorFinalization:
 
         # Encrypt empty message
         ct, tag = aegis256x4.encrypt_detached(key, nonce, b"")
+        del ct
 
         # Decrypt without any updates
         decryptor = aegis256x4.Decryptor(key, nonce)

tests/test_random.py

@@ -0,0 +1,56 @@
+import threading
+
+import pytest
+
+from aeg import aegis128l, aegis128x2, aegis128x4, aegis256, aegis256x2, aegis256x4
+
+CIPHERS = [aegis128l, aegis128x2, aegis128x4, aegis256, aegis256x2, aegis256x4]
+
+
+def cipher_id(c):
+    return c.NAME
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_random_is_per_thread_singleton(c):
+    assert c.random() is c.random()
+    other = []
+    t = threading.Thread(target=lambda: other.append(c.random()))
+    t.start()
+    t.join()
+    assert other[0] is not c.random()
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_random_key_nonce_from_csprng(c):
+    assert len(c.random_key()) == c.KEYBYTES
+    assert len(c.random_nonce()) == c.NONCEBYTES
+    # successive outputs never repeat
+    assert c.random_key() != c.random_key()
+    assert c.random_nonce() != c.random_nonce()
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_threads_get_independent_streams(c):
+    results = [None] * 2
+
+    def work(i):
+        results[i] = bytes(c.random_key())
+
+    threads = [threading.Thread(target=work, args=(i,)) for i in range(2)]
+    for t in threads:
+        t.start()
+    for t in threads:
+        t.join()
+    assert results[0] != results[1]
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_bytes_into_callable(c):
+    rng = c.random()
+    a = rng.bytes(64)
+    buf = bytearray(64)
+    rng.into(buf)
+    b = rng(64)
+    assert len(a) == len(b) == 64
+    assert a != bytes(buf) != b  # three consecutive blocks differ

tests/test_securebuf.py

@@ -0,0 +1,106 @@
+"""Tests for aeg.securebuf.SecureBuffer.
+
+CPython-only assumptions: reference counting frees memoryviews as soon as
+they are del'd; live exported views would make mmap.close() fail with
+BufferError.
+"""
+
+import pytest
+
+from aeg import aegis128l
+from aeg.securebuf import SecureBuffer
+from aeg.util import wipe
+
+
+def test_buffer_protocol_read_write():
+    with SecureBuffer(32) as sb:
+        assert len(sb) == 32
+        mv = memoryview(sb)
+        assert len(mv) == 32
+        assert not mv.readonly
+        mv[:4] = b"abcd"
+        assert mv[:4] == b"abcd"
+        del mv
+    assert sb.closed
+
+
+def test_not_convertible_to_bytes():
+    with SecureBuffer(16) as sb, pytest.raises(TypeError, match="cannot be converted"):
+        bytes(sb)
+
+
+def test_invalid_size():
+    for size in (0, -1):
+        with pytest.raises(ValueError):
+            SecureBuffer(size)
+
+
+def test_close_is_idempotent_and_blocks_access():
+    sb = SecureBuffer(16)
+    sb.close()
+    assert sb.closed
+    with pytest.raises(ValueError):
+        memoryview(sb)
+    sb.close()
+    assert sb.closed
+
+
+def test_context_manager_closes_on_exception():
+    with pytest.raises(RuntimeError), SecureBuffer(16) as sb:
+        raise RuntimeError("boom")
+    assert sb.closed
+
+
+def test_del_closes():
+    import gc
+
+    sb = SecureBuffer(16)
+    ref = sb._mmap
+    del sb
+    gc.collect()
+    assert ref.closed
+
+
+def test_getitem_returns_memoryview():
+    with SecureBuffer(16) as sb:
+        sb[:4] = b"abcd"
+        view = sb[2:8]
+        assert isinstance(view, memoryview)
+        assert view.tobytes() == b"cd\x00\x00\x00\x00"
+        del view
+
+
+def test_wipe_fills_ff():
+    sb = SecureBuffer(16)
+    sb[:] = b"A" * 16
+    wipe(sb)
+    mv = memoryview(sb)
+    assert mv.tobytes() == b"\xff" * 16
+    del mv
+    sb.close()
+
+
+def test_roundtrip_as_key_and_into_target():
+    """SecureBuffer works as key input and as an `into` target."""
+    c = aegis128l
+    with SecureBuffer(c.KEYBYTES) as key, SecureBuffer(c.NONCEBYTES) as nonce:
+        key[:] = bytes(range(c.KEYBYTES))
+        nonce[:] = bytes(c.NONCEBYTES)
+        msg = b"hello secure world"
+        ct, mac = c.encrypt_detached(key, nonce, msg)
+        with SecureBuffer(len(msg)) as pt:
+            out = c.decrypt_detached(key, nonce, ct, mac, into=pt)
+            assert len(out) == len(msg)
+            view = pt[:]
+            assert view.tobytes() == msg
+            del out, view
+
+
+def test_stream_into():
+    c = aegis128l
+    key, nonce = c.random_key(), c.random_nonce()
+    with SecureBuffer(64) as buf:
+        c.stream(key, nonce, into=buf)
+        view = buf[:]
+        assert view.tobytes() == bytes(c.stream(key, nonce, 64))
+        del view

tests/test_stream_xor.py

@@ -0,0 +1,54 @@
+import pytest
+
+from aeg import aegis128l, aegis128x2, aegis128x4, aegis256, aegis256x2, aegis256x4
+
+CIPHERS = [aegis128l, aegis128x2, aegis128x4, aegis256, aegis256x2, aegis256x4]
+
+
+def cipher_id(c):
+    return c.NAME
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_stream_xor_roundtrip(c):
+    key, nonce = c.random_key(), c.random_nonce()
+    msg = b"stream cipher test" * 100  # spans multiple blocks, unaligned tail
+    ct = c.stream_xor(key, nonce, msg)
+    assert isinstance(ct, bytearray)
+    assert len(ct) == len(msg)
+    assert ct != msg
+    # the same function decrypts
+    assert c.stream_xor(key, nonce, ct) == msg
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_stream_xor_is_stream_xor(c):
+    """Output equals data XOR stream(key, nonce) keystream."""
+    key, nonce = c.random_key(), c.random_nonce()
+    msg = b"\x5a" * 500
+    ks = c.stream(key, nonce, len(msg))
+    ct = c.stream_xor(key, nonce, msg)
+    assert bytes(ct) == bytes(m ^ k for m, k in zip(msg, ks))
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_stream_xor_into(c):
+    key, nonce = c.random_key(), c.random_nonce()
+    msg = b"into target" * 10
+    buf = bytearray(len(msg) + 10)
+    out = c.stream_xor(key, nonce, msg, into=buf)
+    assert len(out) == len(msg)
+    assert bytes(out) == bytes(c.stream_xor(key, nonce, msg))
+
+
+@pytest.mark.parametrize("c", CIPHERS, ids=cipher_id)
+def test_unauthenticated_deprecated(c):
+    key, nonce = c.random_key(), c.random_nonce()
+    msg = b"legacy format" * 20  # longer than one block: absorbing mode diverges
+    with pytest.warns(DeprecationWarning):
+        ct = c.encrypt_unauthenticated(key, nonce, msg)
+    with pytest.warns(DeprecationWarning):
+        pt = c.decrypt_unauthenticated(key, nonce, ct)
+    assert pt == msg
+    # deprecated pair is NOT compatible with stream_xor beyond the first block
+    assert bytes(ct) != bytes(c.stream_xor(key, nonce, msg))

tests/util.py

@@ -1,6 +1,5 @@
 import random
 
-
 _rng = random.SystemRandom()
 
 

tools/build_backend.py

@@ -1,100 +1,68 @@
 """Custom build backend that builds libaegis with Zig before building the Python package."""
 
+import os
+import platform
 import shutil
 import subprocess
 import sys
 from pathlib import Path
 
-from setuptools import build_meta as _orig
+from setuptools import build_meta
+
+__all__ = [
+    "build_sdist",
+    "build_wheel",
+    "build_editable",
+    "get_requires_for_build_sdist",
+    "get_requires_for_build_wheel",
+    "prepare_metadata_for_build_wheel",
+]
+
+_MACOS_TARGET = "11.0"
+_prepared = False
 
 
-def _check_zig_available():
-    """Check if Zig is installed and available."""
+def _prepare():
+    """Prepare the build environment and build libaegis."""
+    global _prepared
+    if _prepared:
+        return
+    _prepared = True
+
+    # Set macOS deployment target
+    if sys.platform == "darwin" and "MACOSX_DEPLOYMENT_TARGET" not in os.environ:
+        os.environ["MACOSX_DEPLOYMENT_TARGET"] = _MACOS_TARGET
+
+    # Check Zig is available
     if shutil.which("zig") is None:
         raise RuntimeError(
-            "\n" + "=" * 70 + "\n"
-            "ERROR: Zig compiler not found!\n"
-            "\n"
-            "Building aeg requires the Zig compiler to build the libaegis\n"
-            "static library. Please install Zig before building this package.\n"
-            "\n"
-            "Installation instructions:\n"
-            "  - Visit: https://ziglang.org/download/\n"
-            "  - Or use a package manager:\n"
-            "    * macOS:   brew install zig\n"
-            "    * Linux:   See https://github.com/ziglang/zig/wiki/Install-Zig-from-a-Package-Manager\n"
-            "    * Windows: choco install zig  or  scoop install zig\n"
-            "\n"
-            "After installing Zig, please try building again.\n" + "=" * 70 + "\n"
+            "Zig compiler not found. Install from https://ziglang.org/download/"
         )
 
-
-def _build_libaegis():
-    """Build libaegis static library with Zig."""
-    # Check Zig availability first
-    _check_zig_available()
-
+    # Build libaegis
     libaegis_dir = Path(__file__).parent.parent / "libaegis"
-    if not libaegis_dir.exists():
-        raise FileNotFoundError(
-            f"libaegis directory not found at {libaegis_dir}. "
-            "Cannot build static library."
-        )
-
-    print("Building libaegis static library with Zig...")
-    try:
-        subprocess.run(
-            ["zig", "build", "-Drelease"],
-            cwd=libaegis_dir,
-            check=True,
-            capture_output=False,
-        )
-        print("Successfully built libaegis static library")
-    except subprocess.CalledProcessError as e:
-        print(
-            f"\nError: Zig build failed with exit code {e.returncode}\n"
-            f"Command: {' '.join(e.cmd)}\n",
-            file=sys.stderr,
-        )
-        raise
+    cmd = ["zig", "build", "-Drelease"]
+    if sys.platform == "darwin":
+        arch = {"arm64": "aarch64", "x86_64": "x86_64"}.get(platform.machine())
+        if arch:
+            cmd.append(f"-Dtarget={arch}-macos.{_MACOS_TARGET}")
+    subprocess.run(cmd, cwd=libaegis_dir, check=True)
 
 
-# Expose all the standard build backend hooks
-def get_requires_for_build_wheel(config_settings=None):
-    """Return build requirements and ensure libaegis is built first."""
-    _build_libaegis()
-    return _orig.get_requires_for_build_wheel(config_settings)
-
-
-def get_requires_for_build_sdist(config_settings=None):
-    """Return build requirements for sdist and ensure libaegis is built first."""
-    _build_libaegis()
-    return _orig.get_requires_for_build_sdist(config_settings)
-
-
-_orig_prepare_metadata_for_build_wheel = _orig.prepare_metadata_for_build_wheel
-_orig_build_sdist = _orig.build_sdist
-
-
-def prepare_metadata_for_build_wheel(metadata_directory, config_settings=None):
-    """Prepare metadata and ensure libaegis is built (some frontends call this early)."""
-    _build_libaegis()
-    return _orig_prepare_metadata_for_build_wheel(metadata_directory, config_settings)
-
-
-def build_sdist(sdist_directory, config_settings=None):
-    """Build sdist, building libaegis first so the sdist can include built artifacts if needed."""
-    _build_libaegis()
-    return _orig_build_sdist(sdist_directory, config_settings)
+build_sdist = build_meta.build_sdist
+get_requires_for_build_sdist = build_meta.get_requires_for_build_sdist
+get_requires_for_build_wheel = build_meta.get_requires_for_build_wheel
+prepare_metadata_for_build_wheel = build_meta.prepare_metadata_for_build_wheel
 
 
+# Wheel build hooks - need libaegis built first
 def build_wheel(wheel_directory, config_settings=None, metadata_directory=None):
-    """Build wheel with libaegis built first."""
-    _build_libaegis()
-    return _orig.build_wheel(wheel_directory, config_settings, metadata_directory)
+    _prepare()
+    return build_meta.build_wheel(wheel_directory, config_settings, metadata_directory)
 
 
 def build_editable(wheel_directory, config_settings=None, metadata_directory=None):
-    """Build editable install with libaegis built first."""
-    _build_libaegis()
-    return _orig.build_editable(wheel_directory, config_settings, metadata_directory)
+    _prepare()
+    return build_meta.build_editable(
+        wheel_directory, config_settings, metadata_directory
+    )

tools/generate.py

@@ -3,8 +3,8 @@
 
 import pathlib
 import re
+import subprocess
 import sys
-from typing import Dict, Tuple
 
 
 def preprocess_content(content: str) -> str:
@@ -57,27 +57,29 @@ def format_declaration(decl: str, max_width: int = 100) -> str:
     if len(decl) <= max_width:
         return decl
 
-    if "(" in decl and ")" in decl:
-        if match := re.match(r"(.*?\s+\w+\s*)\((.*)\)(.*)", decl):
-            prefix, params, suffix = match.groups()
-            if len(prefix) + len(params) + 2 > max_width:
-                param_list = [p.strip() for p in params.split(",")]
-                if len(param_list) > 1:
-                    formatted_params = (",\n" + " " * (len(prefix) + 1)).join(
-                        param_list
-                    )
-                    return f"{prefix}({formatted_params}){suffix}"
+    if (
+        "(" in decl
+        and ")" in decl
+        and (match := re.match(r"(.*?\s+\w+\s*)\((.*)\)(.*)", decl))
+    ):
+        prefix, params, suffix = match.groups()
+        if (
+            len(prefix) + len(params) + 2 > max_width
+            and len(param_list := [p.strip() for p in params.split(",")]) > 1
+        ):
+            formatted_params = (",\n" + " " * (len(prefix) + 1)).join(param_list)
+            return f"{prefix}({formatted_params}){suffix}"
 
     return decl
 
 
 def generate_cdef(include_dir: pathlib.Path) -> str:
+    # cffi pre-defines uint8_t, size_t, etc. with the correct platform sizes.
+    # Do not typedef them here: "unsigned long" matches size_t on LP64 but is
+    # 4 bytes vs 8 on Win64 (LLP64), breaking struct layout verification.
     lines = [
         "/* This file is generated with tools/generate.py. Do not edit. */",
         "",
-        "typedef unsigned char uint8_t;",
-        "typedef unsigned long size_t;",
-        "",
     ]
 
     headers = [
@@ -88,6 +90,7 @@ def generate_cdef(include_dir: pathlib.Path) -> str:
         "aegis256.h",
         "aegis256x2.h",
         "aegis256x4.h",
+        "aegis_raf.h",
     ]
 
     for header_name in headers:
@@ -105,9 +108,9 @@ def generate_cdef(include_dir: pathlib.Path) -> str:
 
 
 def extract_constants(
-    common_h_path: pathlib.Path, header_path: pathlib.Path
-) -> Dict[str, int]:
-    """Extract constants from common.h (ALIGNMENT, RATE) and main header (KEYBYTES, NPUBBYTES, ABYTES_*)."""
+    common_h_path: pathlib.Path, header_path: pathlib.Path, raf_h_path: pathlib.Path
+) -> dict[str, int]:
+    """Extract constants from common.h (ALIGNMENT, RATE), main header (KEYBYTES, NPUBBYTES, ABYTES_*), and aegis_raf.h (RAF_ALG_ID)."""
     constants = {}
 
     # Extract from common.h
@@ -136,12 +139,20 @@ def extract_constants(
             raise ValueError(f"Could not extract {const_name} from {header_path}")
         constants[const_name] = int(match.group(1))
 
+    # Extract RAF algorithm id from aegis_raf.h
+    raf_content = raf_h_path.read_text(encoding="utf-8")
+    raf_pattern = rf"^\s*#define\s+AEGIS_RAF_ALG_{variant[5:].upper()}\s+(\d+)"
+    raf_match = re.search(raf_pattern, raf_content, re.MULTILINE)
+    if not raf_match:
+        raise ValueError(f"Could not extract RAF_ALG_ID from {raf_h_path}")
+    constants["RAF_ALG_ID"] = int(raf_match.group(1))
+
     return constants
 
 
 def extract_all_constants(
     libaegis_src_dir: pathlib.Path, include_dir: pathlib.Path
-) -> Dict[str, Dict[str, int]]:
+) -> dict[str, dict[str, int]]:
     variants = [
         "aegis128l",
         "aegis128x2",
@@ -155,6 +166,7 @@ def extract_all_constants(
     for variant in variants:
         common_h = libaegis_src_dir / variant / f"{variant}_common.h"
         header_h = include_dir / f"{variant}.h"
+        raf_h = include_dir / "aegis_raf.h"
 
         if not common_h.exists():
             print(f"Warning: {common_h} not found, skipping {variant}", file=sys.stderr)
@@ -165,7 +177,7 @@ def extract_all_constants(
             continue
 
         try:
-            constants[variant] = extract_constants(common_h, header_h)
+            constants[variant] = extract_constants(common_h, header_h, raf_h)
         except Exception as e:
             print(f"Error extracting constants from {variant}: {e}", file=sys.stderr)
 
@@ -184,7 +196,7 @@ def algo_label(name: str) -> str:
     return "AEGIS-" + name[5:].upper()
 
 
-def generate_variant(template_src: str, variant: str, constants: Dict[str, int]) -> str:
+def generate_variant(template_src: str, variant: str, constants: dict[str, int]) -> str:
     """Generate a variant module from the template with substituted constants."""
     s = template_src.replace("aegis256x4", variant).replace(
         "AEGIS-256X4", algo_label(variant)
@@ -215,6 +227,11 @@ def generate_variant(template_src: str, variant: str, constants: Dict[str, int])
         f"MACBYTES_LONG = {constants['ABYTES_MAX']}",
         s,
     )
+    s = re.sub(
+        r"RAF_ALG_ID = \d+",
+        f"RAF_ALG_ID = {constants['RAF_ALG_ID']}",
+        s,
+    )
 
     return s
 
@@ -222,8 +239,8 @@ def generate_variant(template_src: str, variant: str, constants: Dict[str, int])
 def generate_python_modules(
     template_path: pathlib.Path,
     output_dir: pathlib.Path,
-    constants: Dict[str, Dict[str, int]],
-) -> Tuple[list[pathlib.Path], list[pathlib.Path]]:
+    constants: dict[str, dict[str, int]],
+) -> tuple[list[pathlib.Path], list[pathlib.Path]]:
     if not template_path.exists():
         raise FileNotFoundError(f"Template not found: {template_path}")
 
@@ -262,6 +279,11 @@ def generate_python_modules(
                 f"MACBYTES_LONG = {const_dict['ABYTES_MAX']}",
                 new_content,
             )
+            new_content = re.sub(
+                r"RAF_ALG_ID = \d+",
+                f"RAF_ALG_ID = {const_dict['RAF_ALG_ID']}",
+                new_content,
+            )
         else:
             new_content = generate_variant(template_src, variant, const_dict)
 
@@ -274,7 +296,7 @@ def generate_python_modules(
     return updated, unchanged
 
 
-def generate_ciphers_module(constants: Dict[str, Dict[str, int]]) -> str:
+def generate_ciphers_module(constants: dict[str, dict[str, int]]) -> str:
     labels = [algo_label(variant) for variant in constants]
     literal_items = ", ".join(f'"{label}"' for label in labels)
     lines = [
@@ -355,6 +377,17 @@ def main() -> int:
         print(f"Error generating Python modules: {e}", file=sys.stderr)
         return 1
 
+    print("Step 5: Formatting generated files with ruff...", file=sys.stderr)
+    result = subprocess.run(
+        ["uv", "run", "ruff", "format", str(pyaegis_dir)],
+        capture_output=True,
+        text=True,
+        check=False,
+    )
+    if result.returncode != 0:
+        print(f"Error formatting generated files: {result.stderr}", file=sys.stderr)
+        return 1
+
     return 0
 
 

tools/release.py

@@ -1,6 +1,7 @@
 #!/usr/bin/env -S uv run
 """Build wheels for all supported Python versions using uv."""
 
+import os
 import platform
 import shutil
 import subprocess
@@ -13,19 +14,30 @@ from packaging.version import Version
 sys.path.insert(0, str(Path(__file__).parent))
 import generate
 
-PYTHON_VERSIONS = [
-    "3.10",
-    "3.11",
+# Minimum macOS deployment target for compatibility
+MACOS_DEPLOYMENT_TARGET = "11.0"
+
+# ABI3 wheel: built once, works for all GIL-enabled Python versions
+# We use a recent Python to build since it doesn't affect the wheel compatibility
+ABI3_BUILD_VERSION = "3.14+gil"
+
+# All GIL-enabled Python versions covered by the ABI3 wheel
+ABI3_COVERED_VERSIONS = [
     "3.12",
-    "3.13",
-    "3.14",
-    "3.14t",
-    "3.15",
-    "3.15t",
-    "pypy3.10",
-    "pypy3.11",
+    "3.13+gil",
+    "3.14+gil",
+    "3.15+gil",
 ]
 
+# Non-ABI3 wheels: each needs its own build (free-threaded)
+NON_ABI3_VERSIONS = [
+    "3.14t",
+    "3.15t",
+]
+
+# All versions for testing and benchmarking
+ALL_PYTHON_VERSIONS = ABI3_COVERED_VERSIONS + NON_ABI3_VERSIONS
+
 
 def get_version_from_scm():
     """Get version from setuptools-scm (git tags)."""
@@ -65,7 +77,7 @@ def get_next_version(current_version):
 def is_working_copy_clean():
     """Check if git working copy is clean."""
     result = subprocess.run(
-        ["git", "status", "--porcelain"], capture_output=True, text=True
+        ["git", "status", "--porcelain"], capture_output=True, text=True, check=False
     )
     return result.returncode == 0 and not result.stdout.strip()
 
@@ -94,21 +106,29 @@ def make_release_message(version):
     return msg
 
 
-def run_command(cmd, description):
-    """Run a command and handle errors."""
-    print(f"\n{'=' * 70}")
-    print(f"{description}")
-    print(f"{'=' * 70}")
+def run_command(cmd, description=None, env=None):
+    """Run a command and handle errors. If description is None, only print the command."""
+    if description:
+        print(f"\n{'=' * 70}")
+        print(f"{description}")
+        print(f"{'=' * 70}")
     print(f">>> {' '.join(cmd)}")
     try:
-        subprocess.run(cmd, check=True)
-        print(f"✓ {description} completed successfully")
+        subprocess.run(cmd, check=True, env=env)
         return True
     except subprocess.CalledProcessError as e:
-        print(f"✗ {description} failed with exit code {e.returncode}", file=sys.stderr)
+        print(f"✗ Command failed with exit code {e.returncode}", file=sys.stderr)
         return False
 
 
+def get_build_env():
+    """Get environment variables for building wheels."""
+    env = os.environ.copy()
+    if platform.system() == "Darwin":
+        env["MACOSX_DEPLOYMENT_TARGET"] = MACOS_DEPLOYMENT_TARGET
+    return env
+
+
 def normalize_line_endings(repo_root: Path):
     """Normalize all text files to LF line endings."""
     # Patterns for files to normalize
@@ -132,8 +152,168 @@ def normalize_line_endings(repo_root: Path):
                     file_path.write_bytes(content)
 
 
+def get_wheel_pattern(py_version: str, abi3: bool = False) -> str:
+    """Get the glob pattern for finding a wheel file."""
+    if abi3:
+        # ABI3 wheels always use cp312-abi3 tag (minimum supported version)
+        # regardless of which Python version was used to build
+        return "aeg-*-cp312-abi3-*.whl"
+    elif py_version.endswith("t"):
+        # Free-threaded Python wheels use cpXXX-cpXXXt format (e.g., cp314-cp314t)
+        base_version = py_version.replace(".", "").replace("t", "")
+        return f"aeg-*-cp{base_version}-cp{base_version}t-*.whl"
+    else:
+        # Regular CPython wheels use cpXXX-cpXXX format
+        # Strip +gil suffix used to force non-free-threaded build
+        base_version = py_version.replace(".", "").replace("+gil", "")
+        return f"aeg-*-cp{base_version}-cp{base_version}-*.whl"
+
+
+def build_abi3_wheel(dist_dir: Path, py_version: str) -> Path | None:
+    """Build the ABI3 wheel using the specified Python version."""
+    cmd = ["uv", "build", "--python", py_version, "--wheel", "--quiet"]
+
+    if not run_command(cmd, env=get_build_env()):
+        return None
+
+    # Find the ABI3 wheel (always tagged cp312-abi3 regardless of build Python version)
+    wheel_pattern = get_wheel_pattern(py_version, abi3=True)
+    wheels = list(dist_dir.glob(wheel_pattern))
+    if not wheels:
+        print(f"✗ Could not find ABI3 wheel matching {wheel_pattern}", file=sys.stderr)
+        return None
+
+    wheel = wheels[0]
+
+    # Repair wheel with auditwheel for manylinux compatibility (Linux only)
+    if platform.system() == "Linux":
+        wheel = repair_wheel_linux(dist_dir, wheel, py_version, abi3=True)
+        if not wheel:
+            return None
+
+    return wheel
+
+
+def build_wheel_for_version(dist_dir: Path, py_version: str) -> Path | None:
+    """Build a wheel for a specific Python version (non-ABI3)."""
+    cmd = ["uv", "build", "--python", py_version, "--wheel", "--quiet"]
+
+    if not run_command(cmd, env=get_build_env()):
+        return None
+
+    # Find the wheel for this version
+    wheel_pattern = get_wheel_pattern(py_version, abi3=False)
+    wheels = list(dist_dir.glob(wheel_pattern))
+    if not wheels:
+        print(f"✗ Could not find wheel for Python {py_version}", file=sys.stderr)
+        return None
+
+    wheel = wheels[0]
+
+    # Repair wheel with auditwheel for manylinux compatibility (Linux only)
+    if platform.system() == "Linux":
+        wheel = repair_wheel_linux(dist_dir, wheel, py_version, abi3=False)
+        if not wheel:
+            return None
+
+    return wheel
+
+
+def repair_wheel_linux(
+    dist_dir: Path, wheel: Path, py_version: str, abi3: bool
+) -> Path | None:
+    """Repair a wheel with auditwheel for manylinux compatibility (Linux only)."""
+    repair_cmd = [
+        "uv",
+        "run",
+        "auditwheel",
+        "repair",
+        str(wheel),
+        "-w",
+        str(dist_dir),
+    ]
+    if not run_command(repair_cmd):
+        return None
+
+    # Find the repaired wheel (it will have a different name)
+    wheel_pattern = get_wheel_pattern(py_version, abi3=abi3)
+    all_wheels = list(dist_dir.glob(wheel_pattern))
+    repaired_wheels = [w for w in all_wheels if "linux_x86_64" not in str(w)]
+    if not repaired_wheels:
+        print(
+            f"✗ Could not find repaired (manylinux) wheel for Python {py_version}",
+            file=sys.stderr,
+        )
+        return None
+
+    repaired_wheel = repaired_wheels[0]
+
+    # Remove the unrepaired linux_x86_64 wheels
+    for w in all_wheels:
+        if "linux_x86_64" in str(w):
+            w.unlink()
+
+    return repaired_wheel
+
+
+def test_wheel(wheel: Path, py_version: str) -> bool:
+    """Test a wheel with pytest."""
+    # --isolated: avoid .venv conflicts
+    # --no-project: don't build from source in current directory, use the wheel
+    # --refresh-package: force uv to not use cached old versions
+    test_cmd = [
+        "uv",
+        "run",
+        "--isolated",
+        "--no-project",
+        "--refresh-package",
+        "aeg",
+        "--python",
+        py_version,
+        "--with",
+        str(wheel),
+        "--with",
+        "pytest",
+        "pytest",
+        "tests/",
+    ]
+    return run_command(test_cmd)
+
+
+def run_benchmark(wheel: Path, py_version: str) -> bool:
+    """Run benchmark for a wheel."""
+    # --isolated: avoid .venv conflicts
+    # --no-project: don't build from source in current directory, use the wheel
+    # --refresh-package: force uv to not use cached old versions
+    bench_cmd = [
+        "uv",
+        "run",
+        "--isolated",
+        "--no-project",
+        "--refresh-package",
+        "aeg",
+        "--python",
+        py_version,
+        "--with",
+        str(wheel),
+        "-m",
+        "aeg.benchmark",
+    ]
+    return run_command(bench_cmd)
+    return True
+
+
 def main():
     """Build wheels for all supported Python versions."""
+    # Windows CI consoles default to cp1252; emit ✓/✗/⚠ as UTF-8
+    for stream in (sys.stdout, sys.stderr):
+        if hasattr(stream, "reconfigure"):
+            stream.reconfigure(encoding="utf-8", errors="replace")
+
+    # CI matrix jobs build per-platform wheels but never upload; a separate
+    # publish job uploads everything once all platforms have succeeded.
+    no_upload = "--no-upload" in sys.argv
+
     repo_root = Path(__file__).parent.parent
     dist_dir = repo_root / "dist"
 
@@ -146,14 +326,15 @@ def main():
         return 1
 
     # Run ruff to check and fix any issues
-    if not run_command(
-        ["uv", "run", "ruff", "check", "--fix", "."], "Running ruff check --fix"
-    ):
+    print(f"\n{'=' * 70}")
+    print("Linting and formatting")
+    print(f"{'=' * 70}")
+    if not run_command(["uv", "run", "ruff", "check", "--fix", "."]):
         print("✗ Ruff check failed", file=sys.stderr)
         return 1
 
     # Run ruff format
-    if not run_command(["uv", "run", "ruff", "format", "."], "Running ruff format"):
+    if not run_command(["uv", "run", "ruff", "format", "."]):
         print("✗ Ruff format failed", file=sys.stderr)
         return 1
 
@@ -172,7 +353,11 @@ def main():
         f"Packaging aeg-{version}"
         + (" for release" if is_release else " (not release)")
     )
-    print(f"Building wheels for Python versions: {', '.join(PYTHON_VERSIONS)}")
+    print(f"Building: 1 ABI3 wheel (for Python {', '.join(ABI3_COVERED_VERSIONS)})")
+    print(
+        f"        + {len(NON_ABI3_VERSIONS)} non-ABI3 wheels ({', '.join(NON_ABI3_VERSIONS)})"
+    )
+    print(f"Testing/benchmarking: {len(ALL_PYTHON_VERSIONS)} Python versions")
     print(f"Output directory: {dist_dir}", end=" ")
 
     # Clean dist directory
@@ -181,137 +366,89 @@ def main():
         shutil.rmtree(dist_dir)
     else:
         print("(created)")
+
+    # Clean build directory to remove stale CFFI-generated C code and .so files
+    build_dir = repo_root / "build"
+    if build_dir.exists():
+        print(f"Cleaning build directory: {build_dir}")
+        shutil.rmtree(build_dir)
+
+    # Build distributions
+    print(f"\n{'=' * 70}")
+    print("Building distributions")
     print(f"{'=' * 70}")
 
     # Build source distribution first
-    if not run_command(
-        ["uv", "build", "--sdist", "--quiet"], "Building source distribution"
-    ):
+    if not run_command(["uv", "build", "--sdist", "--quiet"], env=get_build_env()):
         print("✗ Source distribution build failed", file=sys.stderr)
         return 1
 
     failed_builds = []
+    failed_tests = []
     successful_wheels = []
+    wheel_for_version = {}  # Map Python version to wheel path
 
-    for py_version in PYTHON_VERSIONS:
-        # Build wheel
-        description = f"Building wheel for Python {py_version}"
-        cmd = ["uv", "build", "--python", py_version, "--wheel", "--quiet"]
+    # Build ABI3 wheel (once, works for all GIL-enabled versions)
+    abi3_wheel = build_abi3_wheel(dist_dir, ABI3_BUILD_VERSION)
+    if abi3_wheel:
+        successful_wheels.append(abi3_wheel)
+        # This wheel works for all ABI3-covered versions
+        for py_version in ABI3_COVERED_VERSIONS:
+            wheel_for_version[py_version] = abi3_wheel
+    else:
+        failed_builds.append(f"abi3 (built with {ABI3_BUILD_VERSION})")
 
-        if not run_command(cmd, description):
-            failed_builds.append(py_version)
-            continue
-
-        # Find the wheel for this version
-        if py_version.startswith("pypy"):
-            # PyPy wheels use pp3XX format
-            wheel_pattern = (
-                f"aeg-*-pp{py_version.replace('pypy', '').replace('.', '')}-*.whl"
-            )
+    # Build non-ABI3 wheels (free-threaded and PyPy)
+    for py_version in NON_ABI3_VERSIONS:
+        wheel = build_wheel_for_version(dist_dir, py_version)
+        if wheel:
+            successful_wheels.append(wheel)
+            wheel_for_version[py_version] = wheel
         else:
-            wheel_pattern = (
-                f"aeg-*-cp{py_version.replace('.', '').replace('t', '')}-*.whl"
-            )
-        wheels = list(dist_dir.glob(wheel_pattern))
-        if not wheels:
-            print(f"✗ Could not find wheel for Python {py_version}", file=sys.stderr)
             failed_builds.append(py_version)
+
+    # Test and benchmark each Python version with its appropriate wheel
+    print(f"\n{'=' * 70}")
+    print("Testing and benchmarking")
+    print(f"{'=' * 70}")
+
+    for py_version in ALL_PYTHON_VERSIONS:
+        wheel = wheel_for_version.get(py_version)
+        if not wheel:
+            # No wheel available for this version (build failed)
             continue
 
-        wheel = wheels[0]
-
-        # Repair wheel with auditwheel for manylinux compatibility (Linux only)
-        if platform.system() == "Linux":
-            repair_cmd = [
-                "uv",
-                "run",
-                "auditwheel",
-                "repair",
-                str(wheel),
-                "-w",
-                str(dist_dir),
-            ]
-            if not run_command(
-                repair_cmd, f"Repairing wheel for Python {py_version} with auditwheel"
-            ):
-                print(
-                    f"✗ Auditwheel repair failed for Python {py_version}",
-                    file=sys.stderr,
-                )
-                failed_builds.append(py_version)
-                continue
-
-            # Find the repaired wheel (it will have a different name)
-            all_wheels = list(
-                dist_dir.glob(f"aeg-*-cp{py_version.replace('.', '')}-*.whl")
-            )
-            repaired_wheels = [w for w in all_wheels if "linux_x86_64" not in str(w)]
-            if not repaired_wheels:
-                print(
-                    f"✗ Could not find repaired (manylinux) wheel for Python {py_version}",
-                    file=sys.stderr,
-                )
-                failed_builds.append(py_version)
-                continue
-
-            wheel = repaired_wheels[0]  # Use the repaired wheel for testing
-
-            # Remove the unrepaired linux_x86_64 wheels
-            for w in all_wheels:
-                if "linux_x86_64" in str(w):
-                    w.unlink()
-
-        # Test the wheel with pytest (use --isolated to avoid .venv conflicts)
-        test_cmd = [
-            "uv",
-            "run",
-            "--isolated",
-            "--python",
-            py_version,
-            "--with",
-            str(wheel),
-            "--with",
-            "pytest",
-            "pytest",
-        ]
-        if not run_command(
-            test_cmd, f"Testing wheel for Python {py_version} with pytest"
-        ):
-            print(f"✗ Tests failed for Python {py_version}", file=sys.stderr)
-            failed_builds.append(py_version)
+        # Test the wheel with pytest
+        if not test_wheel(wheel, py_version):
+            failed_tests.append(py_version)
             continue
 
-        # Run benchmark (use --isolated to avoid .venv conflicts)
-        bench_cmd = [
-            "uv",
-            "run",
-            "--isolated",
-            "--python",
-            py_version,
-            "--with",
-            str(wheel),
-            "-m",
-            "aeg.benchmark",
-        ]
-        if not run_command(bench_cmd, f"Running benchmark for Python {py_version}"):
-            print(f"✗ Benchmark failed for Python {py_version}", file=sys.stderr)
-            failed_builds.append(py_version)
+        # Run benchmark
+        if not run_benchmark(wheel, py_version):
+            failed_tests.append(py_version)
             continue
 
-        successful_wheels.append(wheel)
-
     # Summary
     print(f"\n{'=' * 70}")
     print("BUILD SUMMARY")
     print(f"{'=' * 70}")
     print(
-        f"Successful builds: sdist and {len(successful_wheels)}/{len(PYTHON_VERSIONS)} wheels"
+        f"Successful builds: sdist and {len(successful_wheels)} wheels "
+        f"(1 abi3 + {len(NON_ABI3_VERSIONS)} non-abi3)"
+    )
+    print(
+        f"Tests/benchmarks passed: {len(ALL_PYTHON_VERSIONS) - len(failed_tests) - len(failed_builds)}/{len(ALL_PYTHON_VERSIONS)} Python versions"
     )
 
     if failed_builds:
         print(f"\nFailed builds: {len(failed_builds)}")
-        for version in failed_builds:
-            print(f"  ✗ Python {version}")
+        for failed_version in failed_builds:
+            print(f"  ✗ {failed_version}")
+
+    if failed_tests:
+        print(f"\nFailed tests/benchmarks: {len(failed_tests)}")
+        for failed_version in failed_tests:
+            print(f"  ✗ Python {failed_version}")
 
     if not successful_wheels:
         print("\n✗ No successful wheels to upload")
@@ -325,6 +462,10 @@ def main():
         print(f"  - {file.name}")
 
     # Only upload if this is a clean release version
+    if no_upload:
+        print("\n--no-upload given; upload to PyPI skipped.")
+        return 0
+
     if not is_release:
         print(make_release_message(version))
         return 0
@@ -343,4 +484,7 @@ def main():
 
 
 if __name__ == "__main__":
-    sys.exit(main())
+    try:
+        sys.exit(main())
+    except KeyboardInterrupt:
+        sys.exit(1)