cache: shared-memory-backed Dir for fast restart

CMakeLists.txt

@@ -574,6 +574,9 @@ check_symbol_exists(mcheck_pedantic mcheck.h HAVE_MCHECK_PEDANTIC)
 check_symbol_exists(posix_fadvise fcntl.h HAVE_POSIX_FADVISE)
 check_symbol_exists(posix_fallocate fcntl.h HAVE_POSIX_FALLOCATE)
 check_symbol_exists(posix_madvise sys/mman.h HAVE_POSIX_MADVISE)
+# On glibc < 2.34 shm_open/shm_unlink live in librt, which ATS does not link, so the cache shm feature is compiled out
+# there. This is a link test, so it fails exactly on those platforms.
+check_symbol_exists(shm_open sys/mman.h HAVE_SHM_OPEN)
 check_symbol_exists(accept4 sys/socket.h HAVE_ACCEPT4)
 check_symbol_exists(eventfd sys/eventfd.h HAVE_EVENTFD)
 check_symbol_exists(sysconf unistd.h HAVE_SYSCONF)
@@ -610,6 +613,11 @@ if(HAVE_IOURING AND USE_IOURING)
   set(TS_USE_LINUX_IO_URING 1)
 endif(HAVE_IOURING AND USE_IOURING)
 
+option(ENABLE_CACHE_SHM "Host the cache directory in POSIX shared memory for fast restart" 1)
+if(HAVE_SHM_OPEN AND ENABLE_CACHE_SHM)
+  set(TS_USE_CACHE_SHM 1)
+endif(HAVE_SHM_OPEN AND ENABLE_CACHE_SHM)
+
 list(APPEND CMAKE_REQUIRED_LIBRARIES pthread)
 check_symbol_exists(pthread_getname_np pthread.h HAVE_PTHREAD_GETNAME_NP)
 check_symbol_exists(pthread_get_name_np pthread.h HAVE_PTHREAD_GET_NAME_NP)

doc/admin-guide/files/records.yaml.en.rst

@@ -3068,6 +3068,108 @@ RAM Cache
    Compression runs on task threads. To use more cores for RAM cache
    compression, increase :ts:cv:`proxy.config.task_threads`.
 
+.. _admin-cache-shm-fast-restart:
+
+Shared Memory Fast Restart
+==========================
+
+|TS| can optionally keep the cache directory -- the in-memory index that maps
+cached objects to their location on disk -- in POSIX shared memory so that it
+survives a process restart. On a normal start the directory is read from disk
+and, for a large cache, rebuilt in memory before the cache comes online. When
+this feature is enabled and the previous instance shut down cleanly, the new
+instance attaches the existing shared memory segments and skips that work,
+bringing the cache online much faster.
+
+The shared memory directory is only an optimization for restart time; the
+on-disk cache always remains the source of truth. A new instance discards the
+segments and falls back to reading the directory from disk whenever they cannot
+be trusted, including when:
+
+- the previous instance did not shut down cleanly (for example, it crashed),
+- the |TS| binary's directory structures changed (an ABI mismatch, such as
+  after an upgrade), or
+- the shared memory schema version changed.
+
+A change to the on-disk storage layout in :file:`storage.yaml` does *not*
+discard everything. Each cache stripe attaches by its own identity, so stripes
+that still exist are fast-attached, added or resized stripes are rebuilt from
+disk, and segments for stripes that no longer exist are reclaimed.
+
+Segments left over from a crash can be inspected or removed with
+``traffic_ctl cache shm status`` and ``traffic_ctl cache shm clear``, which act
+directly on the shared memory objects whether or not |TS| is running.
+
+.. note::
+
+   This is an experimental feature, disabled by default. All of its settings
+   take effect only on a restart of |TS|.
+
+.. ts:cv:: CONFIG proxy.config.cache.shm.enabled INT 0
+
+   Enables the shared memory cache directory described above. When ``0`` (the
+   default), the cache directory is always read from disk on start.
+
+   The feature requires ``shm_open()`` in libc, which |TS| checks for at build
+   time. Where it is absent -- glibc older than 2.34, which keeps it in
+   ``librt`` -- the feature is compiled out and this setting has no effect: |TS|
+   logs a warning at startup and every stripe uses a heap directory. That
+   applies to RHEL/Rocky 8, Ubuntu 20.04 and Debian 11 among others; run
+   ``traffic_layout info | grep TS_USE_CACHE_SHM`` to check a given build.
+
+.. ts:cv:: CONFIG proxy.config.cache.shm.name_prefix STRING ats
+
+   The word used to name the POSIX shared memory objects, which on Linux appear
+   under ``/dev/shm``. Set only the middle word (default ``ats``); |TS| frames it
+   as ``/<word>-`` so the leading ``/`` that POSIX requires and the trailing
+   ``-`` separator cannot be mis-typed. With the default the control segment is
+   named ``/ats-control`` and each per-stripe directory segment ``/ats-s<N>``
+   (for example ``/ats-s0``). Any stray framing characters are trimmed, so a
+   value carried over from an older release (such as ``/ats-``) still resolves to
+   the same names. Give each |TS| instance sharing a host a distinct word so
+   their segments do not collide.
+
+   Renaming this value does not remove segments created under the old prefix:
+   |TS| only manages segments under the *current* prefix, so the old ``/dev/shm``
+   objects linger until cleared manually with ``traffic_ctl cache shm clear
+   --prefix <old-word>`` (or a host reboot).
+
+.. ts:cv:: CONFIG proxy.config.cache.shm.use_hugepages INT 0
+
+   When enabled (``1``), |TS| attempts to back the shared memory directory with
+   huge pages, which cut the cost of tearing down the large directory's page
+   tables at process exit (and ease TLB pressure). This requires the shared
+   memory to be eligible for huge pages (for example, ``/dev/shm`` mounted with
+   huge page support on Linux). When it is not, |TS| logs a debug message under
+   the ``cache_shm`` tag and transparently falls back to ordinary pages, so
+   enabling this is always safe.
+
+   This advises transparent huge pages (``MADV_HUGEPAGE``) on the mapping; the
+   reserved ``MAP_HUGETLB`` pages used by the global hugepage allocator cannot
+   back a ``tmpfs``-backed shared memory segment. To avoid silently downgrading a
+   box that already runs with :ts:cv:`proxy.config.allocator.hugepages` enabled,
+   |TS| turns this on automatically when the global allocator is enabled and this
+   record is left at its default; set it to ``0`` explicitly to opt out.
+
+.. ts:cv:: CONFIG proxy.config.cache.shm.purge_stale_on_start INT 0
+
+   When enabled (``1``) and :ts:cv:`proxy.config.cache.shm.enabled` is ``0``,
+   |TS| removes any leftover shared memory segments for
+   :ts:cv:`proxy.config.cache.shm.name_prefix` at startup (the ``<prefix>control``
+   segment and the per-stripe segments it lists). This guards against two
+   hazards of running with the feature disabled after it had been enabled:
+
+   - the leftover segments keep consuming memory (for example ``/dev/shm`` on
+     Linux) even though the disabled instance never reads them, and
+   - a later run with the feature re-enabled would otherwise fast-attach a
+     directory that went stale while |TS| ran disabled and wrote only to disk.
+
+   The purge is skipped if a live process still owns the segments (a concurrent
+   instance using the same prefix), and it never blocks startup. It has no
+   effect when the feature is enabled, when no ``<prefix>control`` segment
+   exists, or when set to ``0`` (the default). ``traffic_ctl cache shm clear``
+   performs the same cleanup on demand.
+
 .. _admin-heuristic-expiration:
 
 Heuristic Expiration

doc/developer-guide/cache-architecture/index.en.rst

@@ -41,5 +41,6 @@ understanding and modifying the source.
    api-functions.en
    consistency.en
    ram-cache.en
+   shm-fast-restart.en
    cache-tool.en
    tiered-storage.en