Fix in-memory cache update races and memory accounting

.agents/skills/foundatio/SKILL.md

@@ -265,6 +265,8 @@ Use `Foundatio.Xunit.v3` for test logging and DI integration. Two base classes:
 - **`TestWithLoggingBase`** -- lightweight, no DI container. `_logger` (`ILogger`) for logging; `Log` (`ILoggerFactory`) for passing to Foundatio services.
 - **`TestLoggerBase`** -- full DI via `TestLoggerFixture`. Override `ConfigureServices` to register services. `Log` (`ILogger`) for logging; `TestLogger` (`ILoggerFactory`) for passing to Foundatio services.
 
+Use `Microsoft.Extensions.Time.Testing.FakeTimeProvider` for expiration and timer tests. Inject the same instance into collaborating caches and lock providers, and advance it explicitly. A real-time timeout may bound an awaited completion, but should not simulate cache expiration. Keep shared test-harness coverage when adding provider-specific fake-time tests. Test observable behavior rather than the number of clock reads.
+
 ```csharp
 using Foundatio.Caching;
 using Foundatio.Xunit;
@@ -306,14 +308,17 @@ public class OrderServiceTests : TestLoggerBase
 
 - **Lock returns null**: `TryAcquireAsync` returns `null` when the lock cannot be acquired -- always guard with `is not null` before doing work. `AcquireAsync` throws `LockAcquisitionTimeoutException` instead of returning null.
 - **Dispose streams and locks**: `ILock` is `IAsyncDisposable` -- use `await using`. Streams from `GetFileStreamAsync` are `IDisposable` -- use `using var`.
-- **Cache TTL floor**: Expiration values below 5ms are treated as already-expired and the key is silently removed. If you compute TTL dynamically (e.g., `expiresAt - now`), guard against near-zero values.
+- **Cache TTL floor**: Expiration values below 5ms are treated as already expired. Key writes remove the key; `ListAddAsync` removes only the supplied values. Guard dynamically computed TTLs against near-zero values.
 - **Cache `GetAsync` returns `CacheValue<T>`**: Check `result.HasValue` before accessing `result.Value`. A missing key returns `HasValue = false`, not an exception.
 - **Cache stampede (thundering herd)**: The cache-aside pattern (`Get` -> miss -> load -> `Set`) is vulnerable to stampedes when a popular key expires and many callers regenerate simultaneously. Use `CacheLockProvider` to serialize regeneration: acquire a lock keyed on the cache key, double-check the cache after acquiring, and only then call the backing store. See the [Cache Stampede Protection](https://foundatio.readthedocs.io/guide/caching.html#cache-stampede-protection) docs for the full pattern.
 - **Queue auto-complete**: `QueueJobBase<T>` auto-completes entries based on `JobResult` by default. Set `AutoComplete = false` only when you need manual `CompleteAsync()`/`AbandonAsync()` control. Manual `DequeueAsync` does NOT auto-complete.
 - **GetQueueEntryLockAsync error handling**: If `GetQueueEntryLockAsync` returns `null`, the queue entry is abandoned. If it throws, the entry is also abandoned and a `JobResult.FromException` is returned. Use `TryAcquireAsync` (not `AcquireAsync`) in your override since the return type is `Task<ILock?>`.
 - **Failure semantics depend on job type**: `JobResult` only has `IsSuccess` -- there is no separate "failed but don't retry" status. For **queue-processed jobs** (`QueueJobBase<T>.ProcessQueueEntryAsync`, or setting `context.Result` in a `WorkItemJob` handler), a non-success result triggers `AbandonAsync`, which re-queues the entry and eventually dead-letters it after `Retries` is exhausted -- reserve `FailedWithMessage`/`FromException` for transient errors you want retried, and log + return `JobResult.Success`/`SuccessWithMessage(...)` for permanent errors to avoid a pointless retry loop. For **standalone/manual jobs** (`JobBase`, a one-off `RunAsync()`/`RunInConsoleAsync()` run, or scheduled/cron jobs via `Foundatio.Extensions.Hosting`), there is no built-in retry or dead letter queue -- a failed result just produces an error-level log, a non-zero exit code from `RunInConsoleAsync`, or a failed entry in the job run history. Returning `FailedWithMessage`/`FromException` there is correct even for permanent errors, since nothing inside Foundatio will retry it.
 - **JobWithLockBase vs manual locking**: Use `JobWithLockBase` when the entire run must be single-instance (leader election). Use manual `ILockProvider.AcquireAsync` inside `JobBase` for finer-grained locking within a job.
 - **JobContext.RenewLockAsync**: Call in long-running jobs (both `JobBase` and `QueueJobBase`) to prevent lock expiration mid-processing.
+- **In-memory expired entries**: Writes treat an expired entry as missing before maintenance sweeps it (as Redis does): increments and `SetIfHigher`/`SetIfLower` start fresh, `ReplaceAsync`, `SetExpirationAsync`, `RemoveIfEqualAsync` and `ReplaceIfEqualAsync` miss. The write reclaims the expired entry and raises `ItemExpired` once. Successful live conditional removal and empty-list removal free memory without that event. Size-rejected updates preserve value, TTL, and tracked size.
+- **In-memory numeric TTLs**: `SetIfHigherAsync` and `SetIfLowerAsync` apply the requested expiration even when the numeric value is unchanged. A zero return value does not mean the TTL was preserved; `null` clears it. Size-rejected updates remain unchanged.
+- **In-memory lists and cloning**: List writes preserve supported dictionary types and comparers; copying an unsupported `IDictionary` throws `NotSupportedException` without changing the entry. Batch large updates because cloning, custom sizing, and raw dictionary access can require whole-list copies. `Items` honors `CloneValues` without affecting LRU. Collection snapshots do not isolate shared mutable elements when cloning is disabled. See the [caching guide](https://foundatio.readthedocs.io/guide/caching.html#in-memory-update-behavior) for details.
 - **Register as singletons**: All infrastructure services (`ICacheClient`, `IMessageBus`, `IQueue<T>`, `IFileStorage`, `ILockProvider`) maintain internal state and connections -- always register as singletons.
 - **CacheLockProvider + IMessageBus**: `IMessageBus` is optional but recommended. Without it, lock release falls back to polling. With it, locks are released instantly via pub/sub notification.
 - **In-memory for tests**: In-memory implementations follow the same interfaces as production providers, but some behaviors differ (delivery delay limits, ordering, deduplication, group ids). Swap via DI for fast, isolated unit tests, and check [provider behavioral gaps](https://foundatio.dev/guide/provider-behavioral-gaps) before relying on provider-specific behavior.
@@ -331,7 +336,7 @@ public class OrderServiceTests : TestLoggerBase
 
 ### Serializers
 
-`ITextSerializer` extends `ISerializer` for human-readable formats (JSON). `ISerializer` covers binary formats. Default is `SystemTextJsonSerializer` (included in core).
+`ITextSerializer` extends `ISerializer` for human-readable formats (JSON). `ISerializer` covers binary and text serialization. Default is `SystemTextJsonSerializer` (included in core).
 
 | Package | Provides |
 | ------- | -------- |

benchmarks/InMemoryListBenchmarks.cs

@@ -0,0 +1,107 @@
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using System.Threading.Tasks;
+using BenchmarkDotNet.Attributes;
+using Foundatio.Caching;
+
+namespace Foundatio.Benchmarks;
+
+[MemoryDiagnoser]
+public class InMemoryListBenchmarks
+{
+    public enum SizingMode { None, Fixed, Custom, Dynamic }
+
+    [Params(false, true)]
+    public bool CloneValues { get; set; }
+
+    [Params(10, 1000)]
+    public int Count { get; set; }
+
+    [Params(SizingMode.None, SizingMode.Fixed, SizingMode.Custom, SizingMode.Dynamic)]
+    public SizingMode Sizing { get; set; }
+
+    private InMemoryCacheClient _cache = null!;
+    private int[] _initial = null!;
+    private int[] _batch = null!;
+    private readonly int[] _item = [-1];
+    private readonly int[] _workers = [0, 1, 2, 3];
+
+    [GlobalSetup]
+    public async Task Setup()
+    {
+        _cache = new InMemoryCacheClient(o =>
+        {
+            o.CloneValues(CloneValues).TimeProvider(new FixedTimeProvider());
+            return Sizing switch
+            {
+                SizingMode.Fixed => o.WithFixedSizing(100000000, 100),
+                SizingMode.Custom => o.MaxMemorySize(100000000).SizeCalculator(_ => 100),
+                SizingMode.Dynamic => o.WithDynamicSizing(100000000),
+                _ => o
+            };
+        });
+        _initial = Enumerable.Range(0, Count).ToArray();
+        _batch = Enumerable.Range(0, Count / 2).ToArray();
+        await _cache.ListAddAsync("set", _initial);
+    }
+
+    [GlobalCleanup]
+    public void Cleanup() => _cache.Dispose();
+
+    [Benchmark]
+    public async Task<long> AddRemove()
+    {
+        await _cache.ListAddAsync("set", _item);
+        return await _cache.ListRemoveAsync("set", _item);
+    }
+
+    [Benchmark]
+    public async Task<long> BulkInsert()
+    {
+        await _cache.RemoveAsync("bulk");
+        return await _cache.ListAddAsync("bulk", _initial);
+    }
+
+    [Benchmark]
+    public async Task<long> BulkRemoveRestore()
+    {
+        await _cache.ListRemoveAsync("set", _batch);
+        return await _cache.ListAddAsync("set", _batch);
+    }
+
+    [Benchmark]
+    public Task ContendedWrites() => Parallel.ForEachAsync(_workers, async (worker, token) =>
+    {
+        int[] item = [-worker - 1];
+        await Task.Yield();
+        await _cache.ListAddAsync("set", item);
+        await _cache.ListRemoveAsync("set", item);
+    });
+
+    [Benchmark]
+    public async Task<long> FirstModification()
+    {
+        await _cache.RemoveAsync("cold");
+        await _cache.ListAddAsync("cold", _initial);
+        return await _cache.ListAddAsync("cold", _item);
+    }
+
+    [Benchmark]
+    public Task<CacheValue<ICollection<int>>> Read() => _cache.GetListAsync<int>("set");
+
+    [Benchmark]
+    public Task<long> RemoveAbsent() => _cache.ListRemoveAsync("set", _item);
+
+    [Benchmark]
+    public async Task<CacheValue<ICollection<int>>> WriteThenRead()
+    {
+        await AddRemove();
+        return await _cache.GetListAsync<int>("set");
+    }
+
+    private sealed class FixedTimeProvider : TimeProvider
+    {
+        public override DateTimeOffset GetUtcNow() => new(2026, 1, 1, 0, 0, 0, TimeSpan.Zero);
+    }
+}

benchmarks/InMemoryWriteBenchmarks.cs

@@ -0,0 +1,40 @@
+using System.Threading.Tasks;
+using BenchmarkDotNet.Attributes;
+using Foundatio.Caching;
+
+namespace Foundatio.Benchmarks;
+
+/// <summary>
+/// Scalar writes on a key that holds a live value, which is the common case. Every write goes through
+/// <c>UpdateEntry</c>, so these show what its wrapper costs.
+/// </summary>
+[MemoryDiagnoser]
+public class InMemoryWriteBenchmarks
+{
+    private InMemoryCacheClient _cache = null!;
+
+    [GlobalSetup]
+    public async Task Setup()
+    {
+        _cache = new InMemoryCacheClient(o => o.CloneValues(false));
+        await _cache.SetAsync("key", 1L);
+    }
+
+    [GlobalCleanup]
+    public void Cleanup() => _cache.Dispose();
+
+    [Benchmark]
+    public Task<bool> Set() => _cache.SetAsync("key", 1L);
+
+    [Benchmark]
+    public Task<bool> Add_ExistingKey() => _cache.AddAsync("key", 1L);
+
+    [Benchmark]
+    public Task<long> Increment() => _cache.IncrementAsync("key", 1L);
+
+    [Benchmark]
+    public Task<long> SetIfHigher_NotHigher() => _cache.SetIfHigherAsync("key", 0L);
+
+    [Benchmark]
+    public Task<bool> ReplaceIfEqual() => _cache.ReplaceIfEqualAsync("key", 1L, 1L);
+}