std/cache

Bounded in-memory caches: LruCache<K, V> evicts the least recently used entry once it holds capacity entries, and TtlCache<K, V> expires each entry a fixed time after it was put, optionally with an LRU capacity as well. Both are built on std/collections' OrderedMap, and every operation is amortized O(1) except TtlCache.purge.

from "std/cache" import { LruCache, TtlCache }

LruCache

get, modify and put count as a use; peek and contains do not.

from "std/cache" import { LruCache }

fn main(): i32 {
    var c = LruCache<string, i64>.new(2)
    c.put("a", 1)
    c.put("b", 2)
    print(c.get("a")!)                      // 1, and "a" is now the most recent
    c.put("c", 3)                           // evicts "b", the least recently used
    print(c.contains("b"))                  // false
    return 0
}

TtlCache

Expiry is lazy: an expired entry is never served, and is dropped when a lookup meets it or purge() sweeps. Until then it still counts in len and toward the capacity. putFor overrides the ttl for one entry.

The clock is epochMillis from std/time; setClockMs swaps in another, which is how a test steps time without sleeping. The closure is move, so it reads a global rather than a captured local:

from "std/cache" import { TtlCache }
from "std/time" import { Duration }

var fakeNow: i64 = 0

fn main(): i32 {
    var c = TtlCache<string, string>.withCapacity(Duration.secs(30), 1000)
    c.setClockMs(move (): i64 => fakeNow)
    c.put("session", "alice")
    fakeNow += 10000
    print(c.get("session")!)                // alice
    fakeNow += 30000
    print(c.get("session").isNone())        // true: expired and dropped
    print(c.purge())                        // 0: nothing left to sweep
    return 0
}

For a value carrying Drop, get (a copy) is not available; read it in place with modify.

API reference

LruCache

pub struct LruCache<K, V>

A cache holding at most capacity entries that evicts the least recently used one to make room. get, modify and put count as a use; contains and peek do not.

Fields: map: OrderedMap<K, V>, cap: i64.

LruCache.capacity

fn LruCache.capacity(self: &LruCache): i64

The most entries the cache holds.

LruCache.clear

fn LruCache.clear(self: &mut LruCache): void

Drop every entry.

LruCache.contains

fn LruCache.contains(self: &LruCache, key: &K): bool

Whether key is cached, without marking it used.

LruCache.get

fn LruCache.get(self: &mut LruCache, key: &K): Option<V>

A copy of the value under key, marking it most recently used; None on a miss.

LruCache.len

fn LruCache.len(self: &LruCache): i64

Number of cached entries.

LruCache.modify

fn LruCache.modify(self: &mut LruCache, key: &K, f: (&mut V) => void): bool

Run f on the value under key in place, marking it most recently used; false (and f not run) on a miss.

LruCache.new

fn LruCache.new(capacity: i64): LruCache<K, V>

An empty cache holding at most capacity entries (at least 1).

A generic static takes its type arguments: LruCache<string, i64>.new(100).

LruCache.peek

fn LruCache.peek(self: &LruCache, key: &K): Option<V>

A copy of the value under key without marking it used.

LruCache.put

fn LruCache.put(self: &mut LruCache, key: K, value: V): void

Insert or replace the value under key and mark it most recently used. When that makes the cache hold more than its capacity, the least recently used entry is dropped.

LruCache.remove

fn LruCache.remove(self: &mut LruCache, key: &K): Option<V>

Remove key and hand back its value, or None when absent.

TtlCache

pub struct TtlCache<K, V>

A cache whose entries expire a fixed time after they were put (or a per-entry time with putFor), optionally also bounded to capacity entries with LRU eviction.

Expiry is lazy: an expired entry is never served, and is dropped when a lookup meets it or when purge sweeps. Until then it still counts in len and toward the capacity, so a cache that is written but rarely read should purge now and then.

The clock is epochMillis from std/time unless setClockMs swaps in another, which is how a test steps time without sleeping.

Fields:

• map: OrderedMap<K, Stamped<V>>
• ttlMs: i64
• cap: i64: 0 means unbounded.
• clock: move () => i64

TtlCache.clear

fn TtlCache.clear(self: &mut TtlCache): void

Drop every entry.

TtlCache.contains

fn TtlCache.contains(self: &TtlCache, key: &K): bool

Whether key holds an unexpired value. Does not mark it used or drop it.

TtlCache.get

fn TtlCache.get(self: &mut TtlCache, key: &K): Option<V>

A copy of the value under key if it has not expired, marking it most recently used; an expired entry is dropped and reads as None.

TtlCache.getWithin

fn TtlCache.getWithin(self: &mut TtlCache, key: &K, maxAge: &Duration): Option<V>

A copy of the value under key if it has not expired and was put less than maxAge ago, marking it most recently used; None otherwise. An entry too old for maxAge but not yet expired stays, untouched, for a later read with a looser bound: put with the longest age any reader accepts (say 60 s for a stale-if-error fallback) and read the fresh path with a shorter one (2 s).

TtlCache.len

fn TtlCache.len(self: &TtlCache): i64

Number of entries held, counting expired ones not yet dropped.

TtlCache.modify

fn TtlCache.modify(self: &mut TtlCache, key: &K, f: (&mut V) => void): bool

Run f on the value under key in place if it has not expired, marking it most recently used; false (and f not run) on a miss or an expired entry.

TtlCache.new

fn TtlCache.new(ttl: Duration): TtlCache<K, V>

An empty, unbounded cache whose entries live for ttl.

A generic static takes its type arguments: TtlCache<string, i64>.new(Duration.secs(30)).

TtlCache.purge

fn TtlCache.purge(self: &mut TtlCache): i64

Drop every expired entry now; returns how many went. O(n).

TtlCache.put

fn TtlCache.put(self: &mut TtlCache, key: K, value: V): void

Insert or replace the value under key, live for the cache's ttl from now.

TtlCache.putFor

fn TtlCache.putFor(self: &mut TtlCache, key: K, value: V, ttl: &Duration): void

Insert or replace the value under key, live for ttl from now instead of the cache's default.

TtlCache.remove

fn TtlCache.remove(self: &mut TtlCache, key: &K): Option<V>

Remove key and hand back its value, expired or not; None when absent.

TtlCache.setClockMs

fn TtlCache.setClockMs(self: &mut TtlCache, nowMs: move () => i64): void

Read the time from nowMs (milliseconds on any fixed origin) instead of the wall clock. Expiry times already stored keep the old clock's origin, so swap it before the first put.

TtlCache.withCapacity

fn TtlCache.withCapacity(ttl: Duration, capacity: i64): TtlCache<K, V>

An empty cache whose entries live for ttl and that holds at most capacity entries (at least 1), evicting the least recently used one to make room.