Milo

Appearance

std/replay ​

The record/replay engine behind MILO_RECORD and MILO_REPLAY. Run a program with MILO_RECORD=trace.mrr and every nondeterministic input it reads (clock, entropy, environment, files, sockets, subprocesses, the green scheduler's order, extern calls, OS thread ordering) is appended to the trace; run it again with MILO_REPLAY=trace.mrr and each of those inputs is answered from the trace instead, so the run repeats exactly. milo trace trace.mrr lists the records.

Programs do not call this module to get that: std's own IO, time, random and process functions are already hooked, and the compiler routes extern calls through generated recording wrappers. The functions here are the hook API those use, for a library that wraps a new source of input itself.

milo
from "std/replay" import { replayOff, replayRet }

Example ​

A hooked call: one compare when neither variable is set, recorded or answered from the trace otherwise.

milo
from "std/replay" import { replayOff, replayRet }

extern fn sensorRead(): i64

fn readSensor(): i64 {
    if replayOff() {
        unsafe { return sensorRead() }
    }
    return replayRet("sensor.read", "", false, (): i64 => {
        unsafe { return sensorRead() }
    })
}

See Record and replay for what is captured, the trace format, holes and divergence.

API reference ​

ReplayCall ​

milo
pub struct ReplayCall

One extern call through the trace. The payload is a header line <ret> <errno> and one length per output buffer (-1 for one the call did not get), then the outputs' bytes back to back.

Fields: kind: string, arg: string, ret: i64, err: i32, lens: Vec<i64>, data: string, dataPos: i64, outIdx: i64.

ReplayCall.argBytes ​

milo
fn ReplayCall.argBytes(self: &mut ReplayCall, p: *u8, n: i64): void

n bytes an argument points at, in hex.

ReplayCall.argCstr ​

milo
fn ReplayCall.argCstr(self: &mut ReplayCall, p: *u8): void

A C string argument (a path, a variable name); - for null.

ReplayCall.argInt ​

milo
fn ReplayCall.argInt(self: &mut ReplayCall, v: i64): void

A scalar argument, part of what names the call.

ReplayCall.nextLen ​

milo
fn ReplayCall.nextLen(self: &mut ReplayCall): i64

Replay: the recorded length of the next output buffer (-1 when the call did not write it); a trace with fewer lengths than outputs is corrupt.

ReplayCall.outCopy ​

milo
fn ReplayCall.outCopy(self: &mut ReplayCall): *u8

Replay: the next recorded output as a fresh heap copy (a returned string or structure), null when the recording returned null.

ReplayCall.outCstr ​

milo
fn ReplayCall.outCstr(self: &mut ReplayCall, p: *u8, cap: i64): void

Record: the NUL-terminated string at p (within cap bytes when cap >= 0), terminator included.

ReplayCall.outFrom ​

milo
fn ReplayCall.outFrom(self: &mut ReplayCall, p: *u8, n: i64): void

Record: n bytes at p as the next output (absent when p is null or n < 0).

ReplayCall.outInto ​

milo
fn ReplayCall.outInto(self: &mut ReplayCall, p: *u8): void

Replay: copy the next recorded output into p (skipped when the recording had none, or p is null).

ReplayCall.put ​

milo
fn ReplayCall.put(self: &mut ReplayCall, net: bool): void

Record: write the record, then put errno back (the write is a syscall too).

ReplayCall.replaying ​

milo
fn ReplayCall.replaying(self: &ReplayCall): bool

True when the call is to be answered from the trace.

ReplayCall.retVal ​

milo
fn ReplayCall.retVal(self: &ReplayCall): i64

Replay: the recorded return value (a float's bits, a handle's address).

ReplayCall.saveErrno ​

milo
fn ReplayCall.saveErrno(self: &mut ReplayCall, net: bool): void

Record: errno right after the live call, before anything else can touch it.

ReplayCall.sep ​

milo
fn ReplayCall.sep(self: &mut ReplayCall): void

Separate the next argument from the previous one in the call's name.

ReplayCall.setRet ​

milo
fn ReplayCall.setRet(self: &mut ReplayCall, r: i64): void

Record: the call's return value (a float's bits, a handle's address).

ReplayCall.take ​

milo
fn ReplayCall.take(self: &mut ReplayCall, net: bool): void

Replay: fetch this call's record and restore its errno.

ReplayLock ​

milo
pub struct ReplayLock

The one lock around the trace. A spin lock with a sleep, not a pthread mutex: it has to work before any initializer has run (a hook can fire from another module's global initializer), and the critical sections are a record's worth of work. Reentrant per thread, so a divergence reported while it is held can still run the exit hook. @synchronized tells the checker's thread pass that the shared state above, written only inside run, is not racing.

ReplayLock.run ​

milo
fn ReplayLock.run(self: &ReplayLock, f: () => void): void

Run f holding the record/replay engine's lock; re-entrant on the same thread.

Functions ​

miloReplayStart ​

milo
pub fn miloReplayStart(argv: *u8): void

Called first thing in main, before any global initializer: decides record/replay (so an initializer's allocations already come from the fixed heap) and restarts with ASLR off when needed, which must happen before the trace is opened.

Only the restart and the heap happen here: the rest of the decision waits for std/replay's own initializer, because its state (the trace window, the fd tables) is set up by the global initializers that run after this.

miloReplayStop ​

milo
pub fn miloReplayStop(seq: i64): void

The function a debugger breaks on to stop a replay at a chosen record: with MILO_REPLAY_STOP=<N>, replay calls it right after consuming record N, so the replaying program is stopped with record N's call just answered. External linkage keeps the symbol (b miloReplayStop); it then raises SIGTRAP, so a debugger attached without a breakpoint stops here too. With MILO_REPLAY_STOP_PRINT=1 it prints instead and the replay continues (for tests and a UI that only wants the position).

replayActive ​

milo
pub fn replayActive(): bool

True when hooked calls must return recorded answers instead of asking the OS.

replayBlob ​

milo
pub fn replayBlob(kind: &string, arg: &string, live: () => string): string

Run a call that answers with arbitrary bytes the caller encodes itself (a stat result, a directory listing, a resolved address).

replayCallBegin ​

milo
pub fn replayCallBegin(kind: string): ReplayCall

Start one recorded extern call of kind kind (x.<name>).

replayClose ​

milo
pub fn replayClose(fd: i32): i32

close(2), unrecorded: when a program closes is deterministic given its inputs. A synthetic fd is only forgotten; the OS never had it.

replayF64Bits ​

milo
pub fn replayF64Bits(v: f64): i64

The bits of a float, for recording a float argument or return.

replayF64FromBits ​

milo
pub fn replayF64FromBits(b: i64): f64

The float with these bits: the inverse of replayF64Bits, for a replayed float.

replayFdCategory ​

milo
pub fn replayFdCategory(fd: i32): u8

What std recorded fd as (see replayFdOpened), 0 when nothing.

replayFdOpened ​

milo
pub fn replayFdOpened(fd: i32, cat: u8): void

Note an fd std just opened (or that replay just handed out) and what it is: 'f' a file, 'n' a socket, 'p' a child's pipe, 't' a pty. Under replay the number came from the trace and has no OS descriptor behind it.

replayHeapDepthIn ​

milo
pub fn replayHeapDepthIn(d: i64): void

Restore the engine depth replayHeapDepthOut took, after a green-task switch back.

replayHeapDepthOut ​

milo
pub fn replayHeapDepthOut(): i64

Green tasks switch inside a thread, so the engine depth belongs to a context, not to the thread: std/runtime takes it out before a swapcontext and puts it back after.

replayHeapFree ​

milo
pub fn replayHeapFree(p: *u8): void

free() for Milo allocations: blocks from the fixed-address heap go back to it, any other pointer (libc's, the engine's own) goes to libc.

replayHeapMalloc ​

milo
pub fn replayHeapMalloc(n: i64): *u8

malloc, free and realloc for every allocation Milo code makes: codegen calls these in place of libc's when std/replay is in the program (src/codegen.ts). With neither variable set they are one compare in front of libc.

replayHeapMap ​

milo
pub fn replayHeapMap(len: i64): *u8

Page-granular memory (a green task's stack) from the fixed heap, or null when the heap is off or this is the engine's own: the caller then maps it itself.

replayHeapOff ​

milo
pub fn replayHeapOff(): bool

True when no fixed heap is running, so replayHeapMap would hand out nothing and a green task's stack may come from anywhere (std/runtime's stack cache). Named "Heap" so a caller is not counted as engine code (src/lower.ts isReplayEngine), which would raise rrDepth around it and send its allocations to libc.

replayHeapRealloc ​

milo
pub fn replayHeapRealloc(p: *u8, n: i64): *u8

realloc() for Milo allocations: a block from the fixed-address heap grows within it, so its new address replays too; any other pointer goes to libc.

replayHeapUnmap ​

milo
pub fn replayHeapUnmap(p: *u8, len: i64): bool

Give back a block replayHeapMap handed out. False when p is not one (the caller unmaps it itself).

replayHex ​

milo
pub fn replayHex(p: *u8, n: i64): string

Lowercase hex of n raw bytes (socket addresses in a record's arg).

replayHole ​

milo
pub fn replayHole(name: &string, site: *u8): void

Report, once per name, that this run called an extern the trace cannot answer, and note it in the trace (trace.hole: arg the name, payload where), so milo trace shows a recording with gaps. Under replay the note is consumed silently; the call itself runs for real in both.

replayHoleAt ​

milo
pub fn replayHoleAt(name: *u8, site: *u8): void

replayHole with C strings, for a report the compiler inserts (a thread closure that shares unsafe memory, see src/checker-program-passes.ts: checkThreadBoundary).

replayInts ​

milo
pub fn replayInts(p: &string): Vec<i64>

The space-separated decimals of a payload (several numbers in one answer).

replayJoinNul ​

milo
pub fn replayJoinNul(items: &Vec<string>): string

Strings joined with a NUL after each: the encoding for argv and the environment, whose entries are C strings and so can never contain one.

replayLseek ​

milo
pub fn replayLseek(fd: i32, off: i64, whence: i32): i64

lseek(2) through the trace: a file's size and position are answers from the OS.

replayOff ​

milo
pub fn replayOff(): bool

True when neither recording nor replaying. The single check every hook makes before touching the OS; a not-yet-decided mode reads as "not off" so the slow path gets to decide it.

replayOpen ​

milo
pub fn replayOpen(path: &string, flags: i32, mode: i32): i32

open(2) through the trace. Under replay nothing is opened (a write-mode open does not create or truncate the file) and the recorded fd comes back as a synthetic one.

replayPut ​

milo
pub fn replayPut(kind: &string, arg: &string, payload: &string): void

Append one record: <seq> <kind> <argLen> <payloadLen>\n<arg>\n<payload>\n. arg names the call (an env var name, a byte count) so replay can tell a different call of the same kind from the recorded one; payload is the answer. No-op unless recording.

replayPutFrom ​

milo
pub fn replayPutFrom(kind: &string, buf: *u8, n: i64): void

Record: the n bytes the OS just wrote into buf.

replayPutI64 ​

milo
pub fn replayPutI64(kind: &string, arg: &string, v: i64): void

replayPut with a decimal payload.

replayRead ​

milo
pub fn replayRead(fd: i32, buf: *u8, len: i64, socket: bool): i64

One read(2) or recv through the trace. The payload is = and the bytes, or ! and the errno (EAGAIN included: a green read that would block is recorded too, and the park that follows is what the scheduler records replay).

replayRecording ​

milo
pub fn replayRecording(): bool

True when hooked calls append their answers to a trace.

replayRet ​

milo
pub fn replayRet(kind: &string, arg: &string, net: bool, live: () => i64): i64

Run a call that answers with a number (an fd, a status, an offset; negative with errno set on failure) through the trace: recorded with its errno, or answered from the trace with errno restored. net picks the socket error channel (WSAGetLastError on Windows). Callers check replayOff() first, so the closure costs nothing when off.

replaySchedPut ​

milo
pub fn replaySchedPut(payload: &string): void

Recording: one green-scheduler poll's outcome (std/runtime: pollAndWake). An empty payload is a poll that woke nothing and only bumps a counter; anything else is a sched.pick record naming the fds that came ready and the timer arms that fired.

replaySchedTake ​

milo
pub fn replaySchedTake(): string

Replay: what the poll at this point woke when recorded ("" for nothing). A poll past the end of the trace is idle: the recorded run's trailing idle polls were never written.

replaySplitNul ​

milo
pub fn replaySplitNul(payload: &string): Vec<string>

The inverse of replayJoinNul.

replaySyncAtomic ​

milo
pub fn replaySyncAtomic(op: &string, f: () => i64): i64

One ordered atomic operation (std/sync's Atomic* types): f performs it and returns its result. Recording runs it inside the trace lock, so the record order is the order the operations really happened in; replay runs it on this thread's turn and checks it returned what it returned when recorded.

replaySyncEnter ​

milo
pub fn replaySyncEnter(): void

Replay: wait until this thread's next record is at the head of the trace. std/sync calls it before taking a primitive's own lock, so the operation runs exactly when it ran in the recording, after every operation recorded before it. No-op when recording.

replaySyncNote ​

milo
pub fn replaySyncNote(op: &string, outcome: i64): void

One ordered sync operation, made while the primitive's lock is held: what it did (op, e.g. "recv") and how it came out (outcome: ready or not, the value an atomic returned). Recorded as sync.<op> with both in the arg; under replay the same operation must come out the same way, or the run has diverged.

replaySyncOn ​

milo
pub fn replaySyncOn(): bool

True when cross-thread sync operations are ordered: recording or replaying a run that has started an OS thread. Before the first thread exists every sync operation happens on the main thread, whose order the scheduler records already.

replayTake ​

milo
pub fn replayTake(kind: &string, arg: &string): string

The recorded answer to this call. Exits with "replay diverged at record N: expected <recorded call>, got <this call>" when the next record of this thread is a different call, or the trace has run out. With several threads, waits until this thread's record reaches the head of the trace.

replayTakeI64 ​

milo
pub fn replayTakeI64(kind: &string, arg: &string): i64

replayTake of a decimal payload.

replayTakeInto ​

milo
pub fn replayTakeInto(kind: &string, buf: *u8, n: i64): void

Replay: copy the recorded n bytes into buf. The byte count is the call's arg, so a request for a different count diverges instead of reading past the record.

replayThreadExit ​

milo
pub fn replayThreadExit(): void

Last thing on a thread std started. Under replay a thread waiting for a finished thread's record is then a divergence, not a hang.

replayThreadSpawn ​

milo
pub fn replayThreadSpawn(): i64

Called where std starts an OS thread, before it starts: numbers the new thread and records that it was started (thread.spawn, payload the number), so the numbering replays. From here on every sync operation is ordered. Returns 0 when off.

replayThreadStart ​

milo
pub fn replayThreadStart(tid: i64): void

First thing on a thread std started: take the number its parent recorded.

replayWrite ​

milo
pub fn replayWrite(fd: i32, buf: *u8, len: i64, socket: bool): i64

One write(2) or send through the trace: what is recorded is that it happened, to which fd, how many bytes were offered and how many went (or the errno). Under replay the write reaches the OS only for the inherited stdout and stderr, so the user sees the program's output; a write to a file, socket, pipe or pty is answered from the trace and never performed (the peer does not exist, and a replay must not change the filesystem).