Milo

Appearance

std/binary ​

Fixed-width integer and float codecs over a byte buffer — the encoding/binary / byteorder / Buffer.readUInt32LE layer.

milo
from "std/binary" import { Bytes }

A Milo string is an owned byte buffer, and it is what file-format code carries its bytes in, so reads take &string plus a byte offset and writes append to a &mut string.

Reads return Option ​

A read yields None when fewer than the needed bytes remain at the offset, or when the offset is negative. Nothing aborts, and nothing comes back as a plausible zero.

That is deliberate. src[i] is an index — a claim by you that the position is valid, and a bad claim is a bug worth trapping on. A decode at an offset that came out of the data being decoded is the opposite situation: the input is untrusted, and truncation is an ordinary thing for a file or a network frame to be. None is what lets a parser answer a truncated input with a parse error instead of dying, and — unlike a zero-filled read — it can't be mistaken for a value that was really there.

milo
// a parser turns the None into its own error, once, in a helper
fn le32(src: &string, p: i64): Result<i64, string> {
    let Some(v) = Bytes.readU32Le(src, p) else {
        return Err("truncated archive")
    }
    return Ok(v as i64)
}

Reading ​

Bytes.readU8 / readI8, and Le/Be pairs for every wider type (16, 32 and 64-bit integers, signed and unsigned, and both float widths).

The signed forms are the two's-complement reinterpretation of the same bytes, so readI16Be over ff ff is -1, not 65535.

milo
let hdr = readFile("frame.bin")?
let magic = Bytes.readU32Be(hdr, 0)
let temp = Bytes.readI16Le(hdr, 8)   // -40 stays -40

Bytes.has(src, off, n) answers the bounds question directly when you want to check once and then read several fields.

Writing ​

Writes append and cannot fail, so they return void:

milo
var out = ""
Bytes.writeU32Be(&mut out, body.len as u32)   // body.len is i64 — the cast is required
Bytes.writeI16Le(&mut out, -40)               // a literal adopts the parameter's type
Bytes.writeF64Le(&mut out, 1.5)

A literal needs no cast: it takes the width from the parameter it is passed to. A value of another width does, and that as u32 is where one too large for the field gets truncated — spelling it at the call site is what keeps the truncation visible.

Float bits ​

Bytes.f64ToBits / f64FromBits (and the f32 pair) expose the raw IEEE-754 bit pattern. as between a float and an integer converts the value; these convert the representation, which is what the readF*/writeF* pairs are built on.

milo
print(Bytes.f64ToBits(1.5))   // 4609434218613702656

Varints (LEB128) ​

Bytes.writeUvarint / writeVarint append an unsigned or signed (zigzag, protobuf sint64) LEB128 varint, 1 to 10 bytes. readUvarint / readVarint decode one at an offset and return Result<Varint<T>, VarintError>: the value plus next, the offset just past it, which is where the following field starts. Unlike the fixed-width reads they return a Result, because a varint fails in ways a caller handles differently: Truncated (the input ended first; a streaming reader waits for more), Overlong (a padded, non-canonical encoding such as 80 00 for 0) and Overflow (more than 64 bits).

milo
var msg = ""
Bytes.writeUvarint(&mut msg, 300)    // ac 02
Bytes.writeVarint(&mut msg, -3)      // 05
let a = Bytes.readUvarint(msg, 0)!
let b = Bytes.readVarint(msg, a.next)!
print($"{a.value} {b.value}")        // 300 -3

API reference ​

Bytes ​

milo
pub struct Bytes

Bytes — fixed-width codecs over a byte buffer.

Reads take a byte offset from the start of src and yield None when fewer than the needed bytes remain there, or when the offset is negative — a negative offset is out of bounds, not an index from the end. The signed forms are the two's-complement reinterpretation of the same bytes, so readI16Be over ff ff is -1, not 65535.

Writes append to out and never fail. Pass the value at its declared width (Bytes.writeU32Le(out, n as u32)) — that cast is where a value too large for the field gets truncated, and spelling it at the call site is what keeps the truncation visible.

Bytes.f32FromBits ​

milo
fn Bytes.f32FromBits(b: u32): f32

The f32 whose host bit pattern is b. Inverse of f32ToBits.

Bytes.f32ToBits ​

milo
fn Bytes.f32ToBits(v: f32): u32

The IEEE-754 bit pattern of v, in host representation.

Bytes.f64FromBits ​

milo
fn Bytes.f64FromBits(b: u64): f64

The f64 whose host bit pattern is b. Inverse of f64ToBits.

Bytes.f64ToBits ​

milo
fn Bytes.f64ToBits(v: f64): u64

The IEEE-754 bit pattern of v, in host representation.

as between a float and an integer converts the value; serializing IEEE-754 needs the bits. Milo has no bitcast operator, so this reads the value's own storage through a repointed pointer. That is in bounds by construction (f64 and u64 are both 8 bytes, f32 and u32 both 4), and the caller writes the result out in whichever byte order it asked for, so the encoding is right on a big-endian host too.

Bytes.has ​

milo
fn Bytes.has(src: &string, off: i64, n: i64): bool

Whether n bytes are readable at off. False for a negative off.

Written as src.len - off rather than off + n <= src.len because arithmetic is checked: an offset near i64 max would trap on the addition before the bounds test could reject it.

Bytes.readF32Be ​

milo
fn Bytes.readF32Be(src: &string, off: i64): Option<f32>

The big-endian IEEE-754 f32 at off, or None if 4 bytes do not remain.

Bytes.readF32Le ​

milo
fn Bytes.readF32Le(src: &string, off: i64): Option<f32>

The little-endian IEEE-754 f32 at off, or None if 4 bytes do not remain.

Bytes.readF64Be ​

milo
fn Bytes.readF64Be(src: &string, off: i64): Option<f64>

The big-endian IEEE-754 f64 at off, or None if 8 bytes do not remain.

Bytes.readF64Le ​

milo
fn Bytes.readF64Le(src: &string, off: i64): Option<f64>

The little-endian IEEE-754 f64 at off, or None if 8 bytes do not remain.

Bytes.readI16Be ​

milo
fn Bytes.readI16Be(src: &string, off: i64): Option<i16>

The big-endian i16 at off, or None if 2 bytes do not remain there.

Bytes.readI16Le ​

milo
fn Bytes.readI16Le(src: &string, off: i64): Option<i16>

The little-endian i16 at off, or None if 2 bytes do not remain there.

Bytes.readI32Be ​

milo
fn Bytes.readI32Be(src: &string, off: i64): Option<i32>

The big-endian i32 at off, or None if 4 bytes do not remain there.

Bytes.readI32Le ​

milo
fn Bytes.readI32Le(src: &string, off: i64): Option<i32>

The little-endian i32 at off, or None if 4 bytes do not remain there.

Bytes.readI64Be ​

milo
fn Bytes.readI64Be(src: &string, off: i64): Option<i64>

The big-endian i64 at off, or None if 8 bytes do not remain there.

Bytes.readI64Le ​

milo
fn Bytes.readI64Le(src: &string, off: i64): Option<i64>

The little-endian i64 at off, or None if 8 bytes do not remain there.

Bytes.readI8 ​

milo
fn Bytes.readI8(src: &string, off: i64): Option<i8>

The byte at off as a signed value, or None if off is not in src.

Bytes.readU16Be ​

milo
fn Bytes.readU16Be(src: &string, off: i64): Option<u16>

The big-endian u16 at off, or None if 2 bytes do not remain there.

Bytes.readU16Le ​

milo
fn Bytes.readU16Le(src: &string, off: i64): Option<u16>

The little-endian u16 at off, or None if 2 bytes do not remain there.

Bytes.readU32Be ​

milo
fn Bytes.readU32Be(src: &string, off: i64): Option<u32>

The big-endian u32 at off, or None if 4 bytes do not remain there.

Bytes.readU32Le ​

milo
fn Bytes.readU32Le(src: &string, off: i64): Option<u32>

The little-endian u32 at off, or None if 4 bytes do not remain there.

Bytes.readU64Be ​

milo
fn Bytes.readU64Be(src: &string, off: i64): Option<u64>

The big-endian u64 at off, or None if 8 bytes do not remain there.

Bytes.readU64Le ​

milo
fn Bytes.readU64Le(src: &string, off: i64): Option<u64>

The little-endian u64 at off, or None if 8 bytes do not remain there.

Bytes.readU8 ​

milo
fn Bytes.readU8(src: &string, off: i64): Option<u8>

The byte at off, or None if off is not in src.

Bytes.readUvarint ​

milo
fn Bytes.readUvarint(src: &string, off: i64): Result<Varint<u64>, VarintError>

Decode the unsigned varint that starts at off. On success the result holds the value and next, the offset just past its last byte.

Strict: Truncated when the input ends (or off is outside it) before a byte without the continuation bit, Overflow when the value needs more than 64 bits, and Overlong when the encoding is longer than the value needs (a final 0x00 group after the first byte, as in 80 00 for 0). Rejecting the padded forms gives each value exactly one encoding, which is what a decoder that hashes, compares or re-encodes its input relies on.

Bytes.readVarint ​

milo
fn Bytes.readVarint(src: &string, off: i64): Result<Varint<i64>, VarintError>

Decode the signed (zigzag) varint that starts at off, with the same errors and next as readUvarint.

Bytes.writeF32Be ​

milo
fn Bytes.writeF32Be(out: &mut string, v: f32): void

Append v as 4 big-endian IEEE-754 bytes.

Bytes.writeF32Le ​

milo
fn Bytes.writeF32Le(out: &mut string, v: f32): void

Append v as 4 little-endian IEEE-754 bytes.

Bytes.writeF64Be ​

milo
fn Bytes.writeF64Be(out: &mut string, v: f64): void

Append v as 8 big-endian IEEE-754 bytes.

Bytes.writeF64Le ​

milo
fn Bytes.writeF64Le(out: &mut string, v: f64): void

Append v as 8 little-endian IEEE-754 bytes.

Bytes.writeI16Be ​

milo
fn Bytes.writeI16Be(out: &mut string, v: i16): void

Append v as 2 big-endian bytes, two's complement.

Bytes.writeI16Le ​

milo
fn Bytes.writeI16Le(out: &mut string, v: i16): void

Append v as 2 little-endian bytes, two's complement.

Bytes.writeI32Be ​

milo
fn Bytes.writeI32Be(out: &mut string, v: i32): void

Append v as 4 big-endian bytes, two's complement.

Bytes.writeI32Le ​

milo
fn Bytes.writeI32Le(out: &mut string, v: i32): void

Append v as 4 little-endian bytes, two's complement.

Bytes.writeI64Be ​

milo
fn Bytes.writeI64Be(out: &mut string, v: i64): void

Append v as 8 big-endian bytes, two's complement.

Bytes.writeI64Le ​

milo
fn Bytes.writeI64Le(out: &mut string, v: i64): void

Append v as 8 little-endian bytes, two's complement.

Bytes.writeI8 ​

milo
fn Bytes.writeI8(out: &mut string, v: i8): void

Append v as one byte, two's complement.

Bytes.writeU16Be ​

milo
fn Bytes.writeU16Be(out: &mut string, v: u16): void

Append v as 2 big-endian bytes.

Bytes.writeU16Le ​

milo
fn Bytes.writeU16Le(out: &mut string, v: u16): void

Append v as 2 little-endian bytes.

Bytes.writeU32Be ​

milo
fn Bytes.writeU32Be(out: &mut string, v: u32): void

Append v as 4 big-endian bytes.

Bytes.writeU32Le ​

milo
fn Bytes.writeU32Le(out: &mut string, v: u32): void

Append v as 4 little-endian bytes.

Bytes.writeU64Be ​

milo
fn Bytes.writeU64Be(out: &mut string, v: u64): void

Append v as 8 big-endian bytes.

Bytes.writeU64Le ​

milo
fn Bytes.writeU64Le(out: &mut string, v: u64): void

Append v as 8 little-endian bytes.

Bytes.writeU8 ​

milo
fn Bytes.writeU8(out: &mut string, v: u8): void

Append v as one byte.

Bytes.writeUvarint ​

milo
fn Bytes.writeUvarint(out: &mut string, v: u64): void

Append v as an unsigned LEB128 varint: 7 bits per byte, low group first, the high bit set on every byte but the last. 1 to 10 bytes; always the shortest encoding, so readUvarint accepts everything this writes.

Bytes.writeVarint ​

milo
fn Bytes.writeVarint(out: &mut string, v: i64): void

Append v as a signed varint: zigzag-mapped (0, -1, 1, -2, ... to 0, 1, 2, 3, ...) so a small negative number stays short, then written as an unsigned varint. The protobuf sint64 encoding.

Bytes.zigzagDecode ​

milo
fn Bytes.zigzagDecode(u: u64): i64

The inverse of zigzagEncode.

Bytes.zigzagEncode ​

milo
fn Bytes.zigzagEncode(v: i64): u64

Zigzag: map a signed value onto an unsigned one so that magnitude, not sign, decides the length (0 -> 0, -1 -> 1, 1 -> 2, i64 min -> u64 max). Computed on u64 bits, because v << 1 on an i64 overflows for half the range.

Varint ​

milo
pub struct Varint<T>

A decoded varint and the offset just past it, so a caller can read the next field from next.

Fields: value: T, next: i64.

VarintError ​

milo
pub enum VarintError

Why a varint decode failed. Truncated is the one a streaming reader answers by waiting for more bytes; the other two mean the input is malformed.

Variants:

  • Truncated: the input ended before the varint's final byte, or off was outside it
  • Overlong: the encoding uses more bytes than the value needs (not the canonical form)
  • Overflow: the value does not fit in 64 bits