modem

XMODEM, YMODEM, ZMODEM and Kermit in Zig 0.17, built to sit between two std.Io streams as a filter.

Bytes are passed through unchanged until a ZMODEM frame header or a Kermit S packet appears in the remote stream, at which point the filter stops forwarding and receives the file. Sending works the other way: offer a file and the filter starts a transfer at the next opportunity.

lrzsz and C-Kermit are the references. Every ambiguity in Chuck Forsberg's and Frank da Cruz's specifications was settled by capturing what those programs actually do, and zig build oracle checks both directions of all four protocols against them.

The API documentation is at https://jeff.jcollie.page/zig-modem/. It is generated from the doc comments, which is where most of the explanation of what each protocol does and why it does it that way lives.

The four protocols

They do the same job and agree on almost nothing about how.

	XMODEM	YMODEM	ZMODEM	Kermit
Files per session	one	many	many	many
Carries a file name	no	yes	yes	yes
Carries a length	no	yes	yes	yes
Receiver can decline	no	no	yes	yes
Resume part way	no	no	yes	no
Sender announces	no	no	yes	yes
Block check	sum, CRC16	CRC16	CRC16, CRC32	3 kinds
Sends 8-bit data raw	yes	yes	escaped	quoted
Packets in flight	1, or all	1, or all	all	1 to 31

Each has its own Receiver and Sender with the options its wire format actually needs, under modem.xmodem, modem.ymodem, modem.zmodem and modem.kermit. AnyReceiver and AnySender sit over the top for callers that pick a protocol at run time.

That run-time list has two more names in it, xmodem_1k and ymodem_1k, and each is the protocol above it sending 1024 byte blocks instead of 128. They are separate choices rather than an option because whether the far end takes a 1024 byte block is the one thing here that cannot be negotiated: the sender picks a size, the marker byte says which it picked, and a receiver too old to know the larger one refuses every block without ever saying why. So xmodem and ymodem never send one, and the _1k pair are how a caller says it knows better. A receiver takes whichever sizes it is sent either way, so the distinction is the sender's alone -- which is exactly how terminal programs have offered them for decades.

What "the sender announces" costs is worth spelling out, because it decides what a filter can do. An XMODEM or YMODEM sender says nothing at all until a receiver prompts it, so an incoming transfer cannot be recognised: somebody has to decide to go and fetch. ZMODEM and Kermit senders speak first, so those transfers start by themselves.

Using the filter

const modem = @import("modem");

// The ring the remote stream is carried on, and the buffer its reader
// works out of. Both sizes are recommendations rather than requirements;
// see below.
var storage: [modem.Pipe.storage_size]u8 = undefined;
var pipe_buffer: [modem.Pipe.reader_buffer_size]u8 = undefined;
var pipe: modem.Pipe = .init(io, &storage, &pipe_buffer);

// Scratch for whichever protocol turns up. A wider Kermit window wants
// more; see `modem.receiveBufferSize`.
var receive_buffer: [modem.receive_buffer_size]u8 = undefined;
var send_buffer: [modem.send_buffer_size]u8 = undefined;

var filter: modem.Filter = .init(
    io,
    .{ .in = &remote_in, .out = &remote_out },
    .{ .in = &local_in, .out = &local_out },
    my_sink,
    .{},
    &pipe,
    &receive_buffer,
    &send_buffer,
);

try filter.run(); // forwards both directions until `filter.stop()`

Of those four sizes only one is a real constraint. Pipe.storage_size and Pipe.reader_buffer_size are round numbers with headroom: measured over a pair of in-memory pipes, every protocol here runs within noise of the same speed with 256 bytes of ring as with a quarter of a megabyte, and the same is true of the reader buffer from 128 bytes upwards. What the ring buys is room for the peer to go on talking while this end is busy between frames or waiting on a sink, rather than being throttled until it catches up.

The exception is Pipe.min_reader_buffer_size, which is a floor and not a suggestion. std.Io.Reader.peek asserts rather than fails, so a buffer too small to hold what is peeked is a panic -- and the largest peek is the filter's, which validates a whole Kermit S packet before it will believe a transfer is starting. Filter.init asserts it, so the mistake is caught where it is made rather than the first time a Kermit sender goes past.

Received files go wherever the caller's Sink puts them. The library never touches the filesystem:

fn open(context: *anyopaque, info: modem.file.Info, offset: *u64)
    modem.Sink.OpenError!?*std.Io.Writer
{
    const self: *MySink = @ptrCast(@alignCast(context));
    // `info.name` comes from the peer. `baseName` strips any directory
    // prefix, so a sender cannot ask for `../../etc/passwd`. It is empty
    // for XMODEM, which carries no name at all.
    const name = info.baseName();
    if (name.len == 0) return null; // returning null declines the file

    self.file = try self.dir.createFile(self.io, name, .{});
    self.writer = self.file.writer(self.io, &self.buffer);
    return &self.writer.interface;
}

To resume an interrupted transfer, set offset.* to the number of bytes already held; the sender is asked to restart there. Only ZMODEM can do this.

Saying no

There are three ways to refuse, and which one to reach for depends on how much of the answer is already known.

One file. Return null from Sink.open. It is called before any of the file's data arrives and is handed the name, the length, the timestamp and how much of the batch is left, which is everything there is to decide on. The session carries on to the next file: ZMODEM sends ZSKIP, Kermit refuses the file's attributes. Kermit can only say it through the attribute packet, though, so a sender that offered none leaves nothing to refuse and the file is received and discarded instead. XMODEM carries one file and has no word for skipping, so declining cancels the transfer.

The session, once it has started. Return error.Refused from Sink.open. The session ends and the peer is told to stop -- ZMODEM's abort sequence, a Kermit error packet -- and the caller gets error.Refused back from run rather than a fault. error.SinkFailed does the same thing on the wire and means something different to the caller: a sink that could not go on rather than one that would not.

The whole transfer, before it starts. Only a filter can do this, because only a filter is asked. Filter.Options.accept is consulted the moment an incoming ZMODEM or Kermit transfer is recognised, before a word of it is answered:

fn wanted(context: *anyopaque, protocol: modem.Protocol) bool {
    const self: *MyUi = @ptrCast(@alignCast(context));
    return self.askTheUser(protocol);
}

var f: modem.Filter = .init(io, remote, local, my_sink, .{
    .accept = .{ .context = &my_ui, .wanted = wanted },
}, &pipe, &receive_buffer, &send_buffer);

Returning false tells the peer to abandon the transfer and goes straight back to forwarding. It is asked only about transfers the filter noticed by itself; one asked for through request was already a decision. Nothing remembers a refusal, so a sender that announces itself again is asked again -- which costs one exchange per announcement, and lets a caller change its mind.

Telling the peer is the part that matters, and it is worth being plain about why. A filter goes back to forwarding the instant a session returns. A sender that was not told would go on streaming, and the rest of the file would land on the user's terminal as escaped noise until the sender's own timeout ran out. zig build oracle checks it against the real thing: sz, refused, gives up and exits with an error rather than sending a file nobody was going to keep.

Sending is queued from any task and awaited separately, so a user interface can offer a file while the filter is busy forwarding:

var upload: modem.Filter.Offer = .{
    .info = info,
    .source = source,
    .protocol = .ymodem,
};
try filter.offer(&upload);
switch (try filter.awaitOffer(&upload)) {
    .sent => {},
    .skipped => {},                 // the peer declined it
    .failed => return upload.err.?,
    .pending, .sending => unreachable,
}

Consecutive offers naming the same protocol are sent as one session.

An XMODEM or YMODEM download has to be asked for, since neither sender will speak first:

var download: modem.Filter.Request = .{
    .protocol = .xmodem,
    // XMODEM carries no name, so the receiving end picks one.
    .info = .{ .name = "download.bin" },
};
try filter.request(&download);
_ = try filter.awaitRequest(&download);

Either half of any protocol can also be driven on its own, without the filter, against any pair of std.Io streams: see each protocol's Receiver and Sender.

Progress

Every receiver and sender takes an optional Progress in its options, and tells it when a file starts, as the file moves, and when it ends. It is opt in: a transfer with no Progress does exactly what it did before.

var display: modem.Progress = .{
    .context = &my_display,
    .report = draw,
    // The shortest gap between two `.data` reports; the start and the end
    // are always reported. A ZMODEM sender on a fast link can put a
    // thousand subpackets a second on the wire, and redrawing that often
    // would cost more than the transfer.
    .interval = .fromMilliseconds(250),
};

const options: modem.zmodem.Receiver.Options = .{ .progress = &display };

fn draw(context: *anyopaque, r: modem.Report) void {
    const self: *MyDisplay = @ptrCast(@alignCast(context));
    switch (r.event) {
        .start => self.begin(r.info.name),
        .data => self.update(r),
        .end => |outcome| self.finish(outcome),
    }
}

A report carries what moved and how fast:

bytes	of this file, from info.size when the protocol carried one
session_bytes	of the whole session, every file in a batch included
elapsed	how long the session has been moving bytes
rates.one_minute	bytes per second over the last minute
rates.five_minutes	...over the last five
rates.fifteen_minutes	...over the last quarter of an hour
percent()	0 to 100, or null when nothing carried a length
remaining(), eta()	the same, null for the same reason
rate()	the whole-session average, always available

The three windows are the ones a load average uses, and for the same reason: a rate over one packet says how that packet went, and a rate over the whole transfer stops meaning anything once the line quality has changed. Unlike a load average they are measured rather than exponentially smoothed, so each is null until the session has actually been running that long -- for the first minute of every transfer, and for the whole of most of them, rate() is what there is. currentRate() picks whichever is available.

Percentages need a length, and only three of the four protocols carry one. Plain XMODEM says nothing about the file at all, so a receiver gets byte counts and rates and no percentage, unless the caller was told the length by some other route and passed it in through Start.info.

What "moved" means differs slightly by direction, because each protocol has a different natural moment to count at. An XMODEM or YMODEM sender counts a block once the receiver has acknowledged it, so a block sent three times over a noisy line counts once. A ZMODEM sender counts a subpacket as it goes out, and follows the position back when a ZRPOS rewinds it; only forward movement feeds the rates. A Kermit sender counts a packet as it enters the window, which runs up to a window ahead of the receiver on a link with any delay in it. A receiver, in every case, counts what reached the sink.

The callback runs on the task driving the transfer, in the middle of the protocol, so it should return promptly and must not touch the streams the transfer is using. One Progress covers one session: the rates span every file in a batch and the gaps between them, which is what makes a fifteen minute window mean anything for a protocol sending fifty small files. Two sessions running at once -- an upload and a download through the same filter -- want one each.

Command line

The modem binary exists mostly so the library can be tested against lrzsz and C-Kermit, but it is usable on its own:

modem recv [-p PROTO] [-d DIR] [-n NAME]   receive over stdin/stdout
modem send [-p PROTO] FILE...              send over stdin/stdout
modem filter [options]                     pass stdin/stdout through

-p takes zmodem (the default), xmodem, xmodem-1k, ymodem, ymodem-1k or kermit. -w sets how many Kermit packets may be in flight at once. -n names a file received over XMODEM, which carries no name of its own; the filter takes --send FILE to offer a file, --receive to fetch one, and --refuse to turn away every transfer it notices rather than receiving it.

All three commands draw a progress line on standard error, redrawn in place, which is the whole of what the library reports rather than a tidy subset -- a rate that only appears after five minutes of transfer is exactly the sort of thing nothing else would notice was broken:

modem: receiving disk.img 41.2MiB/220.0MiB 19%  1.1MiB/s  1m 1.2MiB/s  eta 2m41s

Watching the line

A transfer that fails says almost nothing about why. TooManyErrors means the far end refused a block eleven times; it does not say whether it sent a NAK, sent nothing at all, or sent something that was never a reply. tools/probe.zig is what answers that: a small serial terminal that writes down every byte in both directions, with timestamps and a note about what each chunk appears to be.

$ zig build probe -- /dev/ttyUSB0 -b 9600 -f config.txt -l trace.txt
probe: /dev/ttyUSB0 at 9600, tracing to trace.txt
probe: [C-]] s sends, [C-]] r receives, [C-]] q quits, [C-]] ? lists the keys

Your terminal is in front of the port, so you type whatever sets the far end up -- copy ymodem: test.txt on a Cisco switch, say -- and press [C-]] s when it starts prompting. [C-]] r goes the other way, asking for a download, which is the only way to fetch over XMODEM or YMODEM: neither sender says a word until a receiver prompts it. The command key is C-], as telnet uses, and deliberately not the C-a a terminal program would take: the far end usually wants C-a for beginning-of-line. --escape moves it, and any key that names no command is passed on to the far end rather than eaten. The trace is the point:

    0.000      open /dev/ttyUSB0 at 9600 8N1
    0.986  <-  53 77 69 74 63 68 23 0d ...   "Switch#\r\n"
    3.204  <-  43 43 43                      3 x 'C' (prompt, CRC)
    4.010      sending config.txt (4224 bytes) as YMODEM
    4.011  ->  01 00 ff 63 6f 6e 66 69 ...   SOH block 0, 128 data
    4.140  <-  06 43                         ACK, 'C' (prompt, CRC)

It takes an advisory flock on the port and refuses to start if another program holds one, which picocom, minicom and zettacom all take as well. This is not tidiness: two programs reading one serial port each get about half the bytes, so a session shared with a terminal left open in another window shows text with every other character missing and a transfer that fails for no visible reason. --no-lock overrides it.

--hex writes every byte rather than the first eight of each chunk, and --listen is the passive form: it opens the port, prints and logs what arrives, and sends nothing whatsoever -- not a keystroke, not a protocol reply, not an abort. That answers "is the far end prompting at all?" without putting anything on the line that could change the answer.

It is a diagnostic and not a terminal program: no configuration, no dialling, no flow control beyond the driver's, and Linux only, since it reaches for termios directly. What it has that a terminal program does not is the log.

The log is the record even when the screen is not. Everything crossing the port is copied into it by a Reader and a Writer that sit on the wire, so it shows what actually went past rather than what any layer above believed it wrote -- including whatever arrived while a protocol engine held the stream.

Keeping the line busy

Only ZMODEM streams unconditionally: its sender talks until the receiver interrupts, so a round trip costs nothing. The other three wait to be acknowledged, and on a link with any real delay that leaves the line idle most of the time.

XMODEM and YMODEM have one answer between them, YMODEM-G, which drops acknowledgements entirely and gives up on the first error. Kermit has a better one: a window of up to 31 packets, with the receiver holding back anything that arrives early and the sender resending only what actually went missing. A single lost packet then costs one retransmission rather than a window's worth.

var receive_buffer: [modem.receiveBufferSize(16)]u8 = undefined;
var send_buffer: [modem.sendBufferSize(16)]u8 = undefined;

var receiver: modem.kermit.Receiver = .init(
    io, &pipe, &out, my_sink, .{ .window = 16 }, &receive_buffer,
);

The window costs a packet's worth of memory per slot at each end, so it is the buffer the caller hands over that decides how wide it can be: asking for more slots than the buffer holds narrows the window rather than failing, and so does a peer that will not go as wide. modem.receiveBufferSize and modem.sendBufferSize work out what a given window needs.

Why a Pipe sits in the middle

All four protocols are built on timeouts: a ZMODEM receiver that hears nothing for ten seconds prods the sender with a ZRPOS rather than waiting forever, an XMODEM receiver re-prompts every three, a Kermit receiver re-sends its NAK. std.Io.Reader has no notion of a deadline, and cancelling a read already in flight risks losing bytes the transport has handed over.

So a feeder task owns the blocking read and pushes whatever it gets into a ring buffer. The protocol engine consumes from the ring with a deadline it can change between frames, and giving up on a read costs nothing because the bytes stay buffered for whoever asks next. Pipe is that ring; it is the reason Filter.run needs real concurrency rather than Io.async.

What is implemented

All four. A sender's chatter option says where bytes that are not part of the protocol go. An XMODEM or YMODEM sender waits to be prompted, and on a console line what arrives while it waits is a shell prompt, an echo, or the far end explaining why it is not going to prompt; a filter passes that to the local screen rather than dropping it, which is the difference between a terminal that goes dead for a minute and one that says what happened.

XMODEM. Both block checks, 128 and 1024 byte blocks, the C/NAK handshake with a fallback from CRC to checksum, duplicate block detection and retransmission, and the two-CAN abort. Only the first prompt of a session chooses the check: once a block has been acknowledged the two ends have agreed, and a prompt arriving later means no more than "I am waiting". That matters for YMODEM, where the receiver prompts a second time to start the file and some of them say NAK where the specification says C. A sender chosen as xmodem sends 128 byte blocks and nothing else, which is what sx does unless asked for -k; xmodem_1k is the same protocol sending 1024 byte ones, and it drops back to 128 when the receiver refuses them: the 1K extension came years after the protocol and plenty of receivers -- a Cisco switch taking a file over its console, for one -- were never taught it. Optionally strips the trailing SUB padding a sender adds to the final block, which recovers the original length of a text file and corrupts a binary one that genuinely ends in SUB.

YMODEM. Everything XMODEM has, plus block zero carrying the name, length, timestamp and mode; batches; exact truncation to the stated length; and YMODEM-G, which streams without acknowledgements and abandons the transfer on the first error, because that is all it can do. ymodem and ymodem_1k divide by block size the same way XMODEM's two do. The empty block zero that ends a batch can be left off with end_batch, for a receiver that asks for another file, is given the marker that says there are none, and aborts. eot_delay waits before announcing the end of a file, for a receiver still writing the last of it to slow storage when the EOT arrives.

ZMODEM. Binary and hex headers, 16- and 32-bit frame checks, ZDLE escaping including the control-character and carriage-return-after-@ rules, streaming with ZCRCG, windowed flow control with ZCRCW/ZACK for receivers that advertise a bounded buffer, ZRPOS error recovery, batch transfers, ZSKIP, resume at an offset, and the five-CAN abort.

Kermit. The S/F/A/D/Z/B/E/N/Y packets, all three block checks, control quoting, the 8th-bit prefix, run-length encoding with the repeat prefix, the full parameter exchange, extended packet lengths up to four kilobytes, attribute packets carrying the length and timestamp, and sliding windows of up to 31 packets.

What is not

ESC8, the ZCOMPRESS transport options, and ZCOMMAND — the last deliberately, since honouring it would hand the peer arbitrary code execution. It is always answered with ZNAK.

Kermit's locking shifts, RESEND and server mode.

Notes on other implementations

lrzsz and C-Kermit are checked on every push, so what is written here is what has been found in the field and could not be checked automatically.

Cisco IOS 15.2(7)E4, copy ymodem: over the console (a Catalyst 2960-CX). Three things, of which the library now handles two.

It acknowledges block zero and then prompts to start the file with NAK rather than the C the specification asks for. A sender that reads that the way it reads an opening NAK -- as a request for the 8-bit checksum -- puts a one byte check on every block from then on, where this receiver is counting two, so it refuses every one and says nothing about why. Only the first prompt of a session settles the check here; see Transmit.awaitRequest.

It asks for another file after acknowledging the EOT, and answers the empty block zero that says there is none with the five-CAN abort. end_batch ends the session at the EOT instead. Turning it off costs forty seconds against this receiver, which spends them asking for the file that is not coming, so it is not the setting to reach for here.

And it writes 63 bytes fewer than block zero declares. Not a rounding error and not a race: exactly 63 on a 1000 byte file and exactly 63 on a 7600 byte one, unchanged by a two second pause before the EOT, and the bytes it does write are a byte-exact prefix of the original. Rebuilding the wire from a --hex probe trace shows every byte of the file crossing correctly in blocks this receiver acknowledged one by one, so there is nothing a sender can do about it.

XMODEM is the way round it, and the switch offers it in the same breath -- "Begin the Xmodem or Xmodem-1K transfer now...". XMODEM carries no length, so nothing reads block zero and nothing loses 63 bytes; what lands is the file padded out with SUB to a whole number of blocks, which is a head -c away from the original:

$ zig build probe -- /dev/ttyUSB0 -b 9600 -f config.txt -p xmodem-1k
Switch#copy xmodem: config.txt

Kermit's timestamps are written and read as UTC, while C-Kermit writes local time, so a timestamp exchanged with it is offset by the local zone. The value is handed to the caller's sink and nothing here acts on it; the alternative was carrying a timezone database for a field that is only ever displayed.

ZMODEM and YMODEM carry file positions in 32 bits, so files of 4 GiB and beyond cannot be described. That is a limit of the protocols.

Building

zig build              # the CLI
zig build test         # unit tests, in-process sessions, and the fuzz corpora
zig build oracle       # interoperability (cases skip if their tool is absent)
zig build test --fuzz  # search for inputs that break a fuzz target
zig build bench        # measure what each protocol carries
zig build docs         # the API documentation, into zig-out/docs
zig build docs-serve   # and read it, on http://127.0.0.1:8000
zig build probe -- ... # a serial line, traced; see below

docs-serve exists because the documentation has to be served rather than opened: the viewer Zig emits fetches sources.tar and main.wasm at runtime, and a browser refuses either from a file:// page. -Ddocs-port moves it off 8000. What the workflow publishes from main is the same bundle.

zig build test -Dtest-filter=kermit narrows a run to the tests whose name contains the filter, which is also how one fuzz target is driven on its own.

A flake.nix provides Zig 0.17, lrzsz, C-Kermit and a python for the oracle script, and a Forgejo workflow runs all of it -- REUSE lint, zig fmt --check, the tests, a build, and the interoperability cases -- on every push. A push to main that passes all of that also publishes the documentation.

v0.1.0 is the last release built with Zig 0.16, and the zig-0.16 branch carries it for anyone who cannot move yet.

Speed

zig build bench runs a sender and a receiver against each other over a pair of in-memory pipes and reports what each protocol carries; zig build bench -- --micro measures the parts a transfer is made of, so that a number that looks wrong can be accounted for rather than guessed at; and zig build bench -- --latency measures what the filter costs when it is not transferring anything at all, which is what an interactive session feels. The first two take --mib and --iterations, the third takes --samples, and all three take a name to run one case on its own.

Over in-memory pipes on one machine, with 8 MiB of text, a transfer runs at roughly a gigabyte a second for ZMODEM, rather more for YMODEM-G, and about half that for Kermit, whose encoding does more work per byte. Random bytes cost the escaping rules more, so they run at about two thirds of that.

The ymodem and ymodem 1k cases are worth reading together, because they are the same protocol and differ by a factor of three: a block costs a round trip whatever it carries, so eight times the payload per block is most of that. That is the whole argument for ymodem_1k, and the transfer that never finishes because the far end refuses the larger block is the whole argument against reaching for it by default.

The numbers describe this library and nothing else: a serial line runs at a ten-thousandth of these rates, and even a fast network transport gives a transfer far less to do than an in-memory pipe does. What they are good for is noticing when a change makes the library slower.

Three things dominate what is left. zig build bench -- --micro shows the frame checks running at some 2.4 GB/s, which is the floor for every protocol here; escaping costs a further pass over the bytes, and how much depends entirely on what is in them; and a protocol that stops for an acknowledgement after every block spends its time waiting rather than working, which is why XMODEM's 128 byte blocks are the slowest case measured and no amount of faster arithmetic changes that.

Every case runs with a Progress attached, and the table prints what the library reported about itself beside what was measured from outside it:

case             payload       MiB/s         ms   data MiB/s  reports
----------------------------------------------------------------------
zmodem crc32     text         1006.8        7.9       1030.4        2
xmodem 128       text          239.0       33.5        240.4        2
kermit w1        text          184.7       43.3        185.7        2

The two rates measure different spans -- the stopwatch covers the whole session, the reports cover the data phase alone -- so the gap between them is what the handshake and the teardown cost. That the two track each other at all is worth checking on every run: a rate nothing compares against anything is a rate nothing would notice had gone wrong.

The reports are inside the timing rather than beside it, so what a caller pays for them is in the numbers. It is a clock read and a comparison per subpacket or block, which the progress rows of --micro put at about thirty nanoseconds -- three per cent of a ZMODEM transfer over an in-memory pipe, five of one made of 128 byte blocks, and nothing measurable on any transport that is not memory.

Latency

A filter spends nearly all of its life not transferring anything: it sits between a terminal and a remote peer copying bytes both ways, and every microsecond it adds there is paid on every keystroke and every character echoed back. Throughput says nothing about that, so zig build bench -- --latency measures it separately -- one byte at a time, with the line otherwise idle and both pump tasks parked between samples, because a busy filter is not the case anyone is waiting on.

case                  p50 us    p90 us    p99 us    max us   late
------------------------------------------------------------------
pipe hop                11.0      14.3      19.6      19.6      0
keystroke out           10.6      13.4      23.4      23.4      0
output in               19.9      23.6      48.9      48.9      0
output in *             19.4      25.3      40.4      40.4      0
output in SOH           22.0      33.3     252.4     252.4      0
output in **            30.3      43.5     246.2     246.2      0
output in a*b           22.1      32.6     248.8     248.8      0
output ending rz     50079.0   50098.5   50110.9   50110.9      0
echo round trip         29.2      38.3      67.3      67.3      0
echo of *               29.4      48.4      67.8      67.8      0

Ordinary bytes cost tens of microseconds, most of it the ring the remote stream is carried on -- pipe hop is one byte through one ring with nothing above it, and is the floor the rest stands on.

The interesting bytes are *, which opens a ZMODEM header, and SOH, which opens a Kermit packet. Both are ordinary output the rest of the time: * is a printable character a terminal is full of, and SOH is what Ctrl-A echoes as. Seeing one, the filter has to decide whether a transfer is starting behind it, and the only way to know is to wait for what follows -- which used to mean holding the byte for a tenth of a second, on every * that went past.

A byte that might begin a transfer is forwarded before that is decided. It costs a stray character on the screen when a transfer really is starting, which is what a terminal without a filter shows anyway, and in exchange a * costs exactly what an ordinary byte costs. hold_candidate_bytes puts the old behaviour back for a caller who would rather keep the local stream clean and pay detect_interval for it.

A whole run of pads goes at once rather than one per interval, since a hex header opens with two of them and the first therefore cannot rule out the second -- which is what keeps a line ending in ** from having its last character held. They are shown, not consumed: a header can begin at the second pad of a run even when it does not begin at the first, so each is still examined in its turn.

Deciding afterwards is then free, which is what lets the deciding be thorough. detect_interval is per byte rather than for the whole header: a sender writes a header in one go, so what separates its bytes at the far end is the line, and every byte that arrives extends the wait. A slow line is followed for as long as it keeps delivering, and only a gap longer than the interval ends it. Fifty milliseconds covers a byte every 50 ms, which is slower than 300 baud, and the filter also stops as soon as the bytes in hand settle the question -- only another pad or a ZDLE can follow a pad, so output in a*b never consults the clock at all.

One row still shows the interval, and it is the filter withholding something on purpose: a withheld rz command has to wait to find out whether the header it announces is coming, which is the whole of what suppress_announcement does. Turn that off and nothing in the passthrough path waits for anything.

Fuzzing

Everything a peer can say is parsed by code in src, and src/fuzz_test.zig points a fuzzer at all of it: each protocol's framing, the metadata a sender chooses, Kermit's data field encoding, attribute lists and parameter exchange, each of the four receivers driven as a whole session against a script nobody meant, each of the four senders offering a file to a peer that answers nonsense, and the filter deciding whether what went past was a transfer.

The session targets put the whole script into the pipe before the engine starts and close it behind them, so a target that drives a protocol engine runs as fast as one that drives a parser: no read ever waits for a peer that will never speak.

Two targets ask for more than "does not crash". A sender and a receiver are run against each other with a wire between them that flips bits the fuzzer chooses, and whatever happens on the way, a file the receiver calls finished has to be the file that was sent -- damage is capped at the two flipped bits every block check here is guaranteed to notice, so a difference would mean a mistake in this library rather than arithmetic working as designed. Kermit's sliding windows are checked against a plain model of themselves, because the same packet is addressed three ways there -- by sequence number, by offset from the bottom of the window, and by physical slot -- and modular arithmetic is where that sort of thing goes wrong.

Both of those, and the session targets, carry a plain test beside them that checks the harness really moves a file across. A fuzz target that quietly stopped reaching the code it was written for would otherwise pass for ever.

Each target also carries a corpus of wire captures, which zig build test runs on every build. That is what keeps a case the fuzzer found from coming back, and it means the targets are worth having even where the fuzzer cannot be run.

zig build test --fuzz runs the fuzzer until interrupted, with a web interface, and --fuzz=1M makes a bounded run and prints a report. The report has one entry per test binary rather than per target, named after the binary's first fuzz test; every target in it ran. The test binary is always compiled with LLVM, even in Debug, because the self-hosted backend emits none of the coverage instrumentation the fuzzer steers by.

References

lrzsz and C-Kermit settle what the specifications leave open, but the specifications are still where the wire formats come from, and two documents are named often enough in the source to be worth writing down.

• Chuck Forsberg (ed.), XMODEM/YMODEM Protocol Reference, 18 June 1988: original, archived. Section 2 requires a receiver to "accept any mixture of 128 and 1024 byte blocks", section 4.3 defines the 1024 byte block and forbids a sender to change block length without an acknowledgement, and section 5 gives the layout of YMODEM's block zero. Wherever src/xmodem and src/ymodem say "the specification", this is it.

• Cisco, Cisco IOS Configuration Fundamentals Configuration Guide, Release 12.2SR: Loading and Managing System Images: original, archived. Describes copy xmodem: and copy ymodem:, the console file transfer a Cisco switch offers, and is why the sender falls back to 128 byte blocks. It documents the commands and not their limits: that this receiver refuses 1024 byte blocks outright is an observation from a transfer to a switch, not something Cisco writes down.

Where this lives

git clone https://git.jcollie.dev/jeff/zig-modem.git

The repository is mirrored on Tangled, at https://tangled.org/jcollie.dev/zig-modem.

Cloning with Radicle

The repository is also published on Radicle, a peer-to-peer code forge, where it is findable only by its Repository ID:

rad clone rad:z3Jbb3rZUQcrZrFeZHQmKkDGPdxZW

If you do not have Radicle set up yet, install the rad CLI and create an identity first:

curl -sSf https://radicle.xyz/install | sh
rad auth

Cloning also seeds the repository, which helps keep it available on the network.