HTTP head framing

Operations and limits

The explicit byte limit includes the final four delimiter bytes. It must be a fixnum from 4 through 1048576. Invalid arguments raise before reading. A complete block exactly at the limit succeeds; reaching the limit without a complete delimiter fails immediately without consuming the next byte. The four-byte empty delimiter can be framed, but requires rejection by a complete request parser.

The accumulator is allocated once at the limit; bytes are appended without rebuilding a growing string. The delimiter scan starts where the previous span ended. The I/O reader's bounded queue is the only read-ahead queue. It may prefetch beyond the header, but framing consumes only the selected prefix. io.peek-some waits for at least one byte and exposes the currently available span, so capture does not wait for a full reader buffer or for the peer to close.

Returned bytes include CRLFCRLF and own their storage. They survive later reader use, refill and endpoint close. On completion the framer drops its accumulator. Peak storage is bounded by the head limit, its final copy, and reader/endpoint chunk capacities; retaining many returned views remains the caller's choice. A framer captures one head. It remembers its first successful result, EOF or error; repeated calls return that result without consuming another message. Use a complete body-framing policy before reusing input for another request. One task owns the framer and its reader, with no independent reader consumption while capture is active. The framer does not close its endpoint.

Failure and deadlines

EOF before any byte returns [eof]; EOF within a partial head is "incomplete head". Framing errors use operation http-head, domain library and resource "HTTP head". Transport errors retain native details. Progress counts bytes consumed by this capture; for a transport failure it includes all earlier spans. An invalid byte is included in that consumed count. Unprocessed suffix bytes stay in the I/O reader.

Errors, including timeouts, are terminal for this capture. An unexpected backend exception raises and poisons the framer. Operations on a closed/stale underlying endpoint raise. A saved managed framer can resume with its reader; saved native resources and deadlines must be recreated as described on their pages.

http.read-head-until needs deadline.shen. It checks the same absolute budget before each span and again before publishing a completed block, including when all bytes were already queued. It cannot interrupt an arbitrary backend that ignores deadlines while waiting. On sockets, install that same budget on the endpoint before reading:

(define read-timed-head
  E Limit Milliseconds ->
    (let D (clock.deadline Milliseconds)
         _ (io.socket-deadline E D)
         F (http.head-reader (io.reader E []) Limit)
      (http.read-head-until F D)))

Load ffi.shen, socket.shen, buffer.shen, io.shen, deadline.shen, clock.shen, io-socket.shen and http-head.shen for that function. A new response phase can install a fresh budget; reads that return short prefixes do not create new header budgets. Cancellation, connect/accept/DNS budgets, request parsing, bounded bodies and response encoding are available above this layer; a listener loop remains separate work.

Types and capture example

The public types are opaque http.head-reader and the pattern-matchable http.head-result:

(define head-size
  {http.head-result --> number}
  [head V] -> (buffer.length V)
  [eof] -> 0
  [error Error] -> (io.error-progress Error))

tools/http-head-capture.shen combines the socket policy and framer in an owned connection scope. http-head.capture Endpoint Limit Milliseconds returns [ok OwnedView] or [error Error] and closes the endpoint on success or failure. This debugging example sends no HTTP response; it is not yet an HTTP server. It demonstrates that framing can finish before peer EOF under one total deadline.

make test-io covers every small-header split and truncated prefix, byte limits, invalid controls, body preservation, fake-clock expiry, native timeouts, typed clients and saved-image/GC behavior. The optional independent TCP check is python3 tests/http_head_wire.py build/shen-jit; it verifies captured wire bytes, oversized rejection and a peer that keeps sending slowly beyond its original budget.