neilotoole.io

Web presence of Neil O'Toole.

streamcache

neilotoole/streamcache is a package I’ve had loitering in the backlog for some time. I finally polished it off for use with sq, to handle streaming from stdin or downloads. streamcache implements a Go in-memory byte cache mechanism that allows multiple callers to read some or all of the contents of a source io.Reader, while only reading from the source reader once. When only the final reader remains, the cache is discarded and the final reader reads directly from the source. This is particularly useful for scenarios where multiple readers may wish to sample the start of a stream, but only one reader will read the entire stream.

streamcache_multicase.png