Consider using a stream-based server such as filed

[cvan]

Per #73 (comment)

[mattdesl]

Yup 👍 this is a bit of re-shuffling code but I think it will come out cleaner in the end.

[mattdesl]

Starting to look into this which will hopefully fix #124, #119 and some other irks.

Lots of options.. might use a few of these:

• serve-static
• filed
• inject-lr-script-stream
• wayfarer
• size-stream
• http-ndjson

Or maybe just use express?

Ideally it can be refactored so the core of budo can be split out (see #122) and easily used in somebody's custom server/express app.

I haven't done a lot of server/backend stuff so any suggestions are welcome.

/cc @yoshuawuyts

[yoshuawuyts]

I wouldn't use express as it's not very flexible and kinda gets in the way of streams. The other modules you mentioned should be pretty good.

Only http-ndjson should be used with caution as it's kinda slow, but I don't think anyone will try to use budo to serve >5k req/sec without a cache 😁.

Here's a lil example of a what a server with these packages would look like:

const boleStream = require('bole-stream')
const httpNdjson = require('http-ndjson')
const sizeStream = require('size-stream')
const summary = require('server-summary')
const stdout = require('stdout-stream')
const pumpify = require('pumpify')
const bole = require('bole')
const http = require('http')

module.exports = createServer

function createServer (argv) {
  bole.output({ level: argv.logLevel, stream: stdout })
  const logStream = boleStream({ level: argv.logLevel })
  const port = argv.port

  // create server
  const server = http.createServer(function (req, res) {
    const httpLogger = httpNdjson(req, res)
    httpLogger.pipe(logStream, { end: false })

    const size = sizeStream()
    size.once('size', function (size) {
      httpLogger.setContentLength(size)
    })

    const sink = pumpify(size, res)
    api(req, res).pipe(sink)
  })

  // start the server on port
  server.listen(port, summary(server))
}

Here ./api is a function that exposes wayfarer and returns streams so they can be handled by size-stream. filed, html*, etc. should work well in this case.

[mattdesl]

Refactored the server handling in 8.0.0.

I ended up using connect-style middleware since they are easy to stack and build on, and eventually they can be split out of budo much like webpack-dev-middleware. There is no express/connect in this refactor, just stacked and the http module.

Still some more work to do: modularization, HTTPS support, etc. But hopefully this fixes the major pain points with the previous server code.