Allow recording the raw HTTP/1 headers sent and received #3846
Conversation
This allows feeding them into a WARC file (https://en.wikipedia.org/wiki/WARC_(file_format)). The rest of the request and response is already available as it's either the explicitly set body or the received response body. This includes the final `\r\n\r\n` between the headers and the response (so that it can be distinguished from bare `\n\n`). Needed for a similar request in reqwest: seanmonstar/reqwest#1229.
|
I'm already using this successfully through a patched reqwest to generate WARC files. Remaining questions: Is this the correct API? One could hide the actual extension types this is stored in and instead provide extension methods for the response to get raw request/response header bytes. This wouldn't tie hyper to the exact types, but they could be merged/organized differently later on. |
|
We had something kinda like this for a little while for the C API, but then removed it. There's #3417 that re-introduces it, though it needed to target master. It looks like this does a little more: you distinguish between request and response. Why not use use a single type? I think it's because you're trying to capture what is encoded too. FWIW, that could only be done for the client, the server wouldn't have a way to give that to the user. Depending on the purpose, a logging connection type could also do this without any help from hyper, such as reqwest's |
Yes, that's why. I want to capture both what headers are sent and what headers are received, in a byte-perfect fashion.
The purpose is recording requests I make into WARC files, i.e. permanently storing requests I make and responses I get through reqwest. I could try parsing that out of |
This allows feeding them into a WARC file (https://en.wikipedia.org/wiki/WARC_(file_format)). The rest of the request and response is already available as it's either the explicitly set body or the received response body.
This includes the final
\r\n\r\nbetween the headers and the response (so that it can be distinguished from bare\n\n).Needed for a similar request in reqwest: seanmonstar/reqwest#1229.