http: concatenate duplicate headers by default

[rlidwka]

Currently node.js has a list of headers which are allowed to be duplicated, and is dropping any headers that aren't on that list (or start with x-). I'm suggesting to do the opposite: build a list of known singular headers and treat others as multiple.

Relevant discussion is in issue #2750 (note: I believe a bug with not concatenating DAV still exists).

I took a list of singular headers from firefox source code, since firefox seem to do the same thing I'm suggesting here.

More detailed explanation. There are four kinds of headers:

1. known singular (i.e. Content-Type)
2. known multiple (i.e. Accept)
3. unknown regular (i.e. FooBar)
4. unknown starting with "x-" (i.e. X-FooBar)

Currently node.js treats 2 & 4 as multiple, and 1 & 3 as singular. Suggested change: treat 2 & 3 & 4 as multiple, 1 as singular.

Current behaviour creates a few issues:

• If there are unknown multiple regular (non-"x-") headers, they need to be included in the list. There were countless pull requests to do that in the past (HTTP: Pragma header as per RFC2616 can be comma separated #715, Concatenate WWW-Authenticate HTTP headers. #1083, Support multiple 'link' headers #1187, Add WebSocket RFC6455 multiheader fields to the http parser #2764, handle multiple values of 'proxy-authenticate' #4052), and more (http: support multiple Via headers #6660) are coming.
• Node.js is dropping perfectly valid headers until the patch to include a header is made. Old versions will still do that.
• If someone creates a custom header without X- prefix, it is treated as singular one. We can't possibly make a full list of multiple headers, because people can add custom ones.
• There is a difference between handling X- headers and regular ones. There should not be a difference according to RFC 6648. So current behaviour might be confusing, and is clearly violates this part of RFC:

   Implementations of application protocols MUST NOT make any
   assumptions about the status of a parameter, nor take automatic
   action regarding a parameter, based solely on the presence or absence
   of "X-" or a similar construct in the parameter's name.

Suggested approach will solve all of this at the cost of concatenating unknown singular headers (but those requests are invalid anyway).

[indutny]

+1 for this PR, need to hear @tjfontaine or @trevnorris opinion before landing.

[trevnorris]

The code LGTM, and I'm fine w/ the change. But also suspect there will be unforeseen consequences from module authors that do things with HTTP headers (e.g. connect module)

[indutny]

May I ask you to move dest[field] to the next line (with proper indent)

[indutny]

Ok, very minor style nits, otherwise LGTM too.

[rlidwka]

@indutny , style issues fixed

[indutny]

Thank you, landed in ec57ecc