Support chunked encoding

[diazona]

I came up with this patch set a while ago for another project of mine that was experiencing errors only when receiving chunk-encoded data. It adds the ability to serve a chunked response. I know that users can do this themselves using a custom WSGI application, but since chunking is a part of the HTTP 1.1 standard, it's a common enough use case that it might be reasonable to handle it directly.

I'd be curious for your opinions @redtoad and @coordt about whether this makes sense to add to the project.

[coordt]

I like this change.

I would like to see some documentation for it in the readme, but the code looks fine.

[diazona]

Ah yeah, good call. I'll add a mention of it.

At some point I think it'd be nice to split out the documentation into separate files and use Sphinx to generate proper package documentation which we can then upload to ReadTheDocs, leaving the README for the basics, like installation instructions and the simplest possible usage example. But that's a whole separate discussion.

[diazona]

I was actually just looking at the existing commented-out test case for gzip compression and I noticed it uses an attribute of httpserver itself to enable the compression, rather than basing it on the Accept-Encoding header. That got me thinking about whether it'd be better to do the same with chunking as well, i.e. make the user set httpserver.chunked to True or False to enable/disable chunking, instead of deciding whether to apply chunking based on the Transfer-Encoding header.

@redtoad if you have the chance, maybe you can comment on how intentional that decision was in the gzip case? (to use an attribute of httpserver instead of deciding to apply the encoding based on the header)

Aside from code clarity, the only functional advantage I can think of to using an attribute is that it allows clients to test whether they handle incorrectly encoded HTTP responses, like if a server applies chunking but doesn't set the Transfer-Encoding header, or vice versa. But I could imagine that some clients might want to do that, e.g. if someone is writing a library that has to deal with badly written HTTP servers. (The counterpoint is that they can always write a custom WSGI application for it... 🤷)

[redtoad]

@redtoad if you have the chance, maybe you can comment on how intentional that decision was in the gzip case? (to use an attribute of httpserver instead of deciding to apply the encoding based on the header)

If memory serves right I wanted to explicitly turn compression on/off and not base it on the request header.

Maybe that would something for an "enum": YES, NO, AUTO (i.e. if requested via header)

[diazona]

Good idea, I'll change it accordingly.

[diazona]

I've implemented the version with the enum; thanks for that suggestion 😄

It breaks compatibility with Python 2.7 and 3.3, because the enum module was introduced in 3.4. We could add a dependency on the enum34 package (backport of enum) to continue supporting those older versions, but I'm feeling lazy and kind of hoping we can just drop support for 2.7 and 3.3 instead (see also #25).

[diazona]

I've been trying this out in a private project and am getting a weird bug that is probably due to the other project's code, but I'm not sure. I'm hoping to get to the bottom of it before merging this, just in case there's an issue with the chunked encoding handling here.

[diazona]

I haven't fully gotten to the bottom of the issue I mentioned with my other project, but I think I've convinced myself that it lies in the other project's code, not here. So I'm good to merge this. @redtoad @coordt any further comments before I do?