Performance improvement

[theseion]

• Reworked http module to run tests much faster by removing most of the timeouts.
• Updated dependencies
• Added Python 3.10
• Fixed a couple of deprecations

[fzipi]

Wow @theseion I love what I am seing here! So moving to select gives us probably way better efficiency. And only reading time if we are going to use it, nice catch. I wonder why the time.sleep was being used, maybe @fgsch has better idea. Also, he might know more if this might affect upstream users.

Other questions:

• Do we need to update documentation to reflect on these changes?
• Is there any additional test we should consider on adding, just to be safe?

[dune73]

Cool, @theseion.

For the record, we still plan to move to go-ftw with the project, so this is simply cool, nice2have and good for ftw, but without long term impact as far as CRS is concerned. Correct?

[theseion]

For the record, we still plan to move to go-ftw with the project, so this is simply cool, nice2have and good for ftw, but without long term impact as far as CRS is concerned. Correct?

That is correct. It's mainly for me, so I can run tests for the changes I'm making to the regexp-assemble stuff.

[dune73]

Thank you. It's very nice to PR these fixes nevertheless.

[theseion]

I have this version working now with the CRS test suite. The tests for Apache all pass. The tests for NGiNX don't but at least the failures are now stable (with the current setup we have random failures).

I've also taken the opportunity to update dependencies but that can easily be split into a separate pull request if necessary.

[fzipi]

LGTM. As there is a small change for the Windows platform, should we fire tests using Windows also?

[theseion]

Yes, that's probably a good idea. Do you have it set up on a Windows machine?

[theseion]

I ran the tests on my Windows machine (Windows 11) in Powershell. There's one test that is flaky but it's also flaky on macOS (example.com sometimes returns 404 instead of 200 for "Test multipart using list"). Otherwise, everything looks good.

[fzipi]

@fgsch I think this one is ready. Do you have any comments here?

[fgsch]

👋 I'm adding comments as I go through this PR (so far non-blockers).
It'd be great if you could split this PR into multiple smaller ones so it's easier to review 🙏

[theseion]

This PR has been replaced by #68, #69, #70.