gh-127221: Add colour to unittest output

[hugovk]

Use the existing _colorize module already used for tracebacks and doctest so output can be controlled with the PYTHON_COLORS, NO_COLOR and FORCE_COLOR environment variables.

This PR does a couple of things to add colour:

• Enable colour for tracebacks by passing in colorize to msgLines = list(tb_e.format(colorize=can_colorize()))
• Add colour for pass/fail/errors, using pytest as a guide
• Plus some unrelated tidy-up (use more specific test methods, new-style objects, underscore variables)

Tested with some small demo scripts:

No tests ran

run-unittests-none.py

import unittest

if __name__ == "__main__":
    unittest.main()

Before	After	pytest

Passed

run-unittests-pass.py

Mode	Before	After	pytest
Regular
Verbose

Failing

run-unittests.py

import unittest


class TestThings(unittest.TestCase):
    def test_pass(self):
        self.assertTrue(True)

    def test_fail(self):
        self.assertTrue(False)

    def test_error(self):
        raise Exception("Manually raised exception")

    def test_pass2(self):
        self.assertEqual(1, 1)

    @unittest.skip("demonstrating skipping")
    def test_skip(self):
        self.fail("shouldn't happen")


class ExpectedFailureTestCase(unittest.TestCase):
    @unittest.expectedFailure
    def test_fail(self):
        self.assertEqual(1, 0, "broken")

    @unittest.expectedFailure
    def test_pass(self):
        self.assertEqual(1, 1, "not broken")


if __name__ == "__main__":
    unittest.main()

Mode	Before	After	pytest
Regular
Verbose

Subtests

run-unittests-subtests.py

import unittest


class DoSubTest(unittest.TestCase):
    def test_subtest(self):
        """Test with subtest"""
        for i in range(0, 4):
            with self.subTest(i=i):
                if i == 2:
                    raise Exception("Manually raised exception")
                self.assertEqual(i % 2, 0)


if __name__ == "__main__":
    unittest.main()

Mode	Before	After	pytest
Regular
Verbose

---

📚 Documentation preview 📚: https://cpython-previews--127223.org.readthedocs.build/

• Issue: Add colour to unittest output #127221

[nineteendo]

Looks good, I see you're taking some inspiration from pytest, but I had to zoom in all the way to be able to read the text. :)

[hugovk]

You can click the images to open them full size.

[nineteendo]

Well, I wanted to compare the images so I had to zoom in to see the difference. Maybe we could only show the comparison between pytest? The uncoloured images don't add much value.

[AlexWaygood]

This is great!! With the coloured tracebacks as well, though, I sort-of feel like there might be a bit too much colour now when tests fail. It feels slightly overwhelming!

What about making only the "ERROR"/"FAIL"/"UNEXPECTED SUCCESS" lines coloured red, and not the ======= line above or the ------- below?

[AlexWaygood]

we have to apply this decorator to so many methods that I almost wonder if it's worth having a custom metaclass that automatically adds it to each method on the class... or we could not use the @force_not_colorized decorator, and instead duplicate the logic in setUp and tearDown methods on the TestCleanup class here.

Neither feels ideal; it might be that what you have now is in fact best!

[hugovk]

This is great!! With the coloured tracebacks as well, though, I sort-of feel like there might be a bit too much colour now when tests fail. It feels slightly overwhelming!

What about making only the "ERROR"/"FAIL"/"UNEXPECTED SUCCESS" lines coloured red, and not the ======= line above or the ------- below?

Thanks! Here's without coloured separators:

And verbose:

What do you think?

Also these demo scripts are a bit contrived; hopefully most of the time, most of your tests pass and just one or two fail. But then again, sometimes a lot do fail!

[AlexWaygood]

Nice, that looks great to me now!

[nineteendo]

How about swapping the rows and columns in the table? It makes the text better readable:

show table

Mode	Regular	Verbose
Before
After
pytest

[AlexWaygood]

How about swapping the rows and columns in the table? It makes the text better readable:

To be clear @nineteendo, this is a comment on the table in Hugo's PR description rather than the code he's changing in this PR, correct?

[nineteendo]

Correct, with 3 images on the same row I can't read the text without zooming in.

[AlexWaygood]

I see.

@hugovk, do you think we should add a link to the using-on-controlling-color somewhere in the unittest docs? I see we didn't when we added color support to doctest, but maybe we should have done

[hugovk]

@hugovk, do you think we should add a link to the using-on-controlling-color somewhere in the unittest docs? I see we didn't when we added color support to doctest, but maybe we should have done

Yeah, good idea. Something like 5a1de7c?

Let's also add it to doctest and traceback.

[AlexWaygood]

Nice!

[sobolevn]

Thank you! I like the idea. But, I personally feel like it is still a bit too colorful. Right now almost nothing is white.

One more idea: can you please test this with the white theme as well?

[hugovk]

Thank you! I like the idea. But, I personally feel like it is still a bit too colorful. Right now almost nothing is white.

Almost nothing is white because these demo scripts are intentionally crafted to generate lots of failures :) It's very similar to pytest.

One more idea: can you please test this with the white theme as well?

Sure:

unittest	pytest

Note I've not adjusted my local colours for this light theme. If I was using it normally, I may have adjusted it already for pytest.