Support monochrome AVIF saving and loading

[NogginBops]

What did you do?

I have a bunch of gray-scale datasets what I would like to compress using the AVIF format to reduce the amount of data sent. Currently I'm able to store my grayscale data in an RGB AVIF file, but I want to use the grayscale compression ability of AVIF to hopefully get an even better compression ratio and not have to discard a bunch of extra data when loading the data file.

What did you expect to happen?

For the AVIF encoder/decoder to support gray-scale images.

What actually happened?

The AVIF encoder/decoder only supports RGB images.

What are your OS, Python and Pillow versions?

• OS: Windows 10
• Python: 3.12.10
• Pillow: 12.1.1

im = PIL.Image.fromarray(img_data.astype(np.uint8)) # shape (nx, ny, 1)
im.save("image.avif")
# This saves an RGB avif file instead of a grayscale file.

[radarhere]

This saves an RGB avif file instead of a grayscale file.

This might sound like a silly question, but how do you know that? What tool are you using to inspect the file?

[NogginBops]

This saves an RGB avif file instead of a grayscale file.

This might sound like a silly question, but how do you know that? What tool are you using to inspect the file?

When I decode the file I get a 3 channel output. I mean it could be the decoder deciding it's 3 channels. But I also looked at the AVIF code in pillow and it explicitly does not handle grayscale data so I'm pretty certain it doesn't use the grayscale capabilities of avif, though admittedly I've not inspected the file itself.

[radarhere]

And you're using Pillow as the decoder?

[NogginBops]

Yes.

Here is the code that converts the image to RGB(A) before saving:

Pillow/src/PIL/AvifImagePlugin.py

Lines 248 to 253 in 3a44ba1

	# Make sure image mode is supported
	frame = ims
	rawmode = ims.mode
	if ims.mode not in {"RGB", "RGBA"}:
	rawmode = "RGBA" if ims.has_transparency_data else "RGB"
	frame = ims.convert(rawmode)

And here is the c code that selects the AVIF format to use when saving:

Pillow/src/_avif.c

Lines 506 to 510 in 3a44ba1

	if (strcmp(mode, "RGBA") == 0) {
	rgb.format = AVIF_RGB_FORMAT_RGBA;
	} else {
	rgb.format = AVIF_RGB_FORMAT_RGB;
	}

Here is the code that converts the decoded image to RGB(A):

Pillow/src/_avif.c

Lines 773 to 798 in 3a44ba1

	rgb.depth = 8;
	rgb.format = decoder->alphaPresent ? AVIF_RGB_FORMAT_RGBA : AVIF_RGB_FORMAT_RGB;
	result = avifRGBImageAllocatePixels(&rgb);
	if (result != AVIF_RESULT_OK) {
	PyErr_Format(
	exc_type_for_avif_result(result),
	"Pixel allocation failed: %s",
	avifResultToString(result)
	);
	return NULL;
	}
	Py_BEGIN_ALLOW_THREADS;
	result = avifImageYUVToRGB(image, &rgb);
	Py_END_ALLOW_THREADS;
	if (result != AVIF_RESULT_OK) {
	PyErr_Format(
	exc_type_for_avif_result(result),
	"Conversion from YUV failed: %s",
	avifResultToString(result)
	);
	avifRGBImageFreePixels(&rgb);
	return NULL;
	}

[radarhere]

Thanks.

While there seems to be an obvious way to designate an image as grayscale when encoding, rgb.format = AVIF_RGB_FORMAT_GRAY, I don't see an obvious way to determine if it is grayscale when decoding.

avifDecoder has the alphaPresent flag that we're currently using, but nothing else looks directly helpful. Indirectly, it has an image, but nothing in avifImage looks obvious either. image has depth, but an L mode image has the same depth as an RGB mode image, 8.

I started to wonder if it was even possible to tell, so I thought I'd ask what you were using to check.

I've not given up, I'm just letting you know where I'm at right now.

[NogginBops]

AVIF_PIXEL_FORMAT_YUV400 means grayscale (full-res luminance and no chrominance). So you could use that to detect grayscale.
https://github.com/AOMediaCodec/libavif/blob/559c589b1f3d726c8051ce3e4c876b65b3d4c69a/include/avif/avif.h#L784

See:
AOMediaCodec/libavif#65

It's also worth noting that alpha is separate from color format so both color + alpha, gray + alpha and only alpha are also possible (though I only care about pure grayscale images without alpha).
https://github.com/AOMediaCodec/libavif/blob/559c589b1f3d726c8051ce3e4c876b65b3d4c69a/include/avif/avif.h#L279-L289

[radarhere]

yuvFormat is a separate setting. If you want to use AVIF_PIXEL_FORMAT_YUV400, you can already with im.save("out.avif", subsampling="4:0:0")

I have a bunch of gray-scale datasets what I would like to compress using the AVIF format to reduce the amount of data sent.

If reducing file size is your ultimate goal, I don't think that AVIF_RGB_FORMAT_GRAY achieves it. Using img_data = np.ones([100, 100]), I ran a test in Pillow main, and created a 320 byte file. Testing with AVIF_RGB_FORMAT_GRAY, the result is the same.

However, im.save("out.avif", subsampling="4:0:0") does reduce the file size to 307 bytes.

[NogginBops]

What I'm saying is that YUV400 should be the default setting for grayscale images, and that when seeing YUV400 the decoder should produce a single channel "L" image as the result. This would be done by specifying AVIF_RGB_FORMAT_GRAY for the input texture, setting AVIF_PIXEL_FORMAT_YUV400 for the encoding. Then when loading, if the YUV format is AVIF_PIXEL_FORMAT_YUV400 then a single channel grayscale image should be read using AVIF_RGB_FORMAT_GRAY when decoding.

This would result in smaller binary images and avoid a bunch of work in libavif to detect and convert the color data while encoding/decoding.
Here is some code that explicitly uses AVIF_RGB_FORMAT_GRAY to avoid computation: https://github.com/AOMediaCodec/libavif/blob/a1153dbccbdf005ce5b6632cdb2cf8be3c28d5ce/src/reformat.c#L248-L270
The decoding also has fast-paths for converting AVIF_PIXEL_FORMAT_YUV400 to AVIF_RGB_FORMAT_GRAY.

So say I have data with the shape (512, 512, 1), I want to encode it with AVIF_RGB_FORMAT_GRAY (for speed) with AVIF_PIXEL_FORMAT_YUV400 (for binary size) and then when I decode it I want to detect AVIF_PIXEL_FORMAT_YUV400 and use AVIF_RGB_FORMAT_GRAY (for speed) to convert that into data with the same (512, 512, 1) shape.
Currently I would get data with the shape (512, 512, 3) and a bunch of extra unnecessary work being done.

[radarhere]

Ok, I've created #9471 to resolve this.

[NogginBops]

That is perfect, thank you so much!