[18.0][IMP] queue_job: add index for efficient autovacuum

[florentx]

This query was part of the slowest.

Before:

# EXPLAIN ANALYZE
SELECT id FROM "queue_job"
 WHERE (("date_done" <= '2025-07-01' OR "date_cancelled" <= '2025-07-01') AND "channel" = 'root.custom.purchase')
 ORDER BY "date_created" DESC, "date_done" DESC LIMIT 1000;
                                                                                QUERY PLAN                                                                                 
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=212724.34..212841.01 rows=1000 width=20) (actual time=1949.277..1966.346 rows=1000 loops=1)
   ->  Gather Merge  (cost=212724.34..220532.68 rows=66924 width=20) (actual time=1939.068..1956.102 rows=1000 loops=1)
         Workers Planned: 2
         Workers Launched: 2
         ->  Sort  (cost=211724.32..211807.97 rows=33462 width=20) (actual time=1880.715..1880.798 rows=779 loops=3)
               Sort Key: date_created DESC, date_done DESC
               Sort Method: top-N heapsort  Memory: 179kB
               Worker 0:  Sort Method: top-N heapsort  Memory: 180kB
               Worker 1:  Sort Method: top-N heapsort  Memory: 183kB
               ->  Parallel Bitmap Heap Scan on queue_job  (cost=944.61..209889.63 rows=33462 width=20) (actual time=16.823..1873.439 rows=27435 loops=3)
                     Recheck Cond: ((channel)::text = 'root.custom.purchase'::text)
                     Rows Removed by Index Recheck: 51603
                     Filter: ((date_done <= '2025-07-01 00:00:00'::timestamp without time zone) OR (date_cancelled <= '2025-07-01 00:00:00'::timestamp without time zone))
                     Rows Removed by Filter: 224
                     Heap Blocks: exact=13666 lossy=12625
                     ->  Bitmap Index Scan on queue_job_channel_index  (cost=0.00..924.54 rows=83748 width=0) (actual time=13.932..13.932 rows=82979 loops=1)
                           Index Cond: ((channel)::text = 'root.custom.purchase'::text)
 Planning Time: 1.395 ms
 JIT:
   Functions: 19
   Options: Inlining false, Optimization false, Expressions true, Deforming true
   Timing: Generation 4.315 ms, Inlining 0.000 ms, Optimization 3.396 ms, Emission 37.965 ms, Total 45.676 ms
 Execution Time: 1968.386 ms
(23 rows)

---

After:

# EXPLAIN ANALYZE
SELECT id FROM "queue_job"
 WHERE (("date_done" <= '2025-07-01' OR "date_cancelled" <= '2025-07-01') AND "channel" = 'root.custom.purchase')
 ORDER BY "date_created" DESC, "date_done" DESC LIMIT 1000;

                                                                                                       QUERY PLAN                                                                                                       
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=0.43..66770.92 rows=1000 width=20) (actual time=0.750..93.965 rows=1000 loops=1)
   ->  Index Scan Backward using queue_job_date_created_date_done_index on queue_job  (cost=0.43..5362271.99 rows=80309 width=20) (actual time=0.748..93.895 rows=1000 loops=1)
         Filter: (((channel)::text = 'root.custom.purchase'::text) AND ((date_done <= '2025-07-01 00:00:00'::timestamp without time zone) OR (date_cancelled <= '2025-07-01 00:00:00'::timestamp without time zone)))
         Rows Removed by Filter: 40686
 Planning Time: 0.312 ms
 Execution Time: 94.057 ms
(6 rows)

[OCA-git-bot]

Hi @guewen,
some modules you are maintaining are being modified, check this out!

[guewen]

Thanks @florentx!

The index is scanned backwards because the query sorts by DESC and the index goes ASC.
We could use an index on (date_created DESC, date_done DESC), but changing the ordering in the autovacuum's query to sort by ASC - aside of speeding up the index - would drop the oldest jobs to the newest jobs which actually makes more sense, what do you think?

[florentx]

You're right @guewen .

I've forced an order= argument, and changed the index.

# EXPLAIN ANALYZE                                                                                             
SELECT id FROM "queue_job"
 WHERE (("date_done" <= '2025-07-01' OR "date_cancelled" <= '2025-07-01') AND "channel" = 'root.custom.purchase')
 ORDER BY date_done, date_created LIMIT 1000;
                                                                                 QUERY PLAN                                                                                  
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=0.55..3756.69 rows=1000 width=20) (actual time=0.138..0.877 rows=1000 loops=1)
   ->  Index Scan using queue_job_channel_date_done_date_created_index on queue_job  (cost=0.55..301651.84 rows=80309 width=20) (actual time=0.137..0.842 rows=1000 loops=1)
         Index Cond: ((channel)::text = 'root.custom.purchase'::text)
         Filter: ((date_done <= '2025-07-01 00:00:00'::timestamp without time zone) OR (date_cancelled <= '2025-07-01 00:00:00'::timestamp without time zone))
 Planning Time: 0.200 ms
 Execution Time: 0.918 ms
(6 rows)

[guewen]

Great!

[guewen]

/ocabot merge minor

[OCA-git-bot]

This PR looks fantastic, let's merge it!
Prepared branch 18.0-ocabot-merge-pr-798-by-guewen-bump-minor, awaiting test results.

[OCA-git-bot]

Congratulations, your PR was merged at 45dabc7. Thanks a lot for contributing to OCA. ❤️