Skip to content

High-capacity render servers not fully utilized #1352

@pnorman

Description

@pnorman

During the day nidhogg and culebre have a queue, which means they should be working all-out to render new tiles. When I look at the daytime CPU utilization, it's about 40% with a CPU pressure of 0.3%. The system isn't IO limited either with IO pressure of <1%.

Image

When it does the nightly low-zoom re-render it gets closer to full utilization.

A few notes

  • renderd is taking about 110GB ram and this RAM usage goes away after restarting it. I assume this is a memory leak. The timing of when usage increases makes me wonder if it's going up on the nightly re-render. I've also only ever restarted it in off-peak hours.
  • restarting renderd makes a noticeable difference to the CPU and memory pressure
  • the only obvious tunable in renderd is num_threads which is set to hardware threads - 1, or 111 for these servers.

The obvious first step is to restart renderd in the day, get some data, turn off chef, increase num_threads, and restart again and see how the utilization is.

Metadata

Metadata

Assignees

Labels

service:tilesThe raster map on tile.openstreetmap.org

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions