This repository was archived by the owner on Apr 26, 2024. It is now read-only.
Repository navigation
This repository was archived by the owner on Apr 26, 2024. It is now read-only.
Expand and improve workers documentation and example configuration #9072
Description
A general call;
- Add more documentation surrounding workers (just in general)
- Add example configuration file (a-la
sample_config.yaml) for each worker type, or for all workers in one - Add docker-compose file that includes ready-to-work example for synapse workers
Imo: Currently i think workers are very hard to set up or figure out on its own, because knowledge of it is mostly limited to large setups, and thus that knowledge is never reflected in documentation, or workers seen as an asset to be documented with the same priority.
Activity
[-]Expand and improve workers documentation, add example configuration for each + docker-compose configuration[/-][+]Expand and improve workers documentation and example configuration[/+]clokep commentedon Jan 12, 2021
Is there specific information that you think is missing? the workers docs were recently expanded to include a bit more information.
I think this is pretty much #8557.
The current docker-compose stuff is under contrib, so I unfortunately this will probably need to be provided via a contribution. 😄
ShadowJonathan commentedon Jan 12, 2021
@clokep why was
info-neededadded here, i thought i was pretty clear; improve workers documentation and add samples, for the first one there's already an issue (good, cuz then i can track that via here), and for the second one i havent seen a thing (with issue or PR), or should that issue be added to another repo?clokep commentedon Jan 12, 2021
@ShadowJonathan The bit that is unclear to me is:
What information do you feel is missing?
ShadowJonathan commentedon Jan 12, 2021
Oh, that part is a general call for a look at the documentation, but i'm not holding it for a requirement (thus the absence of checkmark), its generally just "hey, please give a look at the documentation for workers and judge if everything you know about setting up and configuring workers is in there" request, should i remove it to avoid confusion?
richvdh commentedon Feb 11, 2021
we certainly need better documentation for how to run workers under docker
TheDiscordian commentedon Apr 8, 2021
I was asked to drop my opinion in here while we were discussing this issue in chat:
There's a lot of info on types of workers, and little on how to redirect queries from a reverse proxy to the workers effectively. I know it sounds simple to say "just run a worker per CPU", but I didn't find my experience to feel quite so clear-cut.
Put simply, an example showing how to run a server with 4+ cores, using Nginx as a reverse proxy, with an appropriate worker setup / Nginx config would go a long way I think.
ShadowJonathan commentedon Apr 8, 2021
Extra thoughts;
pgeier commentedon Apr 8, 2021
The Loadbalancing Section is a little bit confusing.
Is there loadbalancing by default? I read that it is customizable to redirect same users to same nodes, or specific rooms to same nodes.
However is this already working by default, how can I configure it?
At least it should be statet that i need to bring my own load balancer.
pgeier commentedon Apr 8, 2021
... asked by @jwh:covert.tel from the discourse on #synapse:matrix.org
pgeier commentedon Apr 9, 2021
Maybe reference existing helm charts for Kubernetes?
Thes deprecated repo (https://github.com/ananace/matrix-synapse) references
https://gitlab.com/ananace/charts (espacially https://gitlab.com/ananace/charts/-/tree/master/charts/matrix-synapse up to date!)
and
https://github.com/dacruz21/matrix-chart/
At least concerning "large deployments", concrete kubernetes images are very helpful - at least to mention their existence
EDIT: Within the INSTALL.md some docker images and ansible... but these seem to focus on single process instances - multiple workers/processes are not even mentioned here yet
richvdh commentedon May 24, 2021
Related issues about workers docs:
/typing#9477to_devicestreams and encryption endpoints can be on workers #9046dklimpel commentedon Apr 12, 2022
I have found some more issues with worker docs:
federation_reader-> systemd-with-workers talks about federation_reader, which is undocumented #8740synapse/docs/systemd-with-workers/workers/federation_reader.yaml
Line 1 in 3201863
Event Persistertomainprocess. But thestream writerforeventsis ageneric_workerand starts only with a connection (worker_replication_host) tomainproccessThumbscrew commentedon May 14, 2022
I've raised PR #12737 with some fairly detailed instructions for using Docker Compose to create workers. Please feel free to provide feedback and let me know if this is helpful.