mtr-log-collector

mtr-log-collector is a standalone service designed to replace MTRLogObserver from Buildbot, which has been deprecated in recent Buildbot versions.

Purpose

This service replicates the same database table models previously used by MTRLogObserver.

The key design decision behind this project is based on the observation that mariadb-test can output test results in JUnit XML format or record stdout in a stdout.log file.

How it works

The service exposes a single data processing endpoint: /upload-test-results

This endpoint allows uploading a JUnit-style XML or a stdout.log file along with additional metadata — usually available as build-level properties.

The service parses the content and inserts the data into the same tables that MTRLogObserver used, ensuring compatibility.

Deployment & Communication Model

The service is designed to run on the Buildbot master host, with communication from Buildbot worker hosts handled via WireGuard tunnels. Due to this setup, no authentication module is currently implemented in this version, as the secure communication is handled at the network level.

A typical client implementation is as simple as executing a curl command from a Buildbot build step, where the XML or the stdout.log files are generated by a previous step running the MTR tests.

Example usage in a build step:

XML

curl -f -X POST http://#BB_MASTER_WG_IP#:#SERVICE_PORT#/upload-test-results/ \
  -F branch=main \
  -F revision=#commit_sha# \
  -F platform=#builder_name# \
  -F bbnum=#build_number# \
  -F typ=#mtr_test_type# \
  -F file=@generated_by_mtr.xml;type=application/xml

LOG

curl -f -X POST http://#BB_MASTER_WG_IP#:#SERVICE_PORT#/upload-test-results/ \
  -F branch=main \
  -F revision=#commit_sha# \
  -F platform=#builder_name# \
  -F bbnum=#build_number# \
  -F typ=#mtr_test_type# \
  -F [email protected];type=text/plain

Compatibility

Because the database schema remains unchanged, tools like Cross Reference — used for reporting failed tests — can operate without any modifications. From its perspective, the data continues to come from the same source, even though it may now be populated by both Buildbot and mtr-log-collector.

Health Check & Metrics

The application exposes endpoints for health monitoring and metrics collection:

• The /health endpoint can be used to verify the health status of the application. Will return 200 if the application is up and the database is reachable.

• The /metrics endpoint exposes application metrics in a format compatible with Prometheus. These metrics can be scraped by Prometheus for monitoring and alerting purposes.

Summary

• Drop-in replacement for the deprecated MTRLogObserver.
• Stores mariadb-test test failures from JUnit XML or stdout.log files

---

Feel free to open issues or pull requests if you encounter problems or have suggestions for improvements.