toolkit: add feature to profile & trace to have better diagnostics

[mfrw]

Signed-off-by: Muhammad Falak R Wani falakreyaz@gmail.com

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

• The toolchain has been rebuilt successfully (or no changes were made to it)
• The toolchain/worker package manifests are up-to-date
• Any updated packages successfully build (or no packages were changed)
• Packages depending on static components modified in this PR (Golang, *-static subpackages, etc.) have had their Release tag incremented.
• Package tests (%check section) have been verified with RUN_CHECK=y for existing SPEC files, or added to new SPEC files
• All package sources are available
• cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json, .github/workflows/cgmanifest.json)
• LICENSE-MAP files are up-to-date (./SPECS/LICENSES-AND-NOTICES/data/licenses.json, ./SPECS/LICENSES-AND-NOTICES/LICENSES-MAP.md, ./SPECS/LICENSES-AND-NOTICES/LICENSE-EXCEPTIONS.PHOTON)
• All source files have up-to-date hashes in the *.signatures.json files
• sudo make go-tidy-all and sudo make go-test-coverage pass
• Documentation has been updated to match any changes to the build system
• Ready to merge

---

Summary

What does the PR accomplish, why was it needed?
Add profiling and tracing support to the toolkit. The go toolchain has inbuilt tools to diagnose logic and performance problems in Go programs. This PR integrates that support for our toolkit.

This PR introduces 3 additional flags, by default all the three flags are disabled.

• ENABLE_TRACE: Setting this flag to y, instruments the various tools in the toolkit to record trace data. Tracing is a way to instrument code to analyze latency throughout the lifecycle of a call or user request. Traces provide an overview of how much latency each component contributes to the overall latency in a system. Traces can span multiple Go processes.
• ENABLE_MEM_PROFILE: Setting this flag to y instruments the toolkit to record the memory profile.
• ENABLE_CPU_PROFILE: Setting this flag to y instruments the toolkit to record the cpu profile.

Profiling tools analyze the complexity and costs of a Go program such as its memory usage and frequently called functions to identify the expensive sections of a Go program.

It is advisable to only turn one of the flags for a run to get data that does not have noise due to other profiling/tracing flags.

The trace/profile data will be created in the build/profile directory.
To use the profile data: go tool pprof -http=:8080 file.profile
To use the trace data: go tool trace -http=:8080 file.trace

Change Log

• NA

Does this affect the toolchain?

NO

Associated issues

• NA

Links to CVEs

• NA

Test Methodology

• Local Build

[dmcilvaney]

Linter will probably be grumpy about the capitalization of "Unable". It likes error strings to be all lowercase.

[dmcilvaney]

The big block of flags in the Makefile still seems messy, but not a deal breaker.
Added one suggestion to make the error 'optional' but I think this looks ok.