diff --git a/src/native/eventpipe/README.md b/src/native/eventpipe/README.md new file mode 100644 index 00000000000000..eae5604bf8c458 --- /dev/null +++ b/src/native/eventpipe/README.md @@ -0,0 +1,112 @@ +# EventPipe and DiagnosticServer + +If there is information you wish was here and it isn't, please add it :) + +## Overview + +EventPipe is a cross-platform eventing library written in C with significant inspiration from ETW +on Windows. Previously when .NET primarily ran on Windows we relied solely on ETW, but now that +we run on multiple platforms we wanted to have cross platform logging supported directly in the +runtime. For more info see [the docs](https://docs.microsoft.com/en-us/dotnet/core/diagnostics/eventpipe). + +DiagnosticServer is a simple RPC mechanism that allows external tools to communicate with the +runtime over named pipes and sockets. External tools send the runtime commands which the runtime +executes and responds to. Creating EventPipe logging sessions are one of the commands supported +over the RPC channel but there are also others entirely unrelated to EventPipe functionality. + +## EventPipe concepts + +EventPipe collects structured log messages from instrumented code, potentially buffers it, and then +emits it to a logging sink, typically a serialized file on disk, named pipe or socket. The schema for +each logged message is defined in an EventPipeEvent. EventPipeProviders act as namespaces for +EventPipeEvents. In order to receive log messages a telemetry consumer creates an EventPipeSession +which records the configuration about which EventPipeEvents should be enabled, which logging +sink should be used, and manages storage for intermediate storage. + +A simple workflow looks like this: +1. Managed code creates an EventSource. The implementation of EventSource automatically creates +EventPipeProvider and EventPipeEvent objects that correspond to the EventSource object and the +individual logging methods on the EventSource. At this point nobody is listening so invoking +EventPipe WriteEvent() APIs don't record any data. + +2. At some point a user uses dotnet-trace to send an IPC command to the runtime that creates +a new EventPipeSession. This session enables some providers specifying a level and keywords +which determine the events to enable. The session creates an EventPipeBufferManager to manage +the in-memory log buffering and starts a thread that will dequeue messages from the buffer +and serialize them to an outbound stream. + +3. Now when EventSource APIs are called, WriteEvent() serializes a buffer of data and saves it +in the session's buffer. Asynchronously a session specific thread dequeues it, formats it, and +writes it out to the IPC stream. + +4. dotnet-trace listening at the other end of the IPC stream receives the log messages and +serializes them to disk. Later the user takes that file to Visual Studio or PerfView to visualize +the contents. + + +## Guide to the code + +EventPipe was initially written in C++ for coreclr and then translated to C for use by both Mono and +CoreCLR runtimes. The code depends solely on the C runtime and a limited set of runtime specific +implementations for basic datatypes, locks, threads, etc. The set of functionality each runtime is +expected to provide is defined in ep-rt-* files (ep=EventPipe and rt=Runtime). Each runtime then +needs to compile a separate lib that implements this ABI and link it together. For example CoreCLR's +implementation is in ../../coreclr/vm/eventing/eventpipe and mono's implementation is in +../../mono/mono/eventpipe. Files starting with ep-* are the runtime neutral portions. + +Files starting with ds-* are the runtime neutral implementation of DiagnosticServer. ds-rt-* files +are the runtime specific dependencies, following the same pattern used by EventPipe. + +### Getters and Setters + +The code uses macros to define getter and setter functions in ep-getter-setter.h. It maps like this: +``` +typename_get_fieldname(instance) { return instance->fieldname; } +typename_get_fieldname_ref(instance) { return &(instance->fieldname); } +typename_get_fieldname_cref(const instance) { return &(instance->fieldname); } +typename_set_fieldname(instance, value) { instance->fieldname = value); } +``` + + +### Datatypes + +Many templated datatypes are defined implicitly using macros such as EP_RT_DEFINE_ARRAY(). +These macros define a set of functions following a specific naming pattern and like all *rt* +functionality the expectation is that each runtime will implement it. These indirections make the +code a little harder to follow but you can decode it if you understand the mapping. The source is +always definitive but at this time here is mapping: + +```` + CoreCLR Mono +ARRAY CQuickArrayList GArray +LIST SList GList +QUEUE SList GQueue +HASH_MAP SHash>> GHashTable +HASH_MAP_REMOVE SHash> GHashTable +```` + +Given some method such as ep_rt_thread_session_state_array_init() you can't find it directly in the +source because it is constructed by macros, however we can still track it down if we need to: +1. The naming convention is always ep_rt_datatype_func. Extract just the type + "thread_session_state_array" and do a text search for it in the runtime specific source. + You should find: +``` +EP_RT_DEFINE_ARRAY (thread_session_state_array, ep_rt_thread_session_state_array_t, ep_rt_thread_session_state_array_iterator_t, EventPipeThreadSessionState *) +EP_RT_DEFINE_LOCAL_ARRAY (thread_session_state_array, ep_rt_thread_session_state_array_t, ep_rt_thread_session_state_array_iterator_t, EventPipeThreadSessionState *) +``` +2. EP_RT_DEFINE_ARRAY and EP_RT_DEFINE_LOCAL_ARRAY are defined: +``` +#define EP_RT_DEFINE_ARRAY(array_name, array_type, iterator_type, item_type) \ + EP_RT_DEFINE_ARRAY_PREFIX(ep, array_name, array_type, iterator_type, item_type) +#define EP_RT_DEFINE_LOCAL_ARRAY(array_name, array_type, iterator_type, item_type) \ + EP_RT_DEFINE_LOCAL_ARRAY_PREFIX(ep, array_name, array_type, iterator_type, item_type) +``` +3. Searching for these ARRAY_PREFIX and LOCAL_ARRAY_PREFIX macros we find that LOCAL_ARRAY_PREFIX defined the +init method (where the method name is constructed by the EP_RT_BUILD_TYPE_FUNC_NAME macro) +``` +#define EP_RT_DEFINE_LOCAL_ARRAY_PREFIX(prefix_name, array_name, array_type, iterator_type, item_type) \ + static inline void EP_RT_BUILD_TYPE_FUNC_NAME(prefix_name, array_name, init) (array_type *ep_array) { \ + STATIC_CONTRACT_NOTHROW; \ + } \ +``` +In this case the init() method did nothing other than provide a placeholder for the static contract.