Provide a method to return all flag keys

[bfrearson]

Feature description

I'd like to be able to log the current state of all flags for a given evaluation context for debugging. I realise that I could iterate over my known expected flag keys locally, but this is not very scalable, and because my project is modular, there's not a single place that I store all my keys.

I know that the api response includes all of these details, so we just need a method to expose them in the SDK.

Proposed solution

Add a new property FlagsClient.keys that returns all the key Strings for the current evaluation context.

Clients can then iterate over the keys to fetch all values for logging, debugging etc.

Other relevant information

Providing just the keys rather than the values prevents users from misusing the SDK and creating a cache of all values that might go stale.

The datadog dashboard doesn't currently show evaluations from users, which is why we need this method.

[sameerank]

Thanks for sharing this feature request @bfrearson

The datadog dashboard doesn't currently show evaluations from users, which is why we need this method.

Mind verifying that you've tried the following?

• You've checked the "Client Evaluations" chart on your feature flag's detail page? And your version dependency is recent? Evaluation tracking is enabled by default starting with v3.7.0
• You call RUM.enable(...) in your app before Flags.enable(). If enabled, evaluations are also reported in the RUM Feature Flag Tracking tab

Accessing a .keys property in itself is harmless, but the second part "Clients can then iterate over the keys to fetch all values for logging, debugging etc." could contaminate experimentation data and therefore would need to be used with caution.

// Getting all keys is read-only
let keys = flagsClient.keys

// But looping through and evaluating them "for debugging" has side effects
for key in keys {
    let value = flagsClient.getValue(key: key)  // Records exposures and evaluations
}

Can you share what debugging scenarios aren't covered by the existing evaluation tracking to help us assess whether a keys accessor is the right solution?

[bfrearson]

@sameerank Thanks for this! I hadn't realised that client evaluations were supported in 3.7.0+ (I'd pinned the sdk at some exact version a while ago for some reason!)

Seeing client evaluations in the feature flags page does indeed cover most what I needed.

The second part of what I was thinking about was a way to export or snapshot the current set of flags for use with our analytics platform (in this case Mixpanel). This would be helpful when trying to confirm user-reported behaviour. I'm aware that RUM would also give us this insight, but we've had the budget pulled for that since there was a huge price increase.

This might come down to different approaches to flag evaluations and the expectations of how exposures should take place. I appreciate that datadog needs to be able to cater to a wide range of use cases though. In our case, we are strict about not updating an evaluation context during runtime after login, because we don't want unexpected UI hitches to occur if a feature is unexpectedly enabled. So even though we do only evaluate flags at the point they're needed, I expect the underlying context to be static, and so the result of that evaluation to be the same each time. For us, having a single list of these static values would be good for understanding the current state of the app for debugging user-reported issues. However, I do recognise that not all clients take this approach, and therefore it might not make sense to export a list of all keys/values when those values are dynamic.

(Perhaps actually the best practice for us is to evaluate all required flags on login immediately and cache the results of these locally in a static flags manager rather than having the DD SDK do it for us).

Essentially what I'm looking for is the ability to look at a given user (in datadog, Mixpanel, or whereever) and see what flags were set at a given point (either a log in datadog, or a mixpanel event)