add secondary key dictionaries

[hendrikmuhs]

Secondary key dictionaries match a set of key before the real matching. Those secondary keys can be arbitrary strings, e.g. a user, account or a tenant id.

At compile time, a list of secondary keys must be provided at construction, e.g. ["region", "account_id", "user_id"]. The order defines matching order. For every entry as well as for every match operation an unordered map of keys and values must be provided :

{
    "account_id": "xyz", 
    "region": "eu-west",
    "user_id": "abcd"
}

Example use:

dictionary.complete("sie", {"account_id": "xyz", "region": "eu-west", "user_id": "abcd" })

With other words: APIs are equal to ordinary dictionaries with the extension of a map with secondary keys next to the primary key.

Implementation details

Before doing the real matching, the root state is altered according to the additional secondary key(s) by matching a prefix. Given the "moved" start state, re-use the existing match functionality from Dictionary. This is implemented by wrapping an ordinary dictionary (most code changes are just refactorings to provide the altered start state).

The SecondaryKeyDictionaryCompiler is similar, it wraps the existing DictionaryCompiler. In addition it:

• alters the lookup key by prefixing the secondary keys
• the list of secondary keys are stored as property
• The secondary key values are stored as dictionary which gets concatenated to the normal dictionary

To efficiently store secondary key values, values get replaced by a short representation. This helps e.g. for cases where secondary key values are long, e.g. UUIDs. Equally to the primary dictionary the replacement dict uses memory mapping.

Python bindings

Unfortunately the python bindings required a lot of boiler plate code. In theory it could be auto-generated, however I wonder how much effort we should put into the autowrap generated code. I am playing with the idea to migrate to pybind11.