[Feature] Hook System for Binding Generation

[Togira123]

This draft aims to collect more feedback and ideas on a hook system for binding generation. The core idea is that users should be able to extend/modify the way the bindings are generated. The current implementation adds a new command to SCons, called binding_hook_file, which takes a file path to a python file. The specified python file should define a class that extends BindingGeneratorHooks and is called BindingGeneratorHooksExtension. It can then override the methods provided there to modify the way the bindings are generated. As an example, there is custom_generator.py included in this draft (won't be included in the final PR), which adds names of signals as constants for each header file. It can be tested using scons binding_hook_file=custom_generator.py.

The current implementation was already discussed here: https://chat.godotengine.org/channel/gdextension/thread/2gR6bngEu57tWLQG9

[Togira123]

This second approach uses no SCons arguments, as suggested by @Ivorforce
The way this would work is the following:

Users that want to define their own hooks would still do so in a dedicated file, still extending BindingGeneratorHooks, as before. The way they pass it to SCons is they import the file in their own SConstruct file and then add it to the env exports like so:

from custom_generator import CustomBindingGeneratorHooks

and

env = SConscript("godot-cpp/SConstruct", {"env": env, "customs": customs, "binding_hooks": CustomBindingGeneratorHooks()})

This second snippet is similar to the thing that is already present in the template here: https://github.com/godotengine/godot-cpp-template/blob/8283d9632c00ba4654d4e3201f651df01ef51172/SConstruct#L38.
Essentially the user can just add one more export named binding_hooks. The naming is important, because godot-cpp's SConstruct will import the variable (and default to None if it is not supplied). It then adds it to its environment and can access the instance of the class that way.

Below you can find an example of a custom binding hook, this would be placed in the project's root directory so that godot-cpp is a direct subfolder. To work with the snippet above, name it custom_generator.py (it is basically the same code as from the previous commit, I just moved it out of the repo)

import sys

sys.path.insert(0, "godot-cpp")
from tools.binding_generator_hooks import BindingGeneratorHooks


class CustomBindingGeneratorHooks(BindingGeneratorHooks):
    def alter_engine_class_header(self, class_api, lines):
        signals = []
        if "signals" in class_api:
            for signal_api in class_api["signals"]:
                name = signal_api["name"]
                signal_constant = "\tstatic constexpr char SIGNAL_" + name.upper() + '[] = "' + name + '";'
                signals.append(signal_constant)
            try:
                idx = lines.index("public:") + 1
                for signal_const in signals:
                    lines.insert(idx, signal_const)
                    idx += 1
            except ValueError:
                print("no public keyword found, not adding signals")
        return lines

[dsnopek]

Thanks!

Overall, I think this is a pretty good direction.

However, I'm not crazy about adding binding_generator_hooks.py at the top-level - we're always trying to reduce the amount of stuff we have at the top-level. So, if we can have that in a sub-directory, that would be ideal

And, in order for this to be merged, we'll need to:

• Add support for CMake - it'll likely need to be done differently, maybe giving a file name and using a fixed class name, or something like that?
• Discuss the API of BindingGeneratorHooks to ensure it covers everything we want to cover and that everyone is happy with it

[Togira123]

Alright, before I'll continue with implementation two things:

• Do we allow more than one hook class? If yes there are two options: Either we simply pass a list of classes via the exports (so env = SConscript("godot-cpp/SConstruct", {"binding_hooks": [Hook1(), Hook2()], ...})). This would be the easiest for SCons. Depending on how much consistency we want between the two build systems we could also say that the user passes a function that returns the list. For CMake, the function name would be fixed and be called to obtain the class instances.
• Do we allow the hooks to generate new files as well? instead of just altering files that are being generated already. If yes, we'd also have to change _get_file_list to call some function of the hooks class which returns the additional files to make sure _get_file_list's return value includes all files.

[dsnopek]

My personal opinion is:

• No. A hook class that multiplexes multiple hook classes could always be used if the user wants (or we could even provide one). Only using a single hook class internally is sufficient, IMO
• No. Users can already generate their own files via normal means in SCons or CMake - we don't need to provide extra hooks for this

[Togira123]

Moved binding_generator_hooks.py into tools/.

Regarding CMake:
There is a new option available named GODOTCPP_BINDING_HOOK_FILE which takes a filepath to the custom python file. The python file should define a class named CustomBindingGeneratorHooks (if somebody prefers a different name lmk ;)).

Current API of BindingGeneratorHooks:

• alter_engine_class_header: alter all class header files generated in include/godot_cpp/classes.
• alter_engine_class_source: alter all class source files generated in src/classes.
• alter_global_constants: alter the include/godot_cpp/classes/global_constants.hpp file.
• alter_utility_functions_header: alter the include/godot_cpp/variant/utility_functions.hpp file.
• alter_utility_functions_source: alter the src/variant/utility_functions.cpp file.
• alter_builtin_class_header: alter built-in classes generated in include/godot_cpp/variant/.
• alter_builtin_class_source: alter built-in classes generated in src/variant/.

[dsnopek]

Thanks!

This is looking good to me. At this point, this just needs wider discussion (I've added it to the agenda for the next GDExtension team meeting) and some review from folks who know the build system stuff better