Generic Attributes

[mflibby]

Description

Resolves FSLang 965

RFC FS-1143

Intention:

The intent of this PR is to implement Generic Attribute support, e.g.:

type ARecordAttribute<^T>()=
    inherit Attribute()


type AFieldAttribute<^T>()=
    inherit Attribute()

[<ARecord<int>>]
type SomeRecord =
    {
        [<AField<string>>]
        someField : string
    }

Checklist

• Test cases added
• Performance benchmarks added in case of performance changes
• Release notes entry updated:

[github-actions]

❗ Release notes required

---

✅ Found changes and release notes in following paths:

Change path	Release notes path	Description
src/Compiler	docs/release-notes/.FSharp.Compiler.Service/8.0.400.md
LanguageFeatures.fsi	docs/release-notes/.Language/preview.md

[mflibby]

Adding of test cases, and fullfilment of minutia (documentation, various "feature" tags, general cleanup, etc) still WIP.

Currently, [<SomeAttribute<int>()>] compiles down to the expected IL. I plan to explore what will be required for inferencing before marking as ready.

Drafting now while I complete the minor details in case anyone wants to provide feedback in the meantime.

[mflibby]

@dotnet-policy-service agree company="Hillcrest Research and Development LLC"

[abelbraaksma]

Hi @mflibby, this looks very interesting! In your original description, can you link to the reported issue that this is fixing? And if it is a language change (i.e., a new feature that is being added), can you link to the RFC (in https://github.com/fsharp/fslang-design) and the language proposal or discussion chat (https://github.com/fsharp/fslang-suggestions)?

If none exist, can you update the OP with a summary of what this change is doing and why?

[mflibby]

@abelbraaksma I've updated the description with the desired information. Please let me know if this is satisfactory, or if more information will be helpful :)

[mflibby]

I'm not sure why, but commit 64607ef passed all checks, yet after a few cleanup/a minor change/merges to other changes on main, a large number of the checks have been failing (the only substantive change I made since the passing commit was a minor change to the LexFilter that allowed the type arguments immediately adjacent to the close of an attribute list to be properly parsed).

I'm wondering why I am all of a sudden getting a bunch of seemingly unrelated tests failing? Additionally, windows defender seems to be flagging a bunch of tests, and occasionally quarantining innocuous tests like "Conformance/../E_Literals02.fs".

The investigation continues.

[T-Gro]

Could you please add test cases(s) with mutual recursion (either via and or via a rec module) between the attribute declaration, the type argument and the type it is being applied on?

[mflibby]

Do we have a particular point of failure we are looking for here? Off the top of my head this is what comes to mind:
(in a module rec)

type MyAttribute<^T>() =
    inherit Attribute()

[<MyAttribute<AnActualType>>]
let x = 1

type AnActualType = class end

(Note: it appears that in a rec module, the usage of an attribute on a binding needs to occur after the definition, but on a type it can occur before the definition.)
and maybe something like

(in a regular module)

[<MyAttribute<AnActualType>>]
type SomeClass(arg) =
    member _.AGoodProperty = arg

and MyAttribute<^T>() =
       inherit Attribute()

and AnActualType = int

[mflibby]

Oh great call, both rec and and fail right now (made sure to check they pass when there are no type args and both do)

[T-Gro]

Glad I could suggest a point of failure - I did not have a particular one in mind, I just know that rec modules are a common point of failure for new typechecking features.

Maybe also a type applying "itself" as the type argument to the attribute?

[<MyAttribute<SomeClass<SomeClass<SomeClass<MyAttribute>>>>>]
type SomeClass<'a>(arg) =

[mflibby]

I still need to figure out why the rec/and cases failed, but I implemented the self referencing case (great case btw, I know there must be some cases where it will be useful but this in particular made me laugh quite hard), and was surprised to find it working right out of the gate!

[abelbraaksma]

I've updated the description with the desired information. Please let me know if this is satisfactory, or if more information will be helpful :)

Hey @mflibby, excellent, thanks, now we know where it's coming from and that it was approved 👍. We still need an RFC, though. I may be able to help with that. Language changes require an RFC for documentation and consensus. They capture the what, how and scope of any new feature. The go here: https://github.com/fsharp/fslang-design. It is also used to set up a discussion thread to deal with any unresolved questions.

I can help with the process.

[mflibby]

I realize it's generally a "no-no" to do this to obscure functionality (i.e. I dont yet understand what this was for), but it seems that with the introduction of generic attributes, we will need to fully qualify the environment for attribute usage. Since this function was only used in the generation of attributes, my current position is that this function can be deleted - though I am still curious why the count was being maxed for the non-generic case, if anyone has that wisdom on hand.

[KevinRansom]

Perhaps this:

https://github.com/dotnet/fsharp/blob/main/src/Compiler/AbstractIL/ilwrite.fs#L905

[vzarytovskii]

The problem is we try to maintain the metadata compatibility with the older compilers, and currently there's no preferred way to add new data to it without breaking existing compilers.

@vzarytovskii Do you know if there been any changes in this direction?

Rule of thumb is usually:

1. Don't break it if possible ^(duh). Problem with this is that we don't really have a mechanism of merging pickled data with IL metadata (in this case - merging FSharpAttribute with ILAttribute) as well as not all the information we would like to have, is there (like aliases, or type measures for example), which, as a result will get lost cross-assembly.
2. If breaking change is necessary - make it explicit - i.e. only pickle new data when library author explicitly opted in to use that feature, and put the feature under language flag and don't use it in FSharp.Core (for example - let bindings in types, or, in this case, generic attributes).

We have one more case - when we introduced another resource for nullability data, it has multiple problems unfortunately:

1. We don't have standard mechanisms for managing those, code for handling every single one has to be literally copy-pasted, which kinda makes it prone to errors.
2. It turns out to be suboptimal - a lot of additional excessive data - a bunch of new tables, instead of just new bit/type somewhere.
3. It needs to be taken care of in the trimming config files every time it's introduced.

We probably want to think of such mechanism in future, but it will require a very thorough design and will be a huge chunk of work.

I would assume that the purpose of the pickled resources is for the sake of performance improvements (correct me if I'm wrong)i n which case I would think that there are some kind of benchmarks concerning the gain of using pickling as opposed to loading the same resources from the IL?

Not necessary, main purpose is to have/store additional metadata, which we want to use cross-assembly (mostly). I don't really think it will affect perf significatly.

Assuming that the cost in this case wouldn't be great, and that it is feasible, loading the info from the IL and not breaking backward compatibility sounds like the better option.

Yeah, but please, refer to part of this comment below - not all the info we would like to have is in the IL metadata.

TL;DR - I would personally go with second point in the beginning of my comment if possible. We have done it before. We will likely do it in future. And at the moment we, unfortunately, don't have a better mechanism for it.

cc @T-Gro @dsyme

[KevinRansom]

I would like to see more interop tests,

• using C# Generic Attributes from F#
• using F# Generic Attributes from C# and VB - I suppose VB supports them.
• tests using reflection to access the attribute values, Although I can't imagine much problem with that.
• tests from C# using reflection to access the attributes.
• test in FSI

• Declare and use an attrribute when scripting.
  

• Consume C# and F# dlls with GenericAttributes from scripts
  

There probably needs to be work to ensure that the IDE experience of Generic Attributes works OK, but I think that can be a separate PR.

Needs a language flag so at least we have about a year to work through the kinks.

Needs a much more detailed writeup of what's enabled, why, and what the interop experience is supposed to be.

This looks like a good start, thanks for taking care of it.

[psfinaki]

We'll convert this to draft for now, due to low activity. @mflibby ping us if you need help :)

[xperiandri]

It would be nice to finally have them

[mflibby]

We'll convert this to draft for now, due to low activity. @mflibby ping us if you need help :)

Yeah my apologies for letting this drift a little, I have a project at work that really ramped up at the end of May so this fell by the wayside. I fully intend on returning to this and busting it out the rest of the way starting January. I will definitely take you up on that offer as I know that there are several things that are confusing me in terms of both the build process and general path-forward development things.
Here's what I know I'm missing right now in case someone can leave some crumbs for me for when I return:

1. How does one "opt in" to using a feature? Per @vzarytovskii's comment in June, is this something that just happens via the use of that feature? I've used let bindings in types but never otherwise had to "opt-in" to that feature.
2. Is there a particular location within the tests where VB/C# tests are placed? I imagine there must be a precedent already, but I haven't touched any of that yet.
3. At some point, running the tests in this branch suddenly started creating a massive diff for files that - to my understanding - shouldn't have been getting effected, e.g. some of the IL baselines were being generated slightly differently. In hindsight, I suspect this may be due to allowing the version of the net9 pre-release I was using to get a little stale (hopefully). Any wisdom on this matter will be helpful, but I will explore this more next month.
4. @KevinRansom would a detailed writeup belong in the feature request?

I look forward to getting back to this :)

[psfinaki]

@mflibby no need to apologize, we all get overwhelmed by stuff sometimes :)

Happy you're going to be back in a game! On that note, seeing that it's your first PR in F#, I would like to suggest you joining the Amplifying F# initiative. Folks in AF# can be happy and fast to answer all (or most of) the questions you've put here.

Moreover, recently we've started doing community PR reviews and I thought this one can be also a good case for such a session - generic attributes are a rather hard yet interesting topic and this can be a great learning exercise for the community (and maintainers as well, me for sure :D)

@edgarfgp et al will guide through the process then.

[vzarytovskii]

• How does one "opt in" to using a feature? Per @vzarytovskii's comment in June, is this something that just happens via the use of that feature? I've used let bindings in types but never otherwise had to "opt-in" to that feature.

Just adding a feature for language version preview, and then using standard check whether this feature is available should do it, so when SDK is updated, default language version too, so it is enabled for new versions of language, but not old ones. Here's an example how to add and add it: https://github.com/dotnet/fsharp/pull/18098/files#diff-e13d713f6cdc23cdaa222873de7a0eebda890ac2d0b9fd2374566ec79fe5e06f, and here how to use it: https://github.com/dotnet/fsharp/pull/18098/files#diff-33f19e1cff6e1df09e36df423722449340f685c5c08f041543334c173cc259f8 (see all the calls to g.langVersion.SupportsFeature LanguageFeature.SupportValueOptionsAsOptionalParameters)

• Is there a particular location within the tests where VB/C# tests are placed? I imagine there must be a precedent already, but I haven't touched any of that yet.

Anywhere under tests/FSharp.Compiler.Service/Interop/ should do, here are some examples: https://github.com/dotnet/fsharp/blob/main/tests/FSharp.Compiler.ComponentTests/Interop/RequiredAndInitOnlyProperties.fs, I don't think we ever done VB tests, I'm not sure they are needed here.

• At some point, running the tests in this branch suddenly started creating a massive diff for files that - to my understanding - shouldn't have been getting effected, e.g. some of the IL baselines were being generated slightly differently. In hindsight, I suspect this may be due to allowing the version of the net9 pre-release I was using to get a little stale (hopefully). Any wisdom on this matter will be helpful, but I will explore this more next month.

One important thing to remember, is that you want to run tests (not debug them) in the Release configuration, since it's what we care codegen-wise the most, I think this is what you were facing. Otherwise, we can solve it as it comes.

• @KevinRansom would a detailed writeup belong in the feature request?

Not Kevin, but I think either here or in the suggestion/RFC would be a good place for it to be.