Surface the distinction between an explicit "null" value and an absent value

When using a schema that has a field from a reference that is both non-required and "nullable" this is a common way of defining it in OpenAPI: 3.1.0

UpdateTaskOccurrenceCompletionResponse:
      properties:
        old_task:
          $ref: '#/components/schemas/TaskOccurenceDTO'
        next_task:
          oneOf:
            - $ref: '#/components/schemas/TaskOccurenceDTO'
            - type: 'null'
      type: object
      required:
        - old_task

but this currently generates the following API:

/// - Remark: Generated from `#/components/schemas/UpdateTaskOccurrenceCompletionResponse`.
        public struct UpdateTaskOccurrenceCompletionResponse: Codable, Hashable, Sendable {
            /// - Remark: Generated from `#/components/schemas/UpdateTaskOccurrenceCompletionResponse/old_task`.
            public var old_task: Components.Schemas.TaskOccurenceDTO
            /// - Remark: Generated from `#/components/schemas/UpdateTaskOccurrenceCompletionResponse/next_task`.
            @frozen public enum next_taskPayload: Codable, Hashable, Sendable {
                /// - Remark: Generated from `#/components/schemas/UpdateTaskOccurrenceCompletionResponse/next_task/case1`.
                case TaskOccurenceDTO(Components.Schemas.TaskOccurenceDTO)
                /// - Remark: Generated from `#/components/schemas/UpdateTaskOccurrenceCompletionResponse/next_task/case2`.
                case case2(OpenAPIRuntime.OpenAPIValueContainer)
            }
            /// - Remark: Generated from `#/components/schemas/UpdateTaskOccurrenceCompletionResponse/next_task`.
            public var next_task: Components.Schemas.UpdateTaskOccurrenceCompletionResponse.next_taskPayload

This is quite confusing syntax at the callsite, since we have this weird case2, instead of just having only an optional TaskOccurenceDTO. Is this expected behaviour or could we do something to improve the ergonomics before the 1.0.0 release? I'll happily work on a PR, if you could point me in the right direction to where in the codebase the relevant parts are!

@madsodgaard

Hi @madsodgaard,

what's the meaning of marking the value as nullable beyond just defining as

old_task:
  $ref: '#/components/schemas/TaskOccurenceDTO'

and not putting it in the required list?

How would you want to see it represented in Swift, if not as a simple optional property of TaskOccurenceDTO?

@czechboy0

@czechboy0 One place where I use this pattern a lot is when designing PATCH endpoints that allows partial updates of models. If I have field name that can actually be NULL in the database, I would give that the type:

type:
  - string
  - "null"

since this is a partial update endpoint, I want to differentiate between the user providing a "null" value (thus updating the value of name to NULL) and not providing a value at all (not updating name). That's why I don't include it in the required. From what I understand required means that the property must be present, regardless of the value. As you said I could just not include it in the required and remove the null type. But then I would only be doing that because I know I am going to use this library to generate a client, and I'll get a nicer API.

The ideal situtation, in my mind, is that the library could somehow recognize this pattern and just provide an optional property of TaskOccurenceDTO, as it does currently when not specifying the property in the required list and with only one type. Hope that makes some sense:)

Swift OpenAPI Generator delegates JSON parsing to automatic Codable implementations (wherever possible) and to Foundation's JSONDecoder, which doesn't make the distinction between a null value and an absent value, so unfortunately what you're trying to do won't work. Several pieces of the stack, including those not owned by this tool (such as Foundation itself), would have to change.

You'll need to make resetting values somehow an explicit part of the API, not relying on the distinction between null and an absent value. And for values you're not trying to overwrite, just define them as regular properties and omit them from the required array.

@czechboy0

@czechboy0 Thanks! Of course the fact that Codable does not differentiate between absent values and null is quite challenge to this issue. But, this pattern is so common in APIs today, that perhaps we should find a way to support? By "this pattern", I am referring to PATCH endpoints for partial updates instead of PUT endpoints to replace the entire resource. I have used this property wrapper previously in server-side applications. Do you think such a type or something similar would have a place in the library? (or is it too niché?) It would be used in places where you have a non-required nullable field.

@propertyWrapper
struct PotentiallyAbsent<T: Codable>: Codable {
    var wrappedValue: T? {
        switch state {
        case .present(let value):
            return value
        default:
            return nil
        }
    }
    
    enum State {
        case absent
        case null
        case present(T)
    }
    
    var projectedValue: PotentiallyAbsent<T> {
        return self
    }
    
    var state: State
    
    init(_ state: State) {
        self.state = state
    }
    
    static var null: PotentiallyAbsent<T> {
        .init(.null)
    }
    
    static var absent: PotentiallyAbsent<T> {
        .init(.absent)
    }
    
    static func present(_ value: T) -> PotentiallyAbsent<T> {
        .init(.present(value))
    }
    
    init(from decoder: Decoder) throws {
        fatalError("Attempted to decode `PotentiallyAbsent` as a standalone value")
    }
    
    func encode(to encoder: Encoder) throws {
        fatalError("Attempted to encode `PotentiallyAbsent` as a standalone value")
    }
}

extension KeyedDecodingContainer {
    func decode<T>(_ type: PotentiallyAbsent<T>.Type, forKey key: K) throws -> PotentiallyAbsent<T> {
        do {
            if try self.decodeNil(forKey: key) {
                return .init(.null)
            } else {
                return .init(.present(try decode(T.self, forKey: key)))
            }
        } catch DecodingError.keyNotFound {
            return .init(.absent)
        }
    }
}

extension KeyedEncodingContainer {
    mutating func encode<T>(_ value: PotentiallyAbsent<T>, forKey key: KeyedEncodingContainer<K>.Key) throws {
        switch value.state {
        case .absent:
            break
        case .null:
            try encodeNil(forKey: key)
        case let .present(value):
            try encode(value, forKey: key)
        }
    }
}

I don't think it's likely for us to try to solve this in Swift OpenAPI Generator. The fix should be in Codable and JSONDecoder. Once it's there, we can see what'd remain for Swift OpenAPI Generator to have to do to support this case.

But only doing it here would be a significant amount of work that introduces a lot of complexity.

One way this could be better represented that'd work today would be a PATCH call that takes two values, a "toSet" dictionary, which has keys matching the property to set, and the value to set. And if a value is nil for a property, the property isn't changed.

And the second value would be called "toReset", which would just be a list of property names whose values to reset to their default.

Renamed the issue to reflect that this is a missing feature. See my comment above explaining that this goes deeper in the stack we're based on, so would likely need some improvements from JSONDecoder first, but keeping open to allow tracking progress.

I ran into this (or a similar) issue. The encoded output of my server omits a field with a nil value, but some code consuming this service expects an explicit null value.

One idea is this property wrapper: https://stackoverflow.com/a/53142581/67280

I think it'd be best to first help Foundation get the desired behavior, and then Swift OpenAPI Generator and all other projects relying on JSONDecoder get it for free.

https://github.com/apple/swift-foundation

I understand, but the tradeoff isn't symmetric. Users consuming documents with inline schemas in PATCH requests will be forced to do a significant refactor of the document (cf. just add an annotation).

I'm not suggestion we only do the extension and not config—my original suggestion upthread was to consider both—just that if we're really gonna pick one, I would suggest picking the one that solves the problem for all cases.

I'm sympathetic to folks preferring the config file where that is an option to them: is it significantly more work to offer both?

Or, alternatively, is it possible to identify an inline schema from the config file, by some JSON Path syntax? That way we could do only config but still support inline schema PATCH requests.

Yeah maybe, here's the OpenAPIKit impl of it: https://github.com/mattpolzin/OpenAPIKit/blob/fd27b297993923c9a1725dc62553d06579c7a33e/Sources/OpenAPIKit30/JSONReference.swift#L193

But since this is growing in scope, I might not be able to get to this as quickly as I hoped, just a heads up. If anyone else wants to investigate this, go ahead.

@czechboy0

Thanks for the continued discussion here!

@czechboy0 what was the reason for the shift in preference to do this in the config rather than the document considering the option to do OpenAPI Overlays?

Btw, here's the implementation of the extension in OpenAPITools.

I think in the fullness of time we could support all of these, just that the quickest and easiest to implement is the config approach + named schemas only. That'll only require changes in a handful of methods in the codegen logic, and historically adopters often preferred a codegen option over having to edit their OpenAPI doc.

The OpenAPI doc + overlay, and inline schema support are both great enhancements, they just also add quite a lot of scope both in generator implementation and documentation enhancements to show it all working end-to-end.

I would love if there was at least one way to achieve this and would hope that the approach can then be expanded over time to support the full extent of use cases. Right now there's no way to achieve this and no workaround has been found.

One alternative we discussed a little, is to solve this in a way that uses additive API without the need to opt-in for specific operations or schemas, which would hopefully make it more widely useful, more discoverable, and use a unified implementation under the hood.

The idea would be to add a side-array (likely an OptionSet) to each generated type for schemas with at least one nullable property, with one option per nullable property, matching the generated name of the property. For example:

S:
  type: object
  required: [a, b]
  properties:
    a:
      type: string
      nullable: true
    b:
      type: string
      nullable: false

struct S: Codable, Hashable, Sendable {
    var a: Swift.String?
    var b: Swift.String

    var __swiftOpenAPISetProperties: [...]  // <-- new: populated during decoding

    init(
        a: Swift.String? = nil,
        b: Swift.String,
        explicitlySerializing: [...] = .empty  // <-- new: influences the encoding
    )
}

This would require us to stop relying on the synthesized Codable conformance or to provide a wrapped JSON encoder and decoder.

I'm curious about feedback here as it seems to be a more general solution to the problem of surfacing and influencing explicitly null properties.

I like the flexibility and built in discoverability this solution provides!

We'd probably only want this functionality exposed to PATCH endpoints but otherwise looks good to me 🫡

We'd probably only want this functionality exploded to PATCH endpoints but otherwise looks good to me 🫡

This would be on the schema, but I suppose we could only apply it to schemas that are used in at least one PATCH request. Are there any other scenarios where distinguishing explicitly- vs implicitly-nil is useful?

Personally, I'd prefer for this to be opt-in more explicitly. Searching all operations and identifying which schemas end up being (transitively) used by PATCH operations could result in unexpected spooky action at a distance.

@czechboy0

For simplicity, perhaps doesn't matter to search if the schemas is used in at least one PATCH endpoint and can just generally apply to all schemas that have at least one nullable field.

Not seeing any other scenarios where this distinction can be beneficial but also don't see anything spooky with applying to all schemas with a nullable field.

@czechboy0 thoughts if we apply as originally suggested? Still think it should be more explicit? If so, which of the more explicit solutions are you leaning towards?

For simplicity, perhaps doesn't matter to search if the schemas is used in at least one PATCH endpoint and can just generally apply to all schemas that have at least one nullable field.

This was my original suggestion in #419 (comment). If we managed to achieve this with a custom Encoder/Decoder instead of having to provide a custom Codable conformance then it shouldn't drastically affect the generated code size either.

Yep, totally good with the original suggestion!

A different feature landed that offers a temporary workaround for the lack of proper PATCH support here - you can hand-write the request body to correctly encode the difference between null and absent, and then use that type instead of the generated one: https://swiftpackageindex.com/apple/swift-openapi-generator/main/documentation/swift-openapi-generator/configuring-the-generator#Type-overrides

Navigation Menu

Repository navigation

Surface the distinction between an explicit "null" value and an absent value #419

18 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Surface the distinction between an explicit "null" value and an absent value #419

Description

Activity

czechboy0 commented on Dec 3, 2023

madsodgaard commented on Dec 3, 2023

czechboy0 commented on Dec 3, 2023

madsodgaard commented on Dec 10, 2023

czechboy0 commented on Dec 10, 2023

czechboy0 commented on Dec 11, 2023

brandonbloom commented on Feb 19, 2024

czechboy0 commented on Feb 19, 2024

18 remaining items

simonjbeaumont commented on Mar 21, 2025

simonjbeaumont commented on Mar 21, 2025

czechboy0 commented on Mar 21, 2025

nikitaame commented on Mar 22, 2025

czechboy0 commented on Mar 22, 2025

jpsim commented on Mar 25, 2025

simonjbeaumont commented on Mar 27, 2025

nikitaame commented on Mar 27, 2025

simonjbeaumont commented on Mar 28, 2025

czechboy0 commented on Mar 28, 2025

nikitaame commented on Mar 28, 2025

simonjbeaumont commented on Mar 28, 2025

nikitaame commented on Mar 28, 2025

czechboy0 commented on Jun 18, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions