Skip to content

Fix Variables API handling of non-string JSON values - #71018

Open
kaxil wants to merge 1 commit into
apache:mainfrom
astronomer:fix-variables-api-non-string-values
Open

Fix Variables API handling of non-string JSON values#71018
kaxil wants to merge 1 commit into
apache:mainfrom
astronomer:fix-variables-api-non-string-values

Conversation

@kaxil

@kaxil kaxil commented Aug 3, 2026

Copy link
Copy Markdown
Member

Fixes #71010
Closes #71015

Problem

VariableBody.value is typed JsonValue, so the Variables REST API accepts any JSON type, but everything downstream assumed a string:

  • POST /api/v2/variables with "value": ["a", "b"] returned 201 but stored the Python repr ['a', 'b']. That is not valid JSON, so the variable silently breaks the next Variable.get(key, deserialize_json=True).
  • PATCH /api/v2/variables/{key} with the same payload failed with a masked 500: the raw list reached the Fernet encryption step, which raised TypeError: encoding without a string argument.
  • Bulk update failed the same way, and one non-string entity took down the whole request. Bulk create already JSON-encoded dicts and lists, but stored booleans and nulls as "True"/"None".

Solution

VariableBody now JSON-encodes non-string values once, at request validation, so POST, PATCH and both bulk actions behave identically: strings are stored verbatim, everything else is stored as JSON (indent=2, byte-identical to the existing bulk-create format) and round-trips through deserialize_json=True. The now-redundant serialize_json special case in bulk create is removed, and Variable.set_val raises a clear TypeError pointing at serialize_json=True instead of the cryptic encoding error.

Request Before After
POST "value": ["a", "b"] 201, stores ['a', 'b'], unreadable as JSON 201, stores ["a", "b"]
PATCH "value": ["a", "b"] 500 200
Bulk update with a dict value Whole request 500 200
POST or bulk "value": true / null Stores "True" / "None" Stores true / null
PATCH "value": null 200 but silently kept the old value 200, stores null

String values, including JSON passed as a string, are stored byte-for-byte as before, and the OpenAPI schema is unchanged, so generated clients are unaffected.

Why coerce instead of rejecting with 422

Non-string values are an intentional part of the API contract since #49844: the UI Import Variables flow sends raw parsed JSON (ImportVariablesForm.tsx#L71), the docs describe bulk-uploading variables as a JSON file, and airflow variables import applies the identical encode-iff-not-a-string rule (variable_command.py#L165). Tightening value back to str would 422 all of those.

Notes

POST /api/v2/variables with a non-string JSON value (array, object,
bool, null) silently stored the Python repr of the value, which cannot
be read back with deserialize_json. PATCH with the same payload failed
with a masked 500 because the raw value hit the ORM Fernet encryption
step.

VariableBody now JSON-encodes non-string values once at request
validation, covering POST, PATCH and bulk consistently, matching the
behaviour the bulk create path already had for dicts and lists.
Variable.set_val also raises a clear TypeError instead of the cryptic
"encoding without a string argument".
@boring-cyborg boring-cyborg Bot added the area:API Airflow's REST/HTTP API label Aug 3, 2026
@kaxil
kaxil marked this pull request as ready for review August 3, 2026 20:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Variables API: non-string value returns masked 500 on PATCH and silently corrupts data on POST

1 participant