Ignore unrecognised codelist xml in gen.py (version-1.05) - #509
Merged
Conversation
extrasaction='ignore' applies only to the csv writer, so the namespaced
`extras` elements added in IATI-Codelists-NonEmbedded#427 still reached
the json output as fully-qualified keys, e.g.
"{https://namespaces.iatistandard.org/codelist_extras}crs".
Filtering in codelist_item_todict covers both writers, so the csv and
json output agree. The elements remain in the xml output, consistent with
activation-date and withdrawal-date, which are also xml-only.
extrasaction='ignore' is kept, so unexpected non-namespaced elements
cannot break the csv writer either.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
gen.sh had no error handling around its python steps and no `set -e`, so it returned the exit status of its last command. When gen.py crashed the script still exited 0, and callers' `./gen.sh || exit 1` guards never tripped. CI's "Build and test" step passed, and IATI-Reference-Generator produced a release with codelists.json and codelists.xml missing and truncated csv output. Guarding the python steps makes that failure visible. The git, cp and rm steps are left unguarded rather than using `set -o errexit`, to avoid changing behaviour around the optional IATI-Codelists-NonEmbedded pull. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Author
|
1.05 is deprecated |
Contributor
|
We've realised that the dashboard will break without this fix, so we've re-opened the PR. |
Bjwebb
approved these changes
Aug 4, 2026
Bjwebb
left a comment
Bjwebb
left a comment
Contributor
There was a problem hiding this comment.
I've tested this branch with the dashboard unit tests, and it now works.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replicates #505 for
version-1.05.IATI/IATI-Codelists-NonEmbedded#427 added namespaced
extraselements (crs,tossd), which crashgen.py's csv writer. As @Bjwebb noted on #505, this needs doing for each version of the standard that we build.The two
gen.pycommits are cherry-picked cleanly from #505. Thegen.shcommit was applied by hand, becausegen.shon this branch has a different structure to 2.x — it converts the non-embedded xml withv3tov2.pyon the way in and has an extrav2tov3.pystep, so there are five python steps to guard rather than four.Verified on this branch
Ran
./gen.shwith IATI-Codelists-NonEmbedded atmaster:codelists.jsonandcodelists.xmlpresent (both were missing from the last IATI-Reference-Generator release)FinanceType.csvback to 88 rows / 6 columnsextras:crs/extras:tossdstill present in the xml output