Skip to content

3482 Add schema verification for metadata#3865

Merged
Nic-Ma merged 34 commits into
Project-MONAI:devfrom
Nic-Ma:3482-add-schema-verify
Mar 14, 2022
Merged

3482 Add schema verification for metadata#3865
Nic-Ma merged 34 commits into
Project-MONAI:devfrom
Nic-Ma:3482-add-schema-verify

Conversation

@Nic-Ma

@Nic-Ma Nic-Ma commented Feb 28, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor

Task step 5 of #3482 .

Description

This PR added the utility to download schema and verify the metadata in a manifest.

Status

Ready

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

@Nic-Ma Nic-Ma mentioned this pull request Feb 28, 2022
Closed
17 tasks
@Nic-Ma
[DLMED] complete temp test schema
09aa7ef 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma

Nic-Ma commented Feb 28, 2022

Copy link
Copy Markdown
Contributor Author

Steps that generated the schema:

  1. Generate a draft schema with online tool:
    https://www.liquid-technologies.com/online-json-to-schema-converter
  2. Modify unnecessary or incorrect items in the schema.
  3. Add descriptions to the main items.

Thanks.

@Nic-Ma
[DLMED] add cmd line API
67bd705 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
[DLMED] fix typo
c72088f 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
[DLMED] add more tests
9ed45df 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
[DLMED] add optional import
2cf9d3a 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
[DLMED] test with download link
d3801ee 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma Nic-Ma mentioned this pull request Feb 28, 2022
Merged
7 tasks
@Nic-Ma
[DLMED] enhance doc-string
b6cd1db 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma

Nic-Ma commented Feb 28, 2022

Copy link
Copy Markdown
Contributor Author

It depends on PR #3832.

Thanks.

@Nic-Ma
[DLMED] fix mypy
c77888a 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma

Nic-Ma commented Mar 1, 2022

Copy link
Copy Markdown
Contributor Author

Hi @ericspod @wyli @rijobro ,

Due to the Blossom CI issue, we can't merge any PRs recently, this PR depends on #3832 (marked the dependency logic with FIXME in the code) but it's almost ready for review. Could you please help review this draft PR first?

Thanks in advance.

@Nic-Ma Nic-Ma requested review from ericspod, rijobro and wyli March 1, 2022 07:59
ericspod
ericspod reviewed Mar 1, 2022
View reviewed changes
Comment thread monai/apps/manifest/utils.py Outdated
ericspod
ericspod reviewed Mar 1, 2022
View reviewed changes
Comment thread monai/apps/manifest/utils.py Outdated
ericspod
ericspod reviewed Mar 1, 2022
View reviewed changes
Comment thread monai/apps/manifest/utils.py Outdated
ericspod
ericspod reviewed Mar 1, 2022
View reviewed changes

@ericspod ericspod left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it looks good to me, the DS_Store files need to be removed, and some pattern modified in .gitignore so that they don't show up again. We should discuss if adding the jsonschema dependency is a problem but I think it's fine as an optional.

@Nic-Ma Nic-Ma mentioned this pull request Mar 2, 2022
Merged
7 tasks
Nic-Ma added 3 commits March 7, 2022 12:27
@Nic-Ma
[DLMED] remove DS_Store
eccc3ab 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
Merge branch 'dev' of https://github.com/Project-MONAI/MONAI into 348…
f37ee56 
…2-add-schema-verify

Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
[DLMED] update according to comments
140b8b6 
Signed-off-by: Nic Ma <nma@nvidia.com>
ericspod
ericspod reviewed Mar 9, 2022
View reviewed changes
Comment thread tests/test_bundle_verify_meta.py Outdated
@Nic-Ma Nic-Ma requested a review from wyli March 11, 2022 17:05
@Nic-Ma

Nic-Ma commented Mar 11, 2022

Copy link
Copy Markdown
Contributor Author

/black

@Nic-Ma

Nic-Ma commented Mar 11, 2022

Copy link
Copy Markdown
Contributor Author

/build

@Nic-Ma

Nic-Ma commented Mar 12, 2022

Copy link
Copy Markdown
Contributor Author

/build

1 similar comment
@Nic-Ma

Nic-Ma commented Mar 12, 2022

Copy link
Copy Markdown
Contributor Author

/build

@Nic-Ma Nic-Ma force-pushed the 3482-add-schema-verify branch from bd35493 to 9be9daa Compare March 13, 2022 02:13
@Nic-Ma

Nic-Ma commented Mar 13, 2022

Copy link
Copy Markdown
Contributor Author

/black

@Nic-Ma

Nic-Ma commented Mar 13, 2022

Copy link
Copy Markdown
Contributor Author

/build

@Nic-Ma
[DLMED] skip windows
caf3421 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma

Nic-Ma commented Mar 13, 2022

Copy link
Copy Markdown
Contributor Author

/black

@Nic-Ma

Nic-Ma commented Mar 13, 2022

Copy link
Copy Markdown
Contributor Author

/build

@Nic-Ma
[DLMED] fix typo
0ce58d9 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma

Nic-Ma commented Mar 14, 2022

Copy link
Copy Markdown
Contributor Author

/black

@Nic-Ma

Nic-Ma commented Mar 14, 2022

Copy link
Copy Markdown
Contributor Author

/build

wyli
wyli approved these changes Mar 14, 2022
View reviewed changes

@wyli wyli left a comment •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, looks good, could you move the schema.json to the testing data repository. Cc @ericspod in case there are more comments

wyli
wyli reviewed Mar 14, 2022
View reviewed changes
Comment thread monai/bundle/scripts.py Outdated
Nic-Ma added 2 commits March 14, 2022 20:10
@Nic-Ma
[DLMED] optimize error log
e7e1d3e 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma
[DLMED] update according to comments
7e4c506 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma Nic-Ma force-pushed the 3482-add-schema-verify branch from 0b182fa to 7e4c506 Compare March 14, 2022 13:21
@Nic-Ma

Nic-Ma commented Mar 14, 2022

Copy link
Copy Markdown
Contributor Author

/black

@Nic-Ma

Nic-Ma commented Mar 14, 2022

Copy link
Copy Markdown
Contributor Author

/build

1 similar comment
@Nic-Ma

Nic-Ma commented Mar 14, 2022

Copy link
Copy Markdown
Contributor Author

/build

@Nic-Ma
[DLMED] update accoding to comments
f80d9a8 
Signed-off-by: Nic Ma <nma@nvidia.com>
@Nic-Ma

Nic-Ma commented Mar 14, 2022

Copy link
Copy Markdown
Contributor Author

/black

@Nic-Ma

Nic-Ma commented Mar 14, 2022

Copy link
Copy Markdown
Contributor Author

/build

@Nic-Ma Nic-Ma enabled auto-merge (squash) March 14, 2022 15:30
@Nic-Ma Nic-Ma merged commit f15ff93 into Project-MONAI:dev Mar 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wyli wyli wyli approved these changes
@rijobro rijobro Awaiting requested review from rijobro
@ericspod ericspod Awaiting requested review from ericspod ericspod is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Nic-Ma @wyli @ericspod