Skip generation of dependencies for on-disk assets - #639
Conversation
openshift-ci-robot
added
size/L
@crawford The behavior when an asset is dirty and is in the state file hasn't changed. If any of the parents of an asset are dirty, then the asset is generated, regardless of whether the asset is in the state file. If the asset is dirty by virtue of being on-disk but none of its parents are dirty, then the on-disk source is used, regardless of whether the asset is in the state file. |
|
@staebler i think we can improve the |
@abhinavdahiya Yeah, I've had that lingering thought in the back of my head. The |
|
8c174a8 we shouldn't need the global variables, but i think we can revisit testing later on... |
@abhinavdahiya Perhaps I'm just too in the weeds to see how it can be done without defining a type for each individual test case. |
|
this looks good. can you squash the commits to more appropriate number so that its merge ready. |
@abhinavdahiya Your last review comment has been addressed. And the commits have been squashed. |
…e of its dependencies are dirty This allows the user to supply and use an on-disk asset (such as install-config.yml) without the need to also supply the state file that was created. This is helpful when re-using an on-disk asset for multiple installations. In particular, hive would like to run openshift-install with a supplied install-config.yml and no state file. To effect this behavior, the asset store loads all of the on-disk assets that a fetched asset depends upon prior to fetching the dependencies for the fetched asset. From this, the asset store can determine whether the fetched asset is dirty or not. If the fetched asset is not dirty and is on-disk or in the state file, then the asset is used as is without generating any of the dependent assets--as they would be ignored in resolving the fetched asset anyway. Conflicts can occur when the asset store resolves in different ways the fetch of two assets that share a dependency. For example, let us say that there are two assets, A and B, that both depend upon asset C. Asset A is present on disk, and asset B is not present on disk. When the asset store fetchs asset A, then asset C will not be generated. However, when the asset store fetches asset B, then asset C will be generated in order to generate asset B. Asset A could potentially have data that conflicts with the data that would have been taken from the asset C that was generated. The new load function creates new asset instances to store the asset state loaded from on-disk and from the state file. The store tests were relying on the same asset instance being used throughout the test. Unfortunately, the tests now need to use a lot of global variables, making the tests more fragile. The assetToPurge field was removed since the same information can be obtained by iterating over the assets map. Also, the parameter passed to the purge function was changed to a single Asset instead of an Asset slice since the function is only ever called with a single Asset. Fixes #545
|
The changes look great. Thanks! 😇 |
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: abhinavdahiya, staebler The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
4 participants
This allows the user to supply and use an on-disk asset (such as install-config.yml) without
need to also supply the state file that was created. This is helpful when re-using an on-disk
asset for multiple installations. In particular, hive would like to run openshift-install
with a supplied install-config.yml and no state file.
To effect this behavior, the asset store loads all of the on-disk assets that a fetched asset
depends upon prior to fetching the dependencies for the fetched asset. From this, the asset
store can determine whether the fetched asset is dirty or not. If the fetched asset is not
dirty and is on-disk, then the on-disk asset is used as is without generating any of the
dependent assets--as they would be ignored in resolving the fetched asset anyway.
Conflicts can occur when the asset store resolves in different ways the fetch of two assets
that share a dependency. For example, let us say that there are two assets, A and B, that both
depend upon asset C. Asset A is present on disk, and asset B is not present on disk. When the
asset store fetchs asset A, then asset C will not be generated. However, when the asset store
fetches asset B, then asset C will be generated in order to generate asset B. Asset A could
potentially have data that conflicts with the data that would have been taken from the asset
C that was generated.
Fixes #545