Fix issue #12883

[etruong42]

Fixed hashicorp/terraform-provider-google#12883

Make one CloudResourceManager call per monitored_project terraform resource rather than per (monitored_project * "sibling" projects)

If this PR is for Terraform, I acknowledge that I have:

• Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
• Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
• Generated Terraform providers, and ran make test and make lint in the generated providers to ensure it passes unit and linter tests.
• Ran relevant acceptance tests using my own Google Cloud project and credentials (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
• Read Write release notes before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

monitoring: fixed scaling issues when deploying terraform changes with many `google_monitoring_monitored_project`

[modular-magician]

Oops! It looks like you're using an unknown release-note type in your changelog entries:

• bug fixed scaling issues when deploying terraform changes with many google_monitoring_monitored_projects

Please only use the types listed in https://github.com/GoogleCloudPlatform/magic-modules/blob/master/.ci/RELEASE_NOTES_GUIDE.md.

[modular-magician]

Hello! I am a robot. It looks like you are a: Community Contributor Googler Core Contributor. Tests will require approval to run.

@hao-nan-li, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 1 file changed, 27 insertions(+), 86 deletions(-))
Terraform Beta: Diff ( 1 file changed, 27 insertions(+), 86 deletions(-))

[hao-nan-li]

Looks like there are some build errors.
https://googlecloudplatform.github.io/magic-modules/develop/run-tests/ might help.

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 1 file changed, 28 insertions(+), 86 deletions(-))
Terraform Beta: Diff ( 1 file changed, 28 insertions(+), 86 deletions(-))

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 1 file changed, 28 insertions(+), 86 deletions(-))
Terraform Beta: Diff ( 1 file changed, 28 insertions(+), 86 deletions(-))

[modular-magician]

Tests analytics

Total tests: 2987
Passed tests 2683
Skipped tests: 296
Affected tests: 8

Action taken

Found 8 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccBigtableAppProfile_bigtableAppProfileAnyclusterExample|TestAccBigtableAppProfile_bigtableAppProfileMulticlusterExample|TestAccMonitoringMonitoredProject_monitoringMonitoredProjectBasicExample|TestAccMonitoringMonitoredProject_projectNumShortForm|TestAccMonitoringMonitoredProject_projectNumLongForm|TestAccMonitoringMonitoredProject_monitoringMonitoredProjectLongFormExample|TestAccDataSourceGoogleServiceAccountIdToken_impersonation|TestAccFolderIamPolicy_basic

Get to know how VCR tests work

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccBigtableAppProfile_bigtableAppProfileAnyclusterExample[Debug log]
TestAccBigtableAppProfile_bigtableAppProfileMulticlusterExample[Debug log]
TestAccDataSourceGoogleServiceAccountIdToken_impersonation[Debug log]
TestAccFolderIamPolicy_basic[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

---

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccMonitoringMonitoredProject_monitoringMonitoredProjectBasicExample[Error message] [Debug log]
TestAccMonitoringMonitoredProject_projectNumShortForm[Error message] [Debug log]
TestAccMonitoringMonitoredProject_projectNumLongForm[Error message] [Debug log]
TestAccMonitoringMonitoredProject_monitoringMonitoredProjectLongFormExample[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 1 file changed, 31 insertions(+), 85 deletions(-))
Terraform Beta: Diff ( 1 file changed, 31 insertions(+), 85 deletions(-))

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 36 insertions(+), 85 deletions(-))
Terraform Beta: Diff ( 1 file changed, 31 insertions(+), 85 deletions(-))

[modular-magician]

Tests analytics

Total tests: 2991
Passed tests 2689
Skipped tests: 296
Affected tests: 6

Action taken

Found 6 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccMonitoringMonitoredProject_monitoringMonitoredProjectBasicExample|TestAccMonitoringMonitoredProject_projectNumShortForm|TestAccMonitoringMonitoredProject_projectNumLongForm|TestAccMonitoringMonitoredProject_monitoringMonitoredProjectLongFormExample|TestAccDataSourceGoogleServiceAccountAccessToken_basic|TestAccDataSourceGoogleServiceAccountJwt

Get to know how VCR tests work

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccDataSourceGoogleServiceAccountAccessToken_basic[Debug log]
TestAccDataSourceGoogleServiceAccountJwt[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{red}{\textsf{Tests failed when rerunning REPLAYING mode:}}$
TestAccDataSourceGoogleServiceAccountJwt[Error message] [Debug log]

Tests failed due to non-determinism or randomness when the VCR replayed the response after the HTTP request was made.

Please fix these to complete your PR. If you believe these test failures to be incorrect or unrelated to your change, or if you have any questions, please raise the concern with your reviewer.

---

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccMonitoringMonitoredProject_monitoringMonitoredProjectBasicExample[Error message] [Debug log]
TestAccMonitoringMonitoredProject_projectNumShortForm[Error message] [Debug log]
TestAccMonitoringMonitoredProject_projectNumLongForm[Error message] [Debug log]
TestAccMonitoringMonitoredProject_monitoringMonitoredProjectLongFormExample[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 1 file changed, 35 insertions(+), 87 deletions(-))
Terraform Beta: Diff ( 1 file changed, 35 insertions(+), 87 deletions(-))

[modular-magician]

Tests analytics

Total tests: 3008
Passed tests 2706
Skipped tests: 296
Affected tests: 6

Action taken

Found 6 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccDataprocClusterIamPolicy|TestAccDataprocJobIamPolicy|TestAccMonitoringMonitoredProject_monitoringMonitoredProjectBasicExample|TestAccMonitoringMonitoredProject_projectNumLongForm|TestAccMonitoringMonitoredProject_projectNumShortForm|TestAccMonitoringMonitoredProject_monitoringMonitoredProjectLongFormExample

Get to know how VCR tests work

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccDataprocClusterIamPolicy[Debug log]
TestAccDataprocJobIamPolicy[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

---

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccMonitoringMonitoredProject_monitoringMonitoredProjectBasicExample[Error message] [Debug log]
TestAccMonitoringMonitoredProject_projectNumLongForm[Error message] [Debug log]
TestAccMonitoringMonitoredProject_projectNumShortForm[Error message] [Debug log]
TestAccMonitoringMonitoredProject_monitoringMonitoredProjectLongFormExample[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 1 file changed, 39 insertions(+), 87 deletions(-))
Terraform Beta: Diff ( 1 file changed, 39 insertions(+), 87 deletions(-))
TF Conversion: Diff ( 1 file changed, 5 insertions(+))

[modular-magician]

Tests analytics

Total tests: 3026
Passed tests 2724
Skipped tests: 297
Affected tests: 5

Action taken

Found 5 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccDataprocClusterIamPolicy|TestAccMonitoringMonitoredProject_monitoringMonitoredProjectBasicExample|TestAccMonitoringMonitoredProject_projectNumLongForm|TestAccMonitoringMonitoredProject_projectNumShortForm|TestAccMonitoringMonitoredProject_monitoringMonitoredProjectLongFormExample

Get to know how VCR tests work

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccDataprocClusterIamPolicy[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

---

$\textcolor{red}{\textsf{Tests failed during RECORDING mode:}}$
TestAccMonitoringMonitoredProject_monitoringMonitoredProjectBasicExample[Error message] [Debug log]
TestAccMonitoringMonitoredProject_projectNumLongForm[Error message] [Debug log]
TestAccMonitoringMonitoredProject_projectNumShortForm[Error message] [Debug log]
TestAccMonitoringMonitoredProject_monitoringMonitoredProjectLongFormExample[Error message] [Debug log]

$\textcolor{red}{\textsf{Please fix these to complete your PR.}}$
View the build log or the debug log for each test

[hao-nan-li]

Line 18 in this file. Delete url to v1/{{name}}

[etruong42]

Thanks! Is there an explanation for why this change is now necessary?

For posterity, here's the (private) conversation that I think led to this recommendation:

I think the problem comes down to whatever is decoding the deletion. In the latest CI run of TestAccMonitoringMonitoredProject_monitoringMonitoredProjectBasicExample, the deletion request is

DELETE /v1/locations/global/metricsScopes/ci-test-project-188019/projects/locations/global/metricsScopes/1067888929963/projects/102753631080

and we can see the path has the format v1/locations/global/metricsScopes/<project>/projects/<metric_scope_path>

when we just want DELETE /v1/<metric_scope_path> which would translate to DELETE /v1/locations/global/metricsScopes/1067888929963/projects/10275363108. I added debug lines to the decoder, and the decoder seems to decode the link just fine.

This leads me to suspect some spooky complexity from deleting the nested_query functionality, in which case I think I definitely need more support in debugging

[hao-nan-li]

It's mainly due to to deleting the nested_query. I don't know why it was implemented with nested_query before. Let the CI runs complete and see if the proposed solution solved the problem for all failing tests.

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 41 insertions(+), 89 deletions(-))
Terraform Beta: Diff ( 2 files changed, 41 insertions(+), 89 deletions(-))
TF Conversion: Diff ( 1 file changed, 5 insertions(+))

[modular-magician]

Tests analytics

Total tests: 3035
Passed tests 2734
Skipped tests: 297
Affected tests: 4

Action taken

Found 4 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccMonitoringMonitoredProject_monitoringMonitoredProjectBasicExample|TestAccMonitoringMonitoredProject_projectNumLongForm|TestAccMonitoringMonitoredProject_monitoringMonitoredProjectLongFormExample|TestAccMonitoringMonitoredProject_projectNumShortForm

Get to know how VCR tests work

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccMonitoringMonitoredProject_monitoringMonitoredProjectBasicExample[Debug log]
TestAccMonitoringMonitoredProject_projectNumLongForm[Debug log]
TestAccMonitoringMonitoredProject_monitoringMonitoredProjectLongFormExample[Debug log]
TestAccMonitoringMonitoredProject_projectNumShortForm[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

---

$\textcolor{green}{\textsf{All tests passed!}}$
View the build log or the debug log for each test