Changed the type of automatic field and added support for customer_managed_encryption in automatic field

[abheda-crest]

Changed the type of the automatic field from TypeBool to TypeList and added support for the replication.automatic.customer_managed_encryption field in the google_secret_manager_secret resource.

fixes hashicorp/terraform-provider-google#10679
fixes hashicorp/terraform-provider-google#9272
fixes hashicorp/terraform-provider-google#14282

If this PR is for Terraform, I acknowledge that I have:

• Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
• Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
• Generated Terraform providers, and ran make test and make lint in the generated providers to ensure it passes unit and linter tests.
• Ran relevant acceptance tests using my own Google Cloud project and credentials (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
• Read Write release notes before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

secretmanager: changed the type of `automatic` field from TypeBool to TypeList

secretmanager: added `replication.automatic.customer_managed_encryption` field to `google_secret_manager_secret` resource

[modular-magician]

Hello! I am a robot. It looks like you are a: Community Contributor Googler Core Contributor. Tests will require approval to run.

@slevenick, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

[abheda-crest]

@slevenick While generating the providers after changing the automatic field to block type in the terraform configs (i.e tests and examples), I encountered that for the documentation for the resources google_dataform_repository and google_dataform_repository_release_config, the automatic field remains boolean i.e automatic = true even though I've changed them in the examples dataform_repository.tf.erb and dataform_repository_release_config.tf.erb respectively. I am facing this issue while generating the GA provider and not the Beta provider. I suspect this is something related to the documentation being generated for the GA provider even though these resources are beta only.

[abheda-crest]

@slevenick Could you please review this PR?

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Breaking Change(s) Detected

The following breaking change(s) were detected within your pull request.

• Field replication.automatic changed from TypeBool to TypeList on google_secret_manager_secret - reference

If you believe this detection to be incorrect please raise the concern with your reviewer. If you intend to make this change you will need to wait for a major release window. An override-breaking-change label can be added to allow merging.

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 25 files changed, 558 insertions(+), 48 deletions(-))
Terraform Beta: Diff ( 28 files changed, 562 insertions(+), 52 deletions(-))
TF Conversion: Diff ( 1 file changed, 42 insertions(+), 3 deletions(-))
TF OiCS: Diff ( 15 files changed, 133 insertions(+), 11 deletions(-))

[modular-magician]

Tests analytics

Total tests: 2948
Passed tests 2625
Skipped tests: 307
Affected tests: 16

Action taken

Found 16 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccDataSourceSecretManagerSecret_basic|TestAccCloudBuildTrigger_cloudbuildTriggerWebhookConfigExample|TestAccCloudfunctions2function_cloudfunctions2SecretVolumeExample|TestAccCloudfunctions2function_cloudfunctions2SecretEnvExample|TestAccCloudBuildTrigger_webhook_config|TestAccSecretManagerSecretIamMemberGenerated|TestAccSecretManagerSecretIamBindingGenerated|TestAccCloudFunctionsFunction_secretMount|TestAccCloudFunctionsFunction_secretEnvVar|TestAccSecretManagerSecretIamPolicyGenerated|TestAccSecretManagerSecret_import|TestAccSecretManagerSecret_secretWithAutomaticCmekExample|TestAccSecretManagerSecret_cmek|TestAccSecretManagerSecret_secretConfigBasicExample|TestAccSecretManagerSecret_automaticCmekUpdate|TestAccSecretManagerSecret_annotationsUpdate

Get to know how VCR tests work

[modular-magician]

$\textcolor{green}{\textsf{Tests passed during RECORDING mode:}}$
TestAccDataSourceSecretManagerSecret_basic[Debug log]
TestAccCloudBuildTrigger_cloudbuildTriggerWebhookConfigExample[Debug log]
TestAccCloudfunctions2function_cloudfunctions2SecretVolumeExample[Debug log]
TestAccCloudfunctions2function_cloudfunctions2SecretEnvExample[Debug log]
TestAccCloudBuildTrigger_webhook_config[Debug log]
TestAccSecretManagerSecretIamMemberGenerated[Debug log]
TestAccSecretManagerSecretIamBindingGenerated[Debug log]
TestAccCloudFunctionsFunction_secretMount[Debug log]
TestAccCloudFunctionsFunction_secretEnvVar[Debug log]
TestAccSecretManagerSecretIamPolicyGenerated[Debug log]
TestAccSecretManagerSecret_import[Debug log]
TestAccSecretManagerSecret_secretWithAutomaticCmekExample[Debug log]
TestAccSecretManagerSecret_cmek[Debug log]
TestAccSecretManagerSecret_secretConfigBasicExample[Debug log]
TestAccSecretManagerSecret_automaticCmekUpdate[Debug log]
TestAccSecretManagerSecret_annotationsUpdate[Debug log]

Rerun these tests in REPLAYING mode to catch issues

$\textcolor{green}{\textsf{No issues found for passed tests after REPLAYING rerun.}}$

---

$\textcolor{green}{\textsf{All tests passed!}}$
View the build log or the debug log for each test

[slevenick]

I need to check if this is an acceptable change.

My understanding is that we need to support an upgrade path through the last 4.x version to 5.0.0, which may not be possible with this PR.

[slevenick]

See #8721

I don't think this is an acceptable change to make as implemented here.

The alternative that I can see would be changing the name of the field so that we can support the side-by-side of the old field and the new field in the same version of 4.x, and then removing the old field in 5.0.0

[gptSanyam]

Where is the kms_key and keyring defined?
For automatic replication location is global and CMEK should also be defined in a global keyring.

[abheda-crest]

I've used the function BootstrapKMSKey as mentioned in the Secret.yaml that will use a particular keyring and a particular key i.e. tftest-shared-keyring-1 and tftest-shared-key-1 respectively in the test environment. The implementation of this function can be found here: https://github.com/GoogleCloudPlatform/magic-modules/blob/main/mmv1/third_party/terraform/acctest/bootstrap_test_utils.go#L42.

[abheda-crest]

See #8721

I don't think this is an acceptable change to make as implemented here.

The alternative that I can see would be changing the name of the field so that we can support the side-by-side of the old field and the new field in the same version of 4.x, and then removing the old field in 5.0.0

@slevenick Just FYI, the automatic field is showing ForceNew behavior and the new field to be added must also be kept ForceNew as patching the replication field from automatic to user_managed (and vice-versa) throws an error 400 Existing secret has automatic replication, but requested secret does not. Updating secret replication is not supported. Thus, Migrating from automatic field to the new field will recreate the secret.

[slevenick]

See #8721
I don't think this is an acceptable change to make as implemented here.
The alternative that I can see would be changing the name of the field so that we can support the side-by-side of the old field and the new field in the same version of 4.x, and then removing the old field in 5.0.0

@slevenick Just FYI, the automatic field is showing ForceNew behavior and the new field to be added must also be kept ForceNew as patching the replication field from automatic to user_managed (and vice-versa) throws an error 400 Existing secret has automatic replication, but requested secret does not. Updating secret replication is not supported. Thus, Migrating from automatic field to the new field will recreate the secret.

It's definitely hard to do to support two fields side-by-side, but I think it can be done. It might require a complicated CustomizeDiff function in the version where both are supported

[abheda-crest]

See #8721
I don't think this is an acceptable change to make as implemented here.
The alternative that I can see would be changing the name of the field so that we can support the side-by-side of the old field and the new field in the same version of 4.x, and then removing the old field in 5.0.0

@slevenick Just FYI, the automatic field is showing ForceNew behavior and the new field to be added must also be kept ForceNew as patching the replication field from automatic to user_managed (and vice-versa) throws an error 400 Existing secret has automatic replication, but requested secret does not. Updating secret replication is not supported. Thus, Migrating from automatic field to the new field will recreate the secret.

It's definitely hard to do to support two fields side-by-side, but I think it can be done. It might require a complicated CustomizeDiff function in the version where both are supported

@slevenick I am thinking of adding custom_flatten and custom_expand for the replication field to check which of the two fields are provided and to call the respective flatteners and expanders.

[c2thorn]

This was abandoned in favor of #8838