Skip to content

Add enable_private_path_for_google_cloud_services field to google_sql_database_instance resource#6986

Merged
roaks3 merged 27 commits into
GoogleCloudPlatform:mainfrom
mmizutani:cloud-sql-private-route-for-google-services
Feb 6, 2023
Merged

Add enable_private_path_for_google_cloud_services field to google_sql_database_instance resource#6986
roaks3 merged 27 commits into
GoogleCloudPlatform:mainfrom
mmizutani:cloud-sql-private-route-for-google-services

Conversation

@mmizutani

@mmizutani mmizutani commented Dec 18, 2022

Copy link
Copy Markdown
Contributor

This PR adds a new field enable_private_path_for_google_cloud_services (settings.ip_configuration.enable_private_path_for_google_cloud_services) to the google_sql_database_instance resource.

The Cloud SQL for MySQL and PostgreSQL now supports a new IP configuration option, enablePrivatePathForGoogleCloudServices.

You can now allow other Google Cloud services such as BigQuery to access data in Cloud SQL for MySQL and make queries against this data over a private connection.

https://cloud.google.com/sql/docs/release-notes#December_15_2022

POST https://sqladmin.googleapis.com/v1/projects/project-id/instances

# request JSON body:
{
  "name": "instance-id",
  ...,
  "settings": {
    ...,
    "ipConfiguration": {
      "ipv4Enabled": false,
      "privateNetwork": "projects/project-id/global/networks/vpc-name",
      "allocatedIpRange": "allocated-ip-range"
      "authorizedNetworks": [authorized-networks],
      
      "enablePrivatePathForGoogleCloudServices": true 🆕 
      
    }
  }
}

Before this new GA feature was introduced, we needed to enable a public IP for Cloud SQL instances for BigQuery Data Transfer Service to reach Cloud SQL. This is no longer needed.

The Golang google.golang.org/api/sqladmin/v1 SDK supports the IpConfiguration.EnablePrivatePathForGoogleCloudServices field since v0.104.0.

type IpConfiguration struct {
        ...
	// EnablePrivatePathForGoogleCloudServices: Controls connectivity to
	// private IP instances from Google services, such as BigQuery.
	EnablePrivatePathForGoogleCloudServices [bool](https://pkg.go.dev/builtin#bool) `json:"enablePrivatePathForGoogleCloudServices,omitempty"`
}

https://pkg.go.dev/google.golang.org/api@v0.104.0/sqladmin/v1#IpConfiguration

The version of the Go library google.golang.org/api that this repository currently uses is v0.102.0. So this PR is blocked by PR #6962, which upgrades the library to v0.105.0.

If this PR is for Terraform, I acknowledge that I have:

  • Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
  • Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
  • Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
  • Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
  • Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

sql: added `enable_private_path_for_google_cloud_services` field to `google_sql_database_instance` resource

@modular-magician

Copy link
Copy Markdown
Collaborator

Hello! I am a robot who works on Magic Modules PRs.

I've detected that you're a community contributor. @roaks3, a repository maintainer, has been assigned to assist you and help review your changes.

❓ First time contributing? Click here for more details

Your assigned reviewer will help review your code by:

  • Ensuring it's backwards compatible, covers common error cases, etc.
  • Summarizing the change into a user-facing changelog note.
  • Passes tests, either our "VCR" suite, a set of presubmit tests, or with manual test runs.

You can help make sure that review is quick by running local tests and ensuring they're passing in between each push you make to your PR's branch. Also, try to leave a comment with each push you make, as pushes generally don't generate emails.

If your reviewer doesn't get back to you within a week after your most recent change, please feel free to leave a comment on the issue asking them to take a look! In the absence of a dedicated review dashboard most maintainers manage their pending reviews through email, and those will sometimes get lost in their inbox.


@mmizutani mmizutani force-pushed the cloud-sql-private-route-for-google-services branch from c9bfae4 to b7a602a Compare December 18, 2022 02:43
@modular-magician

Copy link
Copy Markdown
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 3 files changed, 89 insertions(+), 11 deletions(-))
Terraform Beta: Diff ( 3 files changed, 89 insertions(+), 11 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

@modular-magician

Copy link
Copy Markdown
Collaborator

Tests analytics

Total tests: 0
Passed tests 0
Skipped tests: 0
Failed tests: 0

Errors occurred during REPLAYING mode. Please fix them to complete your PR
View the build log

@modular-magician

Copy link
Copy Markdown
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 3 files changed, 89 insertions(+), 11 deletions(-))
Terraform Beta: Diff ( 3 files changed, 89 insertions(+), 11 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

@modular-magician

Copy link
Copy Markdown
Collaborator

Tests analytics

Total tests: 0
Passed tests 0
Skipped tests: 0
Failed tests: 0

Errors occurred during REPLAYING mode. Please fix them to complete your PR
View the build log

@modular-magician

Copy link
Copy Markdown
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 3 files changed, 89 insertions(+), 11 deletions(-))
Terraform Beta: Diff ( 3 files changed, 89 insertions(+), 11 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

@modular-magician

Copy link
Copy Markdown
Collaborator

Tests analytics

Total tests: 0
Passed tests 0
Skipped tests: 0
Failed tests: 0

Errors occurred during REPLAYING mode. Please fix them to complete your PR
View the build log

@modular-magician

Copy link
Copy Markdown
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 3 files changed, 89 insertions(+), 11 deletions(-))
Terraform Beta: Diff ( 3 files changed, 89 insertions(+), 11 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

@mmizutani mmizutani marked this pull request as ready for review January 7, 2023 12:12
@modular-magician

Copy link
Copy Markdown
Collaborator

Tests analytics

Total tests: 2394
Passed tests 2123
Skipped tests: 252
Failed tests: 19

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests
TestAccSqlDatabaseInstance_settings_basic|TestAccSqlDatabaseInstance_settings_secondary|TestAccSqlDatabaseInstance_basicMSSQL|TestAccSqlDatabaseInstance_basic_with_user_labels|TestAccFirebaserulesRelease_BasicRelease|TestAccContainerCluster_withInvalidGatewayApiConfigChannel|TestAccComposerEnvironment_UpdateComposerV2|TestAccComposerEnvironment_ComposerV2|TestAccSqlDatabaseInstance_updateReadReplicaWithBinaryLogEnabled|TestAccSqlDatabaseInstance_Timezone|TestAccSqlDatabaseInstance_ActiveDirectory|TestAccSqlDatabaseInstance_withPrivateNetwork_withoutAllocatedIpRange|TestAccSqlDatabaseInstance_mysqlMajorVersionUpgrade|TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabled|TestAccSqlDatabaseInstance_authNets|TestAccSqlDatabaseInstance_settingsDowngrade|TestAccSqlDatabaseInstance_settings_upgrade|TestAccSqlDatabaseInstance_settings_deletionProtection|TestAccSqlDatabaseInstance_settings_deletionProtectionEnabled

@mmizutani mmizutani marked this pull request as draft January 7, 2023 13:02
@modular-magician

Copy link
Copy Markdown
Collaborator

Tests passed during RECORDING mode:
TestAccSqlDatabaseInstance_settings_basic[Debug log]
TestAccSqlDatabaseInstance_settings_secondary[Debug log]
TestAccSqlDatabaseInstance_basicMSSQL[Debug log]
TestAccSqlDatabaseInstance_basic_with_user_labels[Debug log]
TestAccFirebaserulesRelease_BasicRelease[Debug log]
TestAccContainerCluster_withInvalidGatewayApiConfigChannel[Debug log]
TestAccComposerEnvironment_UpdateComposerV2[Debug log]
TestAccComposerEnvironment_ComposerV2[Debug log]
TestAccSqlDatabaseInstance_updateReadReplicaWithBinaryLogEnabled[Debug log]
TestAccSqlDatabaseInstance_withPrivateNetwork_withoutAllocatedIpRange[Debug log]
TestAccSqlDatabaseInstance_mysqlMajorVersionUpgrade[Debug log]
TestAccSqlDatabaseInstance_authNets[Debug log]
TestAccSqlDatabaseInstance_settingsDowngrade[Debug log]
TestAccSqlDatabaseInstance_settings_upgrade[Debug log]
TestAccSqlDatabaseInstance_settings_deletionProtection[Debug log]
TestAccSqlDatabaseInstance_settings_deletionProtectionEnabled[Debug log]

Tests failed during RECORDING mode:
TestAccSqlDatabaseInstance_Timezone[Error message] [Debug log]
TestAccSqlDatabaseInstance_ActiveDirectory[Error message] [Debug log]
TestAccSqlDatabaseInstance_PointInTimeRecoveryEnabled[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

@mmizutani mmizutani force-pushed the cloud-sql-private-route-for-google-services branch from 2297c75 to 5cc8402 Compare January 8, 2023 08:06
@modular-magician

Copy link
Copy Markdown
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 3 files changed, 100 insertions(+), 11 deletions(-))
Terraform Beta: Diff ( 3 files changed, 100 insertions(+), 11 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

@mmizutani mmizutani force-pushed the cloud-sql-private-route-for-google-services branch from 5cc8402 to a86ce99 Compare January 8, 2023 09:08
@modular-magician

Copy link
Copy Markdown
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 3 files changed, 101 insertions(+), 11 deletions(-))
Terraform Beta: Diff ( 3 files changed, 101 insertions(+), 11 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

@modular-magician

Copy link
Copy Markdown
Collaborator

Tests analytics

Total tests: 2452
Passed tests 2192
Skipped tests: 256
Failed tests: 4

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests
TestAccRegionInstanceGroupManager_stateful|TestAccComputeForwardingRule_update|TestAccFrameworkProviderMeta_setModuleName|TestAccDataSourceDnsRecordSet_basic

@modular-magician

Copy link
Copy Markdown
Collaborator

Tests passed during RECORDING mode:
TestAccComputeForwardingRule_update[Debug log]
TestAccFrameworkProviderMeta_setModuleName[Debug log]
TestAccDataSourceDnsRecordSet_basic[Debug log]

Tests failed during RECORDING mode:
TestAccRegionInstanceGroupManager_stateful[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

…PathForGoogleCloudServices into an existing test case step
…ad request error for invalid combination of parameters returned from the SQL Admin API
@modular-magician

Copy link
Copy Markdown
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 3 files changed, 52 insertions(+), 15 deletions(-))
Terraform Beta: Diff ( 3 files changed, 52 insertions(+), 15 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

@mmizutani mmizutani force-pushed the cloud-sql-private-route-for-google-services branch from e24f1d4 to db229bf Compare February 3, 2023 05:22
@modular-magician

Copy link
Copy Markdown
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 3 files changed, 52 insertions(+), 15 deletions(-))
Terraform Beta: Diff ( 3 files changed, 52 insertions(+), 15 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

@modular-magician

Copy link
Copy Markdown
Collaborator

Tests analytics

Total tests: 0
Passed tests 0
Skipped tests: 0
Failed tests: 0

Errors occurred during REPLAYING mode. Please fix them to complete your PR
View the build log

@mmizutani mmizutani force-pushed the cloud-sql-private-route-for-google-services branch from db229bf to 3fd6f1d Compare February 4, 2023 05:40
@modular-magician

Copy link
Copy Markdown
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 3 files changed, 52 insertions(+), 15 deletions(-))
Terraform Beta: Diff ( 3 files changed, 52 insertions(+), 15 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

@modular-magician

Copy link
Copy Markdown
Collaborator

Tests analytics

Total tests: 2453
Passed tests 2192
Skipped tests: 255
Failed tests: 6

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests
TestAccFirebaserulesRelease_BasicRelease|TestAccRegionInstanceGroupManager_stateful|TestAccApigeeAddonsConfig_apigeeAddonsTestExample|TestAccSqlDatabaseInstance_withPrivateNetwork_withoutAllocatedIpRange|TestAccFrameworkProviderMeta_setModuleName|TestAccDataSourceDnsRecordSet_basic

@modular-magician

Copy link
Copy Markdown
Collaborator

Tests passed during RECORDING mode:
TestAccFirebaserulesRelease_BasicRelease[Debug log]
TestAccApigeeAddonsConfig_apigeeAddonsTestExample[Debug log]
TestAccSqlDatabaseInstance_withPrivateNetwork_withoutAllocatedIpRange[Debug log]
TestAccFrameworkProviderMeta_setModuleName[Debug log]
TestAccDataSourceDnsRecordSet_basic[Debug log]

Tests failed during RECORDING mode:
TestAccRegionInstanceGroupManager_stateful[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

@roaks3 roaks3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I just tweaked the release note to be sql instead of cloudsql

@mmizutani

Copy link
Copy Markdown
Contributor Author

Thank you.

@mmizutani mmizutani deleted the cloud-sql-private-route-for-google-services branch February 6, 2023 19:49
kubalaguna pushed a commit to kubalaguna/magic-modules that referenced this pull request Feb 27, 2023
…sql_database_instance` resource (GoogleCloudPlatform#6986)

* Add `enable_private_path_for_google_cloud_services` property to `google_sql_database_instance`

* Add a test case for third-party tests

* Add usage examples

* Connect with the VCR test

* Fix test

* Add note

* Make fail-safe

* format

* Upgrade the tier of POSTGRES instances for faster instance creation

* Fix type error

* Also upgrade mysql tier to non shared core in another tiimeout sensitive flaky test

* Revert "Also upgrade mysql tier to non shared core in another timeout sensitive flaky test"

This reverts commit 9c2cb79.

* Revert "Upgrade the tier of POSTGRES instances for faster instance creation"

This reverts commit 52e8a4a.

* Merge TestAccSqlDatabaseInstance_withPrivateNetwork_withEnablePrivatePathForGoogleCloudServices into an existing test case step

* Remove the custom pre-apply validation logic and let users face the bad request error for invalid combination of parameters returned from the SQL Admin API

* Remove an extra blank line

* Revert unnecessary edit to the no-longer-used go template file

* Fix indent

* Revert unnecessary edit to mmv1/products/sql/api.yaml
ericayyliu pushed a commit to ericayyliu/magic-modules that referenced this pull request Jul 26, 2023
…sql_database_instance` resource (GoogleCloudPlatform#6986)

* Add `enable_private_path_for_google_cloud_services` property to `google_sql_database_instance`

* Add a test case for third-party tests

* Add usage examples

* Connect with the VCR test

* Fix test

* Add note

* Make fail-safe

* format

* Upgrade the tier of POSTGRES instances for faster instance creation

* Fix type error

* Also upgrade mysql tier to non shared core in another tiimeout sensitive flaky test

* Revert "Also upgrade mysql tier to non shared core in another timeout sensitive flaky test"

This reverts commit 9c2cb79.

* Revert "Upgrade the tier of POSTGRES instances for faster instance creation"

This reverts commit 52e8a4a.

* Merge TestAccSqlDatabaseInstance_withPrivateNetwork_withEnablePrivatePathForGoogleCloudServices into an existing test case step

* Remove the custom pre-apply validation logic and let users face the bad request error for invalid combination of parameters returned from the SQL Admin API

* Remove an extra blank line

* Revert unnecessary edit to the no-longer-used go template file

* Fix indent

* Revert unnecessary edit to mmv1/products/sql/api.yaml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants