Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
a705911
Properly handle ABP midnight (hours = 0)
GoogleMarcfont Jul 11, 2023
145121a
Add ExpectNonEmptyPlan to test
GoogleMarcfont Jul 13, 2023
01bdf49
Add continuous backup config to alloydb cluster
GoogleMarcfont Jul 14, 2023
1698b7b
Properly handle ABP midnight (hours = 0)
GoogleMarcfont Jul 14, 2023
a0a2a68
add more unit tests
GoogleMarcfont Jul 18, 2023
cd32e8d
Merge branch 'marcfont/cbr' of https://github.com/GoogleMarcfont/magi…
GoogleMarcfont Jul 18, 2023
e886c86
Adding more tests
GoogleMarcfont Jul 20, 2023
50611a2
Merge branch 'main' into marcfont/cbr
GoogleMarcfont Jul 20, 2023
22dcd5c
Fix cluster.yaml lint
GoogleMarcfont Jul 20, 2023
6910f0f
Add default value and update cmek test
GoogleMarcfont Jul 31, 2023
ce93ae5
Merge branch 'main' into marcfont/cbr
GoogleMarcfont Jul 31, 2023
9c2350c
Update test to use kms key bootstrapping util
GoogleMarcfont Jul 31, 2023
95b98e6
condense update tests
GoogleMarcfont Jul 31, 2023
8c6cbba
Add restore fields to cluster
GoogleMarcfont Aug 1, 2023
ef517d6
ongoing
GoogleMarcfont Aug 3, 2023
f37f890
Restore API working
GoogleMarcfont Aug 3, 2023
8c47937
make fields immutable and ignored on read
GoogleMarcfont Aug 3, 2023
5fa7584
Add required_with field
GoogleMarcfont Aug 4, 2023
43565fd
working E2E test for restoring backup and PITR
GoogleMarcfont Aug 7, 2023
a06326f
Added tests for invalid use cases
GoogleMarcfont Aug 7, 2023
3c49109
Added example for how to use cluster restore
GoogleMarcfont Aug 7, 2023
a0a9c6e
Merge branch 'main' into marcfont/restore
GoogleMarcfont Aug 7, 2023
ed9730b
fix bad merge
GoogleMarcfont Aug 7, 2023
81dd01f
remove test comment in favor of inline commenting
GoogleMarcfont Aug 7, 2023
b6c5c8f
fix cluster restore example
GoogleMarcfont Aug 8, 2023
bf6af4b
remove trailing spaces
GoogleMarcfont Aug 8, 2023
f6866e2
Skip example test
GoogleMarcfont Aug 10, 2023
b05962b
change point in time in example to a date
GoogleMarcfont Aug 10, 2023
aa2192c
Merge branch 'marcfont/restore' of https://github.com/GoogleMarcfont/…
GoogleMarcfont Aug 10, 2023
4262154
Merge branch 'main' into marcfont/restore
GoogleMarcfont Aug 16, 2023
8f109a4
move test
GoogleMarcfont Aug 16, 2023
e967f0f
Merge branch 'main' into marcfont/restore
GoogleMarcfont Aug 16, 2023
0344f20
rename network name in restore test
GoogleMarcfont Aug 16, 2023
944b4bb
Update cluster.yaml to use same structure as API
GoogleMarcfont Aug 17, 2023
f9a4379
rename backup and continuous backup source
GoogleMarcfont Aug 18, 2023
22d6b07
Apply suggestions from code review
shuyama1 Aug 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions mmv1/products/alloydb/Cluster.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,24 @@ examples:
primary_resource_id: 'full'
vars:
alloydb_cluster_name: 'alloydb-cluster-full'
- !ruby/object:Provider::Terraform::Examples
name: 'alloydb_cluster_restore'
primary_resource_id: 'source'
Comment thread
GoogleMarcfont marked this conversation as resolved.
skip_test: true
vars:
alloydb_cluster_name: 'alloydb-source-cluster'
alloydb_backup_restored_cluster_name: 'alloydb-backup-restored'
alloydb_pitr_restored_cluster_name: 'alloydb-pitr-restored'
alloydb_backup_id: 'alloydb-backup'
alloydb_instance_name: 'alloydb-instance'
network_name: 'alloydb-network'
test_vars_overrides:
network_name: 'acctest.BootstrapSharedTestNetwork(t, "alloydb-instance-basic")'
ignore_read_extra:
- 'reconciling'
- 'update_time'
custom_code: !ruby/object:Provider::Terraform::CustomCode
pre_create: templates/terraform/pre_create/alloydb_restore_cluster.go.erb
parameters:
- !ruby/object:Api::Type::String
name: 'clusterId'
Expand Down Expand Up @@ -193,6 +211,42 @@ properties:
The initial password for the user.
required: true
sensitive: true
- !ruby/object:Api::Type::NestedObject
name: 'restoreBackupSource'
ignore_read: true
immutable: true
conflicts:
- restore_continuous_backup_source
description: |
The source when restoring from a backup. Conflicts with 'restore_continuous_backup_source', both can't be set together.
properties:
- !ruby/object:Api::Type::String
name: 'backupName'
required: true
immutable: true
description: |
The name of the backup that this cluster is restored from.
- !ruby/object:Api::Type::NestedObject
name: 'restoreContinuousBackupSource'
ignore_read: true
immutable: true
conflicts:
- restore_backup_source
description: |
The source when restoring via point in time recovery (PITR). Conflicts with 'restore_backup_source', both can't be set together.
properties:
- !ruby/object:Api::Type::String
name: 'cluster'
required: true
immutable: true
description: |
The name of the source cluster that this cluster is restored from.
- !ruby/object:Api::Type::String
name: 'pointInTime'
required: true
immutable: true
description: |
The point in time that this cluster is restored to, in RFC 3339 format.
- !ruby/object:Api::Type::NestedObject
name: 'continuousBackupConfig'
default_from_api: true
Expand Down
69 changes: 69 additions & 0 deletions mmv1/templates/terraform/examples/alloydb_cluster_restore.tf.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
resource "google_alloydb_cluster" "<%= ctx[:primary_resource_id] %>" {
cluster_id = "<%= ctx[:vars]['alloydb_cluster_name'] %>"
location = "us-central1"
network = data.google_compute_network.default.id

initial_user {
password = "<%= ctx[:vars]['alloydb_cluster_name'] %>"
}
}

resource "google_alloydb_instance" "<%= ctx[:primary_resource_id] %>" {
cluster = google_alloydb_cluster.<%= ctx[:primary_resource_id] %>.name
instance_id = "<%= ctx[:vars]['alloydb_instance_name'] %>"
instance_type = "PRIMARY"

machine_config {
cpu_count = 2
}

depends_on = [google_service_networking_connection.vpc_connection]
}

resource "google_alloydb_backup" "<%= ctx[:primary_resource_id] %>" {
backup_id = "<%= ctx[:vars]['alloydb_backup_id'] %>"
location = "us-central1"
cluster_name = google_alloydb_cluster.<%= ctx[:primary_resource_id] %>.name

depends_on = [google_alloydb_instance.<%= ctx[:primary_resource_id] %>]
}

resource "google_alloydb_cluster" "restored_from_backup" {
cluster_id = "<%= ctx[:vars]['alloydb_backup_restored_cluster_name'] %>"
location = "us-central1"
network = data.google_compute_network.default.id
restore_backup_source {
backup_name = google_alloydb_backup.<%= ctx[:primary_resource_id] %>.name
}
}

resource "google_alloydb_cluster" "restored_via_pitr" {
cluster_id = "<%= ctx[:vars]['alloydb_pitr_restored_cluster_name'] %>"
location = "us-central1"
network = data.google_compute_network.default.id

restore_continuous_backup_source {
cluster = google_alloydb_cluster.<%= ctx[:primary_resource_id] %>.name
point_in_time = "2023-08-03T19:19:00.094Z"
}
}

data "google_project" "project" {}

data "google_compute_network" "default" {
name = "<%= ctx[:vars]['network_name'] %>"
}

resource "google_compute_global_address" "private_ip_alloc" {
name = "<%= ctx[:vars]['alloydb_cluster_name'] %>"
address_type = "INTERNAL"
purpose = "VPC_PEERING"
prefix_length = 16
network = data.google_compute_network.default.id
}

resource "google_service_networking_connection" "vpc_connection" {
network = data.google_compute_network.default.id
service = "servicenetworking.googleapis.com"
reserved_peering_ranges = [google_compute_global_address.private_ip_alloc.name]
}
33 changes: 33 additions & 0 deletions mmv1/templates/terraform/pre_create/alloydb_restore_cluster.go.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
// Read the restore variables from obj and remove them, since they do not map to anything in the cluster
Comment thread
GoogleMarcfont marked this conversation as resolved.
var backupSource interface{}
var continuousBackupSource interface{}
if val, ok := obj["restoreBackupSource"]; ok {
backupSource = val
delete(obj, "restoreBackupSource")
}
if val, ok := obj["restoreContinuousBackupSource"]; ok {
continuousBackupSource = val
delete(obj, "restoreContinuousBackupSource")
}

restoreClusterRequestBody := make(map[string]interface{})
if backupSource != nil {
// If restoring from a backup, set the backupSource
restoreClusterRequestBody["backup_source"] = backupSource
} else if continuousBackupSource != nil {
// Otherwise if restoring via PITR, set the continuousBackupSource
restoreClusterRequestBody["continuous_backup_source"] = continuousBackupSource
}

if backupSource != nil || continuousBackupSource != nil {
// Use restore API if this is a restore instead of a create cluster call
url = strings.Replace(url, "clusters?clusterId", "clusters:restore?clusterId", 1)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we call the restore endpoint for creation when one of the backup fields is set and users can also specify cluster settings via restore. Will cluster setting override backup ones if there's conflict?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies, I'm not sure what you mean by "users can also specify cluster settings via restore". The restore API has two main fields that need to be set:

  1. The source, i.e. ContinuousBackupSource or BackupSource
  2. The cluster

See proto definition: http://google3/google/cloud/alloydb/v1main/service.proto;l=1492;rcl=553404819

The cluster can contain all the usual fields (encryption, automated backup policy, continuous backup config, etc.). This has no overlap with the source field, which just describes the backup or source cluster/time to restore from.

Let me know if that answers your question.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I was not clear earlier. By "users can also specify cluster settings via restore", I mean if users specify any fields in the cluster object, what will it happen if it conflicts with the source it restores from. I'm not super familiar with the resource, and it seems that you've already answered the question in your earlier response that "cluster has no overlap with the source field". Then I think it should be fine.


// Copy obj which contains the cluster into a cluster map
cluster := make(map[string]interface{})
for k,v := range obj {
cluster[k] = v
}
restoreClusterRequestBody["cluster"] = cluster
obj = restoreClusterRequestBody
}
Loading