Add the latest changes in Datascan API#8541
Conversation
|
Hello! I am a robot. It looks like you are a community contributor. @ScottSuarez, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look. You can help make sure that review is quick by doing a self-review and by running impacted tests locally. |
| Expected input is a list of fully qualified names of fields as in the schema. | ||
| Only top-level field names for nested fields are supported. | ||
| For instance, if 'x' is of nested field type, listing 'x' is supported but 'x.y.z' is not supported. Here 'y' and 'y.z' are nested fields of 'x'. | ||
| item_type: !ruby/object:Api::Type::String |
There was a problem hiding this comment.
| item_type: !ruby/object:Api::Type::String | |
| item_type: Api::Type::String |
| Expected input is a list of fully qualified names of fields as in the schema. | ||
| Only top-level field names for nested fields are supported. | ||
| For instance, if 'x' is of nested field type, listing 'x' is supported but 'x.y.z' is not supported. Here 'y' and 'y.z' are nested fields of 'x'. | ||
| item_type: !ruby/object:Api::Type::String |
There was a problem hiding this comment.
| item_type: !ruby/object:Api::Type::String | |
| item_type: item_type: Api::Type::String |
| - !ruby/object:Api::Type::NestedObject | ||
| name: 'postScanActionsResult' | ||
| decription: | | ||
| The result of post scan actions. | ||
| properties: | ||
| - !ruby/object:Api::Type::NestedObject | ||
| name: 'bigQueryExportResult' | ||
| description: | | ||
| The result of BigQuery export post scan action. | ||
| properties: | ||
| - !ruby/object:Api::Type::Enum | ||
| name: 'state' | ||
| decription: | | ||
| Execution state for the BigQuery exporting. | ||
| values: | ||
| - STATE_UNSPECIFIED | ||
| - SUCCEEDED | ||
| - FAILED | ||
| - SKIPPED | ||
| - !ruby/object:Api::Type::String | ||
| name: 'message' | ||
| description: | | ||
| Additional information about the BigQuery exporting. |
There was a problem hiding this comment.
This whole block seems like it should be an output field.
There was a problem hiding this comment.
Note that we are looking to remove the DataProfileResult and DataQualityResult in this PR: #8583 (comment)
Based on how that discussion goes, we should look at only adding the export and rule name fields in the spec, and not in the result.
There was a problem hiding this comment.
Thanks removed the fields added in this PR, the rest of the Result field to be removed in @saurabh-net PR.
| - !ruby/object:Api::Type::String | ||
| name: 'name' | ||
| description: | | ||
| A mutable name for the rule. | ||
| The name must contain only letters (a-z, A-Z), numbers (0-9), or hyphens (-). | ||
| The maximum length is 63 characters. | ||
| Must start with a letter. | ||
| Must end with a number or a letter. | ||
| - !ruby/object:Api::Type::String | ||
| name: 'description' | ||
| description: | | ||
| Description of the rule. | ||
| The maximum length is 1,024 characters. |
There was a problem hiding this comment.
Thanks removed the fields added in this PR, the rest of the Result field to be removed in @saurabh-net PR.
| - !ruby/object:Api::Type::NestedObject | ||
| name: 'postScanActionsResult' | ||
| decription: | | ||
| The result of post scan actions. | ||
| properties: | ||
| - !ruby/object:Api::Type::NestedObject | ||
| name: 'bigQueryExportResult' | ||
| description: | | ||
| The result of BigQuery export post scan action. | ||
| properties: | ||
| - !ruby/object:Api::Type::Enum | ||
| name: 'state' | ||
| decription: | | ||
| Execution state for the BigQuery exporting. | ||
| values: | ||
| - STATE_UNSPECIFIED | ||
| - SUCCEEDED | ||
| - FAILED | ||
| - SKIPPED | ||
| - !ruby/object:Api::Type::String | ||
| name: 'message' | ||
| description: | | ||
| Additional information about the BigQuery exporting. |
There was a problem hiding this comment.
Thanks removed the fields added in this PR, the rest of the Result field to be removed in @saurabh-net PR.
| - !ruby/object:Api::Type::String | ||
| name: 'resultsTable' | ||
| description: | | ||
| The BigQuery table to export DataProfileScan results to. | ||
| Format:projects/{project}/datasets/{dataset}/tables/{table} |
There was a problem hiding this comment.
Is it possible to test this field. It's okay if not just want to ensure we are conscious in it's exclusion.
There was a problem hiding this comment.
It is required field and has been tested.
|
Hi there, I'm the Modular magician. I've detected the following information about your changes: Diff reportYour PR generated some diffs in downstreams - here they are. Terraform GA: Diff ( 4 files changed, 560 insertions(+), 8 deletions(-)) Missing test reportYour PR includes resource fields which are not covered by any test. Resource: resource "google_dataplex_datascan" "primary" {
data_profile_spec {
post_scan_actions {
big_query_export {
results_table = # value needed
}
}
}
data_quality_spec {
post_scan_actions {
big_query_export {
results_table = # value needed
}
}
rules {
description = # value needed
name = # value needed
}
}
}
|
Tests analyticsTotal tests: Action takenFound 7 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected testsTestAccHealthcareDatasetIamPolicy|TestAccDataplexDatascan_dataplexDatascanBasicProfileExample|TestAccDataplexDatascan_dataplexDatascanFullProfileExample|TestAccDataplexDatascan_dataplexDatascanBasicQualityExample|TestAccDataplexDatascanIamPolicyGenerated|TestAccDataplexDatascanIamMemberGenerated|TestAccDataplexDatascanIamBindingGenerated |
|
Rerun these tests in REPLAYING mode to catch issues
|
|
Hi there, I'm the Modular magician. I've detected the following information about your changes: Diff reportYour PR generated some diffs in downstreams - here they are. Terraform GA: Diff ( 5 files changed, 565 insertions(+), 8 deletions(-)) Missing test reportYour PR includes resource fields which are not covered by any test. Resource: resource "google_dataplex_datascan" "primary" {
data_profile_spec {
post_scan_actions {
big_query_export {
results_table = # value needed
}
}
}
data_quality_spec {
post_scan_actions {
big_query_export {
results_table = # value needed
}
}
}
}
|
Tests analyticsTotal tests: Action takenFound 6 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected testsTestAccDataplexDatascan_dataplexDatascanBasicQualityExample|TestAccDataplexDatascan_dataplexDatascanBasicProfileExample|TestAccBigtableAppProfile_bigtableAppProfileMulticlusterExample|TestAccDataplexDatascanIamMemberGenerated|TestAccDataplexDatascanIamBindingGenerated|TestAccDataplexDatascanIamPolicyGenerated |
|
Rerun these tests in REPLAYING mode to catch issues
|
|
Hi there, I'm the Modular magician. I've detected the following information about your changes: Diff reportYour PR generated some diffs in downstreams - here they are. Terraform GA: Diff ( 4 files changed, 560 insertions(+), 8 deletions(-)) |
Tests analyticsTotal tests: Action takenFound 5 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected testsTestAccDataplexDatascan_dataplexDatascanBasicProfileExample|TestAccDataplexDatascan_dataplexDatascanBasicQualityExample|TestAccDataplexDatascanIamMemberGenerated|TestAccDataplexDatascanIamBindingGenerated|TestAccDataplexDatascanIamPolicyGenerated |
|
|
…at in the examples in yaml
|
Hi there, I'm the Modular magician. I've detected the following information about your changes: Diff reportYour PR generated some diffs in downstreams - here they are. Terraform GA: Diff ( 4 files changed, 560 insertions(+), 8 deletions(-)) |
Tests analyticsTotal tests: Action takenFound 5 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected testsTestAccDataplexDatascanIamPolicyGenerated|TestAccDataplexDatascanIamMemberGenerated|TestAccDataplexDatascanIamBindingGenerated|TestAccDataplexDatascan_dataplexDatascanBasicQualityExample|TestAccDataplexDatascan_dataplexDatascanBasicProfileExample |
|
|
|
Hi there, I'm the Modular magician. I've detected the following information about your changes: Diff reportYour PR generated some diffs in downstreams - here they are. Terraform GA: Diff ( 4 files changed, 522 insertions(+), 22 deletions(-)) Missing test reportYour PR includes resource fields which are not covered by any test. Resource: resource "google_dataplex_datascan" "primary" {
data_profile_spec {
post_scan_actions {
bigquery_export {
results_table = # value needed
}
}
}
data_quality_spec {
post_scan_actions {
bigquery_export {
results_table = # value needed
}
}
}
}
|
Tests analyticsTotal tests: Action takenFound 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected testsTestAccDataplexDatascan_dataplexDatascanBasicQualityExample |
|
Rerun these tests in REPLAYING mode to catch issues
|
|
Hi there, I'm the Modular magician. I've detected the following information about your changes: Diff reportYour PR generated some diffs in downstreams - here they are. Terraform GA: Diff ( 4 files changed, 613 insertions(+), 15 deletions(-)) Missing test reportYour PR includes resource fields which are not covered by any test. Resource: resource "google_dataplex_datascan" "primary" {
data_profile_spec {
exclude_fields {
field_names = # value needed
}
include_fields {
field_names = # value needed
}
post_scan_actions {
bigquery_export {
results_table = # value needed
}
}
}
data_quality_spec {
post_scan_actions {
bigquery_export {
results_table = # value needed
}
}
rules {
description = # value needed
name = # value needed
}
}
}
|
Tests analyticsTotal tests: Action takenFound 5 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected testsTestAccDataplexDatascan_dataplexDatascanBasicProfileExample|TestAccDataplexDatascanIamBindingGenerated|TestAccDataplexDatascanIamMemberGenerated|TestAccDataplexDatascanIamPolicyGenerated|TestAccDataSourceGoogleServiceAccountJwt |
|
Rerun these tests in REPLAYING mode to catch issues
|
|
Hi there, I'm the Modular magician. I've detected the following information about your changes: Diff reportYour PR generated some diffs in downstreams - here they are. Terraform GA: Diff ( 4 files changed, 613 insertions(+), 15 deletions(-)) Missing test reportYour PR includes resource fields which are not covered by any test. Resource: resource "google_dataplex_datascan" "primary" {
data_profile_spec {
exclude_fields {
field_names = # value needed
}
include_fields {
field_names = # value needed
}
post_scan_actions {
bigquery_export {
results_table = # value needed
}
}
}
data_quality_spec {
post_scan_actions {
bigquery_export {
results_table = # value needed
}
}
rules {
description = # value needed
name = # value needed
}
}
}
|
Tests analyticsTotal tests: Action takenFound 4 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected testsTestAccDataplexDatascan_dataplexDatascanBasicProfileExample|TestAccDataplexDatascanIamMemberGenerated|TestAccDataplexDatascanIamBindingGenerated|TestAccDataplexDatascanIamPolicyGenerated |
|
|
|
Hi there, I'm the Modular magician. I've detected the following information about your changes: Diff reportYour PR generated some diffs in downstreams - here they are. Terraform GA: Diff ( 4 files changed, 613 insertions(+), 15 deletions(-)) |
Tests analyticsTotal tests: Action takenFound 4 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected testsTestAccDataplexDatascanIamMemberGenerated|TestAccDataplexDatascan_dataplexDatascanBasicProfileExample|TestAccDataplexDatascanIamBindingGenerated|TestAccDataplexDatascanIamPolicyGenerated |
|
|
|
Hi there, I'm the Modular magician. I've detected the following information about your changes: Diff reportYour PR generated some diffs in downstreams - here they are. Terraform GA: Diff ( 4 files changed, 634 insertions(+), 15 deletions(-)) |
Tests analyticsTotal tests: Action takenFound 6 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected testsTestAccCertificateManagerCertificate_certificateManagerSelfManagedCertificateExample|TestAccCertificateManagerCertificate_certificateManagerGoogleManagedCertificateIssuanceConfigExample|TestAccDataplexDatascanIamMemberGenerated|TestAccDataplexDatascan_dataplexDatascanBasicProfileExample|TestAccDataplexDatascanIamBindingGenerated|TestAccDataplexDatascanIamPolicyGenerated |
|
Rerun these tests in REPLAYING mode to catch issues
|
|
Hi there, I'm the Modular magician. I've detected the following information about your changes: Diff reportYour PR generated some diffs in downstreams - here they are. Terraform GA: Diff ( 4 files changed, 634 insertions(+), 15 deletions(-)) |
|
If you can't get the bigquery export tests to work we can omit them |
Tests analyticsTotal tests: Action takenFound 5 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected testsTestAccBigQueryTable_HivePartitioning|TestAccDataplexDatascanIamBindingGenerated|TestAccDataplexDatascanIamMemberGenerated|TestAccDataplexDatascan_dataplexDatascanBasicProfileExample|TestAccDataplexDatascanIamPolicyGenerated |
|
Rerun these tests in REPLAYING mode to catch issues
|
|
Hi there, I'm the Modular magician. I've detected the following information about your changes: Diff reportYour PR generated some diffs in downstreams - here they are. Terraform GA: Diff ( 4 files changed, 522 insertions(+), 22 deletions(-)) |
Tests analyticsTotal tests:
|
4 participants
Updating Datascan terraform support with the latest changes and additions in Datascan API.
If this PR is for Terraform, I acknowledge that I have:
make testandmake lintin the generated providers to ensure it passes unit and linter tests.Release Note Template for Downstream PRs (will be copied)