Local transform and resource collection by andrew-zhan139 · Pull Request #4020 · aws/aws-sam-cli

andrew-zhan139 · 2022-06-30T00:42:03Z

Which issue(s) does this change fix?

Why is this change necessary?

How does it address the issue?

What side effects does this change have?

Mandatory Checklist

PRs will only be reviewed after checklist is complete

Add input/output type hints to new functions/methods
Write design document if needed (Do I need to write a design document?)
Write/update unit tests
Write/update integration tests
Write/update functional tests if needed
make pr passes
make update-reproducible-reqs if dependencies were changed
Write documentation

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…ages

…on test suite

…solve test failure

… message

…added method annotations, fixed text formatting

…undant init_client call

…into base-command

torresxb1 · 2022-07-06T09:50:38Z

+                    "Please provide a region via the --region parameter or by the AWS_REGION environment variable."
+                )
+
+        client_provider = get_boto_client_provider_with_config(region=self.region, profile=self.profile)


🤔 I don't think you set the profile anywhere

ListContext is inherited by ResourcesContext that may have profile

torresxb1 · 2022-07-06T13:02:07Z

+"""
+import os
+import logging
+import click


I believe we should not use click outside of the commands directory: https://github.com/aws/aws-sam-cli/blob/develop/DEVELOPMENT_GUIDE.md#code-conventions

Moved it to the commands directory

torresxb1 · 2022-07-06T13:04:52Z

@@ -0,0 +1,31 @@
+"""
+Library for Validating Sam Templates


probably doesn't belong here

torresxb1 · 2022-07-06T13:05:20Z

+LOG = logging.getLogger(__name__)
+
+
+def _read_sam_file(template):


if it's meant to be used from outside this file, we should remove the leading underscore

Removed the underscore

torresxb1 · 2022-07-06T13:10:37Z

@@ -0,0 +1,80 @@
+"""
+in progress


torresxb1 · 2022-07-06T13:22:31Z

+import dataclasses
+import logging
+import yaml
+import click


same comment as the other one on not using click outside of commands

torresxb1 · 2022-07-06T13:37:02Z

+        template_dict_format: Optional[Dict]
+            Optional dictionary representing the sam template file to be used instead of the template file


I was a bit confused by this before I looked into the implementation. I think just template_dict might be a better name, and saying representing the sam template rather than representing the sam template file. Also should leave some message that one of this one (and only one) of either this one or template file is required.

Also, since now the template file is not required (as they could provide the dict) we should probable reflect that in the function's parameters and make it an Optional type. Would also need to update the comment to reflect this (and probably also include again here a line that one and only one of either this or template_dict is required). Also, in that case, should handle below in the code what happens if the user doesn't provide template file or dict (should throw an exception)

Made the fixes and added the exception raise for if neither are passed

torresxb1 · 2022-07-06T13:43:32Z

+            resources_dict[local_resource] = "-"
+
+        for logical_id, physical_id in resources_dict.items():
+            resource_data = ResourcesDef(LogicalResourceId=logical_id, PhysicalResourceId=physical_id)


not sure if the dataclass is needed here, since it seems it's immediately turned into a dict in the next line 🤔

I think the dataclass makes it more organized and conforming to the design pattern

torresxb1 · 2022-07-06T13:47:40Z

+        for local_resource in stacks[0].resources:
+            # Set the PhysicalID to "-" if there is no corresponding PhysicalID
+            resources_dict[local_resource] = "-"


is this part unfinished? Will we look for the actual physical ID later on?

Yup, the physical ID part will be added in the next PR

torresxb1 · 2022-07-06T13:52:39Z

+            # Set the PhysicalID to "-" if there is no corresponding PhysicalID
+            resources_dict[local_resource] = "-"
+
+        for logical_id, physical_id in resources_dict.items():


it seems like you could just do the logic that's under this loop in the loop above, no?

Added it to the above loop

torresxb1

Thanks for addressing all the feedback!
Andrew and I talked about about unit testing, and we concluded that we'll merge in this PR and make some changes to unit testing in another PR:

making sure each function has unit tests
making sure the unit tests for each function test the functionality of that specific function (and not the internal helper functions called, since those should have their own unit tests)

* feat: Add table view and rich table (#3851) * Add table view and rich table * Black reformat * Update reproducible reqs * Comment out table comparison assertions * Make rich table class members protected * feat: Adding base commands and help messages for sam list (#3912) * Added the base commands of sam list and their corresponding help messages * Added no-args handling to base commands and added files for integration test suite * Made additions to resources integration tests * Made additions to the sam list integration test suite * Added to sam list integration and unit tests * Added integration tests to test help messages of sam list commands * Reformatted files * Cleaned up unfinished tests * adding check to see what the appveyor test will produce. Trying to resolve test failure * Fixed test to check help messages * modified pattern matching for eliminating newlines when matching help message * Changed the way whitespaces are handled in matching help messages * Addressed PR comments, moved items into common classes * Made modifications based on comments, removed relative import paths, added method annotations, fixed text formatting * Reformatted files * removed folder deletion * removed uneccessary folder creation and deletion * fixed errors with cwd of integration tests Co-authored-by: Andrew Zhan <zhandr@amazon.com> * Added the stack-outputs command implementation and tests (#3947) Co-authored-by: Andrew Zhan <zhandr@amazon.com> * Refactored stack outputs command to the producer mapper consumer design pattern (#3980) * Added the base commands of sam list and their corresponding help messages * Added no-args handling to base commands and added files for integration test suite * Made additions to resources integration tests * Made additions to the sam list integration test suite * Added to sam list integration and unit tests * Added integration tests to test help messages of sam list commands * Reformatted files * Cleaned up unfinished tests * adding check to see what the appveyor test will produce. Trying to resolve test failure * Fixed test to check help messages * modified pattern matching for eliminating newlines when matching help message * Changed the way whitespaces are handled in matching help messages * Addressed PR comments, moved items into common classes * Made modifications based on comments, removed relative import paths, added method annotations, fixed text formatting * Reformatted files * removed folder deletion * removed uneccessary folder creation and deletion * fixed errors with cwd of integration tests * Added implementation and tests for the stack-outputs command * Added test skips for integration tests, added unit tests, removed redundant init_client call * commit to retrigger appveyor tests * Commmit to trigger appveyor * Modified client source, made fixes based on comments * Made fixes based on comments * Combined get_stack_info and stack_exists, and modified unit tests * Empty-Commit * Empty-Commit * Empty-Commit * fixed tests based on comments * reformatted file * Refactored stack outputs command to the producer mapper consumer design pattern * Fixed formatting * Moved interfaces, made changes based on comments * Made fixes based on comments * Made fixes based on comments * Empty commit * Made changes based on comments, added new exceptions * Fixed format * Fixed return type declaration * Fixed return type declaration * Changed return type to list * Fixed error Co-authored-by: Andrew Zhan <zhandr@amazon.com> * Local transform and resource collection (#4020) Co-authored-by: Andrew Zhan <zhandr@amazon.com> * feat: Adding cloud resources to sam list resources output (#4056) * Added the base commands of sam list and their corresponding help messages * Added no-args handling to base commands and added files for integration test suite * Made additions to resources integration tests * Made additions to the sam list integration test suite * Added to sam list integration and unit tests * Added integration tests to test help messages of sam list commands * Reformatted files * Cleaned up unfinished tests * adding check to see what the appveyor test will produce. Trying to resolve test failure * Fixed test to check help messages * modified pattern matching for eliminating newlines when matching help message * Changed the way whitespaces are handled in matching help messages * Addressed PR comments, moved items into common classes * Made modifications based on comments, removed relative import paths, added method annotations, fixed text formatting * Reformatted files * removed folder deletion * removed uneccessary folder creation and deletion * fixed errors with cwd of integration tests * Added implementation and tests for the stack-outputs command * Added test skips for integration tests, added unit tests, removed redundant init_client call * commit to retrigger appveyor tests * Commmit to trigger appveyor * Modified client source, made fixes based on comments * Made fixes based on comments * Combined get_stack_info and stack_exists, and modified unit tests * Empty-Commit * Empty-Commit * Empty-Commit * fixed tests based on comments * reformatted file * Refactored stack outputs command to the producer mapper consumer design pattern * Fixed formatting * Moved interfaces, made changes based on comments * Made fixes based on comments * Made fixes based on comments * Empty commit * Made changes based on comments, added new exceptions * Fixed format * Fixed return type declaration * Fixed return type declaration * Changed return type to list * Fixed error * Implementation of the local transform and resource collection * Empty-Commit * Added section to avoid unused variable * Refactored common code * Added tests, modified PR * Fixed formatting * Made fixes based on PR comments * Fixed formatting * Fixed typing errors * Reverted typing * Fixed error with typing * Made changes to handling optional params * Fixes to typing errors * Made edits based on comments * Fixed error * Changed return type * Reverted return type due to make pr error * Added change to fix make pr error * Removed translate_utils.py file * Added cloud resources to sam list resources output * Empty commit * modified test format * Modified tests * Modified test Co-authored-by: Andrew Zhan <zhandr@amazon.com> * feat: Adding the sam list testable resources command, tests, and table output format support for all sam list commands (#4081) * Added the base commands of sam list and their corresponding help messages * Added no-args handling to base commands and added files for integration test suite * Made additions to resources integration tests * Made additions to the sam list integration test suite * Added to sam list integration and unit tests * Added integration tests to test help messages of sam list commands * Reformatted files * Cleaned up unfinished tests * adding check to see what the appveyor test will produce. Trying to resolve test failure * Fixed test to check help messages * modified pattern matching for eliminating newlines when matching help message * Changed the way whitespaces are handled in matching help messages * Addressed PR comments, moved items into common classes * Made modifications based on comments, removed relative import paths, added method annotations, fixed text formatting * Reformatted files * removed folder deletion * removed uneccessary folder creation and deletion * fixed errors with cwd of integration tests * Added implementation and tests for the stack-outputs command * Added test skips for integration tests, added unit tests, removed redundant init_client call * commit to retrigger appveyor tests * Commmit to trigger appveyor * Modified client source, made fixes based on comments * Made fixes based on comments * Combined get_stack_info and stack_exists, and modified unit tests * Empty-Commit * Empty-Commit * Empty-Commit * fixed tests based on comments * reformatted file * Refactored stack outputs command to the producer mapper consumer design pattern * Fixed formatting * Moved interfaces, made changes based on comments * Made fixes based on comments * Made fixes based on comments * Empty commit * Made changes based on comments, added new exceptions * Fixed format * Fixed return type declaration * Fixed return type declaration * Changed return type to list * Fixed error * Implementation of the local transform and resource collection * Empty-Commit * Added section to avoid unused variable * Refactored common code * Added tests, modified PR * Fixed formatting * Made fixes based on PR comments * Fixed formatting * Fixed typing errors * Reverted typing * Fixed error with typing * Made changes to handling optional params * Fixes to typing errors * Made edits based on comments * Fixed error * Changed return type * Reverted return type due to make pr error * Added change to fix make pr error * Removed translate_utils.py file * Added cloud resources to sam list resources output * Empty commit * modified test format * Modified tests * Modified test * Adding the sam list testable resources command, tests, and table output format support for all sam list commands * Changed table and made changes based on pr comments * Fixed integration test expected outputs * Fixed table heading * Added docstring and re-arranged the testable resources producer to reduce if-elses within a single function * Fixed format Co-authored-by: Andrew Zhan <zhandr@amazon.com> * Renaming the 'testable-resources' command to 'endpoints' (#4116) * Added the base commands of sam list and their corresponding help messages * Added no-args handling to base commands and added files for integration test suite * Made additions to resources integration tests * Made additions to the sam list integration test suite * Added to sam list integration and unit tests * Added integration tests to test help messages of sam list commands * Reformatted files * Cleaned up unfinished tests * adding check to see what the appveyor test will produce. Trying to resolve test failure * Fixed test to check help messages * modified pattern matching for eliminating newlines when matching help message * Changed the way whitespaces are handled in matching help messages * Addressed PR comments, moved items into common classes * Made modifications based on comments, removed relative import paths, added method annotations, fixed text formatting * Reformatted files * removed folder deletion * removed uneccessary folder creation and deletion * fixed errors with cwd of integration tests * Added implementation and tests for the stack-outputs command * Added test skips for integration tests, added unit tests, removed redundant init_client call * commit to retrigger appveyor tests * Commmit to trigger appveyor * Modified client source, made fixes based on comments * Made fixes based on comments * Combined get_stack_info and stack_exists, and modified unit tests * Empty-Commit * Empty-Commit * Empty-Commit * fixed tests based on comments * reformatted file * Refactored stack outputs command to the producer mapper consumer design pattern * Fixed formatting * Moved interfaces, made changes based on comments * Made fixes based on comments * Made fixes based on comments * Empty commit * Made changes based on comments, added new exceptions * Fixed format * Fixed return type declaration * Fixed return type declaration * Changed return type to list * Fixed error * Implementation of the local transform and resource collection * Empty-Commit * Added section to avoid unused variable * Refactored common code * Added tests, modified PR * Fixed formatting * Made fixes based on PR comments * Fixed formatting * Fixed typing errors * Reverted typing * Fixed error with typing * Made changes to handling optional params * Fixes to typing errors * Made edits based on comments * Fixed error * Changed return type * Reverted return type due to make pr error * Added change to fix make pr error * Removed translate_utils.py file * Added cloud resources to sam list resources output * Empty commit * modified test format * Modified tests * Modified test * Adding the sam list testable resources command, tests, and table output format support for all sam list commands * Changed table and made changes based on pr comments * Fixed integration test expected outputs * Fixed table heading * Added docstring and re-arranged the testable resources producer to reduce if-elses within a single function * Fixed format * Renamed command from testable resources to endpoints Co-authored-by: Andrew Zhan <zhandr@amazon.com> * Cleanup integration tests * Cleanup tests, address comments * Move boto3 imports, update unit tests * Add comments, use constants for resources * Update unit test mocking type * Add missing parameters * Add additional comments * Fix spelling, minor updates Co-authored-by: Mehmet Nuri Deveci <5735811+mndeveci@users.noreply.github.com> Co-authored-by: andrewzhan <andrewzhan8@gmail.com> Co-authored-by: Andrew Zhan <zhandr@amazon.com>

* feat: List Command (#4587) * feat: Add table view and rich table (#3851) * Add table view and rich table * Black reformat * Update reproducible reqs * Comment out table comparison assertions * Make rich table class members protected * feat: Adding base commands and help messages for sam list (#3912) * Added the base commands of sam list and their corresponding help messages * Added no-args handling to base commands and added files for integration test suite * Made additions to resources integration tests * Made additions to the sam list integration test suite * Added to sam list integration and unit tests * Added integration tests to test help messages of sam list commands * Reformatted files * Cleaned up unfinished tests * adding check to see what the appveyor test will produce. Trying to resolve test failure * Fixed test to check help messages * modified pattern matching for eliminating newlines when matching help message * Changed the way whitespaces are handled in matching help messages * Addressed PR comments, moved items into common classes * Made modifications based on comments, removed relative import paths, added method annotations, fixed text formatting * Reformatted files * removed folder deletion * removed uneccessary folder creation and deletion * fixed errors with cwd of integration tests Co-authored-by: Andrew Zhan <zhandr@amazon.com> * Added the stack-outputs command implementation and tests (#3947) Co-authored-by: Andrew Zhan <zhandr@amazon.com> * Refactored stack outputs command to the producer mapper consumer design pattern (#3980) * Added the base commands of sam list and their corresponding help messages * Added no-args handling to base commands and added files for integration test suite * Made additions to resources integration tests * Made additions to the sam list integration test suite * Added to sam list integration and unit tests * Added integration tests to test help messages of sam list commands * Reformatted files * Cleaned up unfinished tests * adding check to see what the appveyor test will produce. Trying to resolve test failure * Fixed test to check help messages * modified pattern matching for eliminating newlines when matching help message * Changed the way whitespaces are handled in matching help messages * Addressed PR comments, moved items into common classes * Made modifications based on comments, removed relative import paths, added method annotations, fixed text formatting * Reformatted files * removed folder deletion * removed uneccessary folder creation and deletion * fixed errors with cwd of integration tests * Added implementation and tests for the stack-outputs command * Added test skips for integration tests, added unit tests, removed redundant init_client call * commit to retrigger appveyor tests * Commmit to trigger appveyor * Modified client source, made fixes based on comments * Made fixes based on comments * Combined get_stack_info and stack_exists, and modified unit tests * Empty-Commit * Empty-Commit * Empty-Commit * fixed tests based on comments * reformatted file * Refactored stack outputs command to the producer mapper consumer design pattern * Fixed formatting * Moved interfaces, made changes based on comments * Made fixes based on comments * Made fixes based on comments * Empty commit * Made changes based on comments, added new exceptions * Fixed format * Fixed return type declaration * Fixed return type declaration * Changed return type to list * Fixed error Co-authored-by: Andrew Zhan <zhandr@amazon.com> * Local transform and resource collection (#4020) Co-authored-by: Andrew Zhan <zhandr@amazon.com> * feat: Adding cloud resources to sam list resources output (#4056) * Added the base commands of sam list and their corresponding help messages * Added no-args handling to base commands and added files for integration test suite * Made additions to resources integration tests * Made additions to the sam list integration test suite * Added to sam list integration and unit tests * Added integration tests to test help messages of sam list commands * Reformatted files * Cleaned up unfinished tests * adding check to see what the appveyor test will produce. Trying to resolve test failure * Fixed test to check help messages * modified pattern matching for eliminating newlines when matching help message * Changed the way whitespaces are handled in matching help messages * Addressed PR comments, moved items into common classes * Made modifications based on comments, removed relative import paths, added method annotations, fixed text formatting * Reformatted files * removed folder deletion * removed uneccessary folder creation and deletion * fixed errors with cwd of integration tests * Added implementation and tests for the stack-outputs command * Added test skips for integration tests, added unit tests, removed redundant init_client call * commit to retrigger appveyor tests * Commmit to trigger appveyor * Modified client source, made fixes based on comments * Made fixes based on comments * Combined get_stack_info and stack_exists, and modified unit tests * Empty-Commit * Empty-Commit * Empty-Commit * fixed tests based on comments * reformatted file * Refactored stack outputs command to the producer mapper consumer design pattern * Fixed formatting * Moved interfaces, made changes based on comments * Made fixes based on comments * Made fixes based on comments * Empty commit * Made changes based on comments, added new exceptions * Fixed format * Fixed return type declaration * Fixed return type declaration * Changed return type to list * Fixed error * Implementation of the local transform and resource collection * Empty-Commit * Added section to avoid unused variable * Refactored common code * Added tests, modified PR * Fixed formatting * Made fixes based on PR comments * Fixed formatting * Fixed typing errors * Reverted typing * Fixed error with typing * Made changes to handling optional params * Fixes to typing errors * Made edits based on comments * Fixed error * Changed return type * Reverted return type due to make pr error * Added change to fix make pr error * Removed translate_utils.py file * Added cloud resources to sam list resources output * Empty commit * modified test format * Modified tests * Modified test Co-authored-by: Andrew Zhan <zhandr@amazon.com> * feat: Adding the sam list testable resources command, tests, and table output format support for all sam list commands (#4081) * Added the base commands of sam list and their corresponding help messages * Added no-args handling to base commands and added files for integration test suite * Made additions to resources integration tests * Made additions to the sam list integration test suite * Added to sam list integration and unit tests * Added integration tests to test help messages of sam list commands * Reformatted files * Cleaned up unfinished tests * adding check to see what the appveyor test will produce. Trying to resolve test failure * Fixed test to check help messages * modified pattern matching for eliminating newlines when matching help message * Changed the way whitespaces are handled in matching help messages * Addressed PR comments, moved items into common classes * Made modifications based on comments, removed relative import paths, added method annotations, fixed text formatting * Reformatted files * removed folder deletion * removed uneccessary folder creation and deletion * fixed errors with cwd of integration tests * Added implementation and tests for the stack-outputs command * Added test skips for integration tests, added unit tests, removed redundant init_client call * commit to retrigger appveyor tests * Commmit to trigger appveyor * Modified client source, made fixes based on comments * Made fixes based on comments * Combined get_stack_info and stack_exists, and modified unit tests * Empty-Commit * Empty-Commit * Empty-Commit * fixed tests based on comments * reformatted file * Refactored stack outputs command to the producer mapper consumer design pattern * Fixed formatting * Moved interfaces, made changes based on comments * Made fixes based on comments * Made fixes based on comments * Empty commit * Made changes based on comments, added new exceptions * Fixed format * Fixed return type declaration * Fixed return type declaration * Changed return type to list * Fixed error * Implementation of the local transform and resource collection * Empty-Commit * Added section to avoid unused variable * Refactored common code * Added tests, modified PR * Fixed formatting * Made fixes based on PR comments * Fixed formatting * Fixed typing errors * Reverted typing * Fixed error with typing * Made changes to handling optional params * Fixes to typing errors * Made edits based on comments * Fixed error * Changed return type * Reverted return type due to make pr error * Added change to fix make pr error * Removed translate_utils.py file * Added cloud resources to sam list resources output * Empty commit * modified test format * Modified tests * Modified test * Adding the sam list testable resources command, tests, and table output format support for all sam list commands * Changed table and made changes based on pr comments * Fixed integration test expected outputs * Fixed table heading * Added docstring and re-arranged the testable resources producer to reduce if-elses within a single function * Fixed format Co-authored-by: Andrew Zhan <zhandr@amazon.com> * Renaming the 'testable-resources' command to 'endpoints' (#4116) * Added the base commands of sam list and their corresponding help messages * Added no-args handling to base commands and added files for integration test suite * Made additions to resources integration tests * Made additions to the sam list integration test suite * Added to sam list integration and unit tests * Added integration tests to test help messages of sam list commands * Reformatted files * Cleaned up unfinished tests * adding check to see what the appveyor test will produce. Trying to resolve test failure * Fixed test to check help messages * modified pattern matching for eliminating newlines when matching help message * Changed the way whitespaces are handled in matching help messages * Addressed PR comments, moved items into common classes * Made modifications based on comments, removed relative import paths, added method annotations, fixed text formatting * Reformatted files * removed folder deletion * removed uneccessary folder creation and deletion * fixed errors with cwd of integration tests * Added implementation and tests for the stack-outputs command * Added test skips for integration tests, added unit tests, removed redundant init_client call * commit to retrigger appveyor tests * Commmit to trigger appveyor * Modified client source, made fixes based on comments * Made fixes based on comments * Combined get_stack_info and stack_exists, and modified unit tests * Empty-Commit * Empty-Commit * Empty-Commit * fixed tests based on comments * reformatted file * Refactored stack outputs command to the producer mapper consumer design pattern * Fixed formatting * Moved interfaces, made changes based on comments * Made fixes based on comments * Made fixes based on comments * Empty commit * Made changes based on comments, added new exceptions * Fixed format * Fixed return type declaration * Fixed return type declaration * Changed return type to list * Fixed error * Implementation of the local transform and resource collection * Empty-Commit * Added section to avoid unused variable * Refactored common code * Added tests, modified PR * Fixed formatting * Made fixes based on PR comments * Fixed formatting * Fixed typing errors * Reverted typing * Fixed error with typing * Made changes to handling optional params * Fixes to typing errors * Made edits based on comments * Fixed error * Changed return type * Reverted return type due to make pr error * Added change to fix make pr error * Removed translate_utils.py file * Added cloud resources to sam list resources output * Empty commit * modified test format * Modified tests * Modified test * Adding the sam list testable resources command, tests, and table output format support for all sam list commands * Changed table and made changes based on pr comments * Fixed integration test expected outputs * Fixed table heading * Added docstring and re-arranged the testable resources producer to reduce if-elses within a single function * Fixed format * Renamed command from testable resources to endpoints Co-authored-by: Andrew Zhan <zhandr@amazon.com> * Cleanup integration tests * Cleanup tests, address comments * Move boto3 imports, update unit tests * Add comments, use constants for resources * Update unit test mocking type * Add missing parameters * Add additional comments * Fix spelling, minor updates Co-authored-by: Mehmet Nuri Deveci <5735811+mndeveci@users.noreply.github.com> Co-authored-by: andrewzhan <andrewzhan8@gmail.com> Co-authored-by: Andrew Zhan <zhandr@amazon.com> * feat: Add warning about not providing stack name option (#4624) * fix: Fix failing list tests on Windows (#4623) * fix: Fix failing list tests on Windows * Black reformat * Add event tracking for sam validate --lint metrics (#4612) * Update lint helpand output message * Add event tracking for sam validate --lint metrics * Add unit test for tracking --------- Co-authored-by: Sriram Madapusi Vasudevan <3770774+sriram-mv@users.noreply.github.com> Co-authored-by: Qingchuan Ma <69653965+qingchm@users.noreply.github.com> Co-authored-by: Mehmet Nuri Deveci <5735811+mndeveci@users.noreply.github.com> * Use safe yaml parse in list producer (#4632) * Revert an integration test change related to permission change revert (#4633) --------- Co-authored-by: Daniel Mil <84205762+mildaniel@users.noreply.github.com> Co-authored-by: Mehmet Nuri Deveci <5735811+mndeveci@users.noreply.github.com> Co-authored-by: andrewzhan <andrewzhan8@gmail.com> Co-authored-by: Andrew Zhan <zhandr@amazon.com> Co-authored-by: David <114027923+cdavidxu-hub@users.noreply.github.com> Co-authored-by: Sriram Madapusi Vasudevan <3770774+sriram-mv@users.noreply.github.com> Co-authored-by: Qingchuan Ma <69653965+qingchm@users.noreply.github.com>

* chore: Merge from develop into feat/apigw-lambda-auth (#4642) * feat: List Command (#4587) * feat: Add table view and rich table (#3851) * Add table view and rich table * Black reformat * Update reproducible reqs * Comment out table comparison assertions * Make rich table class members protected * feat: Adding base commands and help messages for sam list (#3912) * Added the base commands of sam list and their corresponding help messages * Added no-args handling to base commands and added files for integration test suite * Made additions to resources integration tests * Made additions to the sam list integration test suite * Added to sam list integration and unit tests * Added integration tests to test help messages of sam list commands * Reformatted files * Cleaned up unfinished tests * adding check to see what the appveyor test will produce. Trying to resolve test failure * Fixed test to check help messages * modified pattern matching for eliminating newlines when matching help message * Changed the way whitespaces are handled in matching help messages * Addressed PR comments, moved items into common classes * Made modifications based on comments, removed relative import paths, added method annotations, fixed text formatting * Reformatted files * removed folder deletion * removed uneccessary folder creation and deletion * fixed errors with cwd of integration tests Co-authored-by: Andrew Zhan <zhandr@amazon.com> * Added the stack-outputs command implementation and tests (#3947) Co-authored-by: Andrew Zhan <zhandr@amazon.com> * Refactored stack outputs command to the producer mapper consumer design pattern (#3980) * Added the base commands of sam list and their corresponding help messages * Added no-args handling to base commands and added files for integration test suite * Made additions to resources integration tests * Made additions to the sam list integration test suite * Added to sam list integration and unit tests * Added integration tests to test help messages of sam list commands * Reformatted files * Cleaned up unfinished tests * adding check to see what the appveyor test will produce. Trying to resolve test failure * Fixed test to check help messages * modified pattern matching for eliminating newlines when matching help message * Changed the way whitespaces are handled in matching help messages * Addressed PR comments, moved items into common classes * Made modifications based on comments, removed relative import paths, added method annotations, fixed text formatting * Reformatted files * removed folder deletion * removed uneccessary folder creation and deletion * fixed errors with cwd of integration tests * Added implementation and tests for the stack-outputs command * Added test skips for integration tests, added unit tests, removed redundant init_client call * commit to retrigger appveyor tests * Commmit to trigger appveyor * Modified client source, made fixes based on comments * Made fixes based on comments * Combined get_stack_info and stack_exists, and modified unit tests * Empty-Commit * Empty-Commit * Empty-Commit * fixed tests based on comments * reformatted file * Refactored stack outputs command to the producer mapper consumer design pattern * Fixed formatting * Moved interfaces, made changes based on comments * Made fixes based on comments * Made fixes based on comments * Empty commit * Made changes based on comments, added new exceptions * Fixed format * Fixed return type declaration * Fixed return type declaration * Changed return type to list * Fixed error Co-authored-by: Andrew Zhan <zhandr@amazon.com> * Local transform and resource collection (#4020) Co-authored-by: Andrew Zhan <zhandr@amazon.com> * feat: Adding cloud resources to sam list resources output (#4056) * Added the base commands of sam list and their corresponding help messages * Added no-args handling to base commands and added files for integration test suite * Made additions to resources integration tests * Made additions to the sam list integration test suite * Added to sam list integration and unit tests * Added integration tests to test help messages of sam list commands * Reformatted files * Cleaned up unfinished tests * adding check to see what the appveyor test will produce. Trying to resolve test failure * Fixed test to check help messages * modified pattern matching for eliminating newlines when matching help message * Changed the way whitespaces are handled in matching help messages * Addressed PR comments, moved items into common classes * Made modifications based on comments, removed relative import paths, added method annotations, fixed text formatting * Reformatted files * removed folder deletion * removed uneccessary folder creation and deletion * fixed errors with cwd of integration tests * Added implementation and tests for the stack-outputs command * Added test skips for integration tests, added unit tests, removed redundant init_client call * commit to retrigger appveyor tests * Commmit to trigger appveyor * Modified client source, made fixes based on comments * Made fixes based on comments * Combined get_stack_info and stack_exists, and modified unit tests * Empty-Commit * Empty-Commit * Empty-Commit * fixed tests based on comments * reformatted file * Refactored stack outputs command to the producer mapper consumer design pattern * Fixed formatting * Moved interfaces, made changes based on comments * Made fixes based on comments * Made fixes based on comments * Empty commit * Made changes based on comments, added new exceptions * Fixed format * Fixed return type declaration * Fixed return type declaration * Changed return type to list * Fixed error * Implementation of the local transform and resource collection * Empty-Commit * Added section to avoid unused variable * Refactored common code * Added tests, modified PR * Fixed formatting * Made fixes based on PR comments * Fixed formatting * Fixed typing errors * Reverted typing * Fixed error with typing * Made changes to handling optional params * Fixes to typing errors * Made edits based on comments * Fixed error * Changed return type * Reverted return type due to make pr error * Added change to fix make pr error * Removed translate_utils.py file * Added cloud resources to sam list resources output * Empty commit * modified test format * Modified tests * Modified test Co-authored-by: Andrew Zhan <zhandr@amazon.com> * feat: Adding the sam list testable resources command, tests, and table output format support for all sam list commands (#4081) * Added the base commands of sam list and their corresponding help messages * Added no-args handling to base commands and added files for integration test suite * Made additions to resources integration tests * Made additions to the sam list integration test suite * Added to sam list integration and unit tests * Added integration tests to test help messages of sam list commands * Reformatted files * Cleaned up unfinished tests * adding check to see what the appveyor test will produce. Trying to resolve test failure * Fixed test to check help messages * modified pattern matching for eliminating newlines when matching help message * Changed the way whitespaces are handled in matching help messages * Addressed PR comments, moved items into common classes * Made modifications based on comments, removed relative import paths, added method annotations, fixed text formatting * Reformatted files * removed folder deletion * removed uneccessary folder creation and deletion * fixed errors with cwd of integration tests * Added implementation and tests for the stack-outputs command * Added test skips for integration tests, added unit tests, removed redundant init_client call * commit to retrigger appveyor tests * Commmit to trigger appveyor * Modified client source, made fixes based on comments * Made fixes based on comments * Combined get_stack_info and stack_exists, and modified unit tests * Empty-Commit * Empty-Commit * Empty-Commit * fixed tests based on comments * reformatted file * Refactored stack outputs command to the producer mapper consumer design pattern * Fixed formatting * Moved interfaces, made changes based on comments * Made fixes based on comments * Made fixes based on comments * Empty commit * Made changes based on comments, added new exceptions * Fixed format * Fixed return type declaration * Fixed return type declaration * Changed return type to list * Fixed error * Implementation of the local transform and resource collection * Empty-Commit * Added section to avoid unused variable * Refactored common code * Added tests, modified PR * Fixed formatting * Made fixes based on PR comments * Fixed formatting * Fixed typing errors * Reverted typing * Fixed error with typing * Made changes to handling optional params * Fixes to typing errors * Made edits based on comments * Fixed error * Changed return type * Reverted return type due to make pr error * Added change to fix make pr error * Removed translate_utils.py file * Added cloud resources to sam list resources output * Empty commit * modified test format * Modified tests * Modified test * Adding the sam list testable resources command, tests, and table output format support for all sam list commands * Changed table and made changes based on pr comments * Fixed integration test expected outputs * Fixed table heading * Added docstring and re-arranged the testable resources producer to reduce if-elses within a single function * Fixed format Co-authored-by: Andrew Zhan <zhandr@amazon.com> * Renaming the 'testable-resources' command to 'endpoints' (#4116) * Added the base commands of sam list and their corresponding help messages * Added no-args handling to base commands and added files for integration test suite * Made additions to resources integration tests * Made additions to the sam list integration test suite * Added to sam list integration and unit tests * Added integration tests to test help messages of sam list commands * Reformatted files * Cleaned up unfinished tests * adding check to see what the appveyor test will produce. Trying to resolve test failure * Fixed test to check help messages * modified pattern matching for eliminating newlines when matching help message * Changed the way whitespaces are handled in matching help messages * Addressed PR comments, moved items into common classes * Made modifications based on comments, removed relative import paths, added method annotations, fixed text formatting * Reformatted files * removed folder deletion * removed uneccessary folder creation and deletion * fixed errors with cwd of integration tests * Added implementation and tests for the stack-outputs command * Added test skips for integration tests, added unit tests, removed redundant init_client call * commit to retrigger appveyor tests * Commmit to trigger appveyor * Modified client source, made fixes based on comments * Made fixes based on comments * Combined get_stack_info and stack_exists, and modified unit tests * Empty-Commit * Empty-Commit * Empty-Commit * fixed tests based on comments * reformatted file * Refactored stack outputs command to the producer mapper consumer design pattern * Fixed formatting * Moved interfaces, made changes based on comments * Made fixes based on comments * Made fixes based on comments * Empty commit * Made changes based on comments, added new exceptions * Fixed format * Fixed return type declaration * Fixed return type declaration * Changed return type to list * Fixed error * Implementation of the local transform and resource collection * Empty-Commit * Added section to avoid unused variable * Refactored common code * Added tests, modified PR * Fixed formatting * Made fixes based on PR comments * Fixed formatting * Fixed typing errors * Reverted typing * Fixed error with typing * Made changes to handling optional params * Fixes to typing errors * Made edits based on comments * Fixed error * Changed return type * Reverted return type due to make pr error * Added change to fix make pr error * Removed translate_utils.py file * Added cloud resources to sam list resources output * Empty commit * modified test format * Modified tests * Modified test * Adding the sam list testable resources command, tests, and table output format support for all sam list commands * Changed table and made changes based on pr comments * Fixed integration test expected outputs * Fixed table heading * Added docstring and re-arranged the testable resources producer to reduce if-elses within a single function * Fixed format * Renamed command from testable resources to endpoints Co-authored-by: Andrew Zhan <zhandr@amazon.com> * Cleanup integration tests * Cleanup tests, address comments * Move boto3 imports, update unit tests * Add comments, use constants for resources * Update unit test mocking type * Add missing parameters * Add additional comments * Fix spelling, minor updates Co-authored-by: Mehmet Nuri Deveci <5735811+mndeveci@users.noreply.github.com> Co-authored-by: andrewzhan <andrewzhan8@gmail.com> Co-authored-by: Andrew Zhan <zhandr@amazon.com> * feat: Add warning about not providing stack name option (#4624) * fix: Fix failing list tests on Windows (#4623) * fix: Fix failing list tests on Windows * Black reformat * Add event tracking for sam validate --lint metrics (#4612) * Update lint helpand output message * Add event tracking for sam validate --lint metrics * Add unit test for tracking --------- Co-authored-by: Sriram Madapusi Vasudevan <3770774+sriram-mv@users.noreply.github.com> Co-authored-by: Qingchuan Ma <69653965+qingchm@users.noreply.github.com> Co-authored-by: Mehmet Nuri Deveci <5735811+mndeveci@users.noreply.github.com> * Use safe yaml parse in list producer (#4632) * Revert an integration test change related to permission change revert (#4633) --------- Co-authored-by: Daniel Mil <84205762+mildaniel@users.noreply.github.com> Co-authored-by: Mehmet Nuri Deveci <5735811+mndeveci@users.noreply.github.com> Co-authored-by: andrewzhan <andrewzhan8@gmail.com> Co-authored-by: Andrew Zhan <zhandr@amazon.com> Co-authored-by: David <114027923+cdavidxu-hub@users.noreply.github.com> Co-authored-by: Sriram Madapusi Vasudevan <3770774+sriram-mv@users.noreply.github.com> Co-authored-by: Qingchuan Ma <69653965+qingchm@users.noreply.github.com> * feat: Collect Lambda authorizers in swagger definition (#4641) * Initial suppport to gather lambda authorizers in swagger * Added more unit tests * Made it clear that tests are Lambda auth related * Added function docstring * Added missed case where identity sources differ depending on API Gateway version * Changed some values to constants * Addressed comments * Added empty check to other security definition check * Updated log messages to change some info to warnings, and no auth info to debug * feat: Collect Lambda Authorizers found in Cloudformation resources (#4668) * Initial suppport to gather lambda authorizers in swagger * Added more unit tests * Made it clear that tests are Lambda auth related * Added function docstring * Added missed case where identity sources differ depending on API Gateway version * Changed some values to constants * Addressed comments * Added empty check to other security definition check * Added collection of Lambda authorizers for CFN resources * Addressed comments by moving validation logic to its own methods * feat: Collect Lambda Authorizers under the Auth property for Serverless resources (#4654) * Initial suppport to gather lambda authorizers in swagger * Added more unit tests * Made it clear that tests are Lambda auth related * Added function docstring * Added missed case where identity sources differ depending on API Gateway version * Added parsing Authorizers inside of Auth properties for Serverless resources * Added unit tests * Added test for the HTTP API extraction method * Changed some values to constants * Changed some variables to constants * Addressed comments * Added empty check to other security definition check * Correctly name identity sources * Addressed some comments * Changed LOGs to debugs to reduce spam --------- Co-authored-by: Sriram Madapusi Vasudevan <3770774+sriram-mv@users.noreply.github.com> * feat: Added identity source validation and removed empty string state (#4683) * Added identity source validation and removed empty string state * make black reformat * Moved identity source validator into validators folder * Fixed linting errors * Run make black * Compile regular expressions * feat: Added identity source validation in request handling (#4762) * Added identity source validation in request handling * Removed call to create flask app * Added context to id validator * Add route type check for operation_name * Convert to dictionary to avoid typing issue * Addressed comments and moved Route class to it's own module * feat: Event construction refactor (#4798) * Refactored LocalApigwService by moving some event generation logic out * Cleaned operation name generation * Addressed comments * feat: Event constructors for authorizers (#4807) * Refactored LocalApigwService by moving some event generation logic out * Cleaned operation name generation * Addressed comments * Added tests * Fixed typing for identity getter functions * feat: Invoke Lambda authorizer (#4840) * Added initial invocation logic for Lambda authorizer * Added response validation * Removed unused method * Added tests * Updated principalId get logic * Changed typing to be correct * Addressed comments * Format * Moved lambda auth invocation to its own method * Addressed comments * feat: Integration testing of local Lambda authorizers in serverless properties (#4872) * Added initial invocation logic for Lambda authorizer * Added response validation * Removed unused method * Added tests * Updated principalId get logic * Changed typing to be correct * Addressed comments * Format * Moved lambda auth invocation to its own method * Addressed comments * Initial integration testing setup and work * Change test class name and add comment to make it more clear that this is a bad test case * make black * feat: Validate headers against validation expression property (#4910) * Added identity validation expression check * Added additional test cases * fix: Fix APIGW V2 context passing (#4916) * Updated context passing logic to consider V2 payloads * Addressed comments by adding checking for API event * feat: Integration testing of Lambda authorizers defined as CFN resources (#4917) * Added identity validation expression check * Updated context passing logic to consider V2 payloads * Initial CFN authorizer resource testing * Added tests to validate template validation * fix: Added missing checks for Swagger parsing of Lambda authorizers and fixed some existing ones (#4938) * Added missing check for simple responses and fixed validation string check * Added missing check for simple responses * Added checks for payload version * make format * Addressed comments * feat: Integration testing of Lambda authorizers defined under the Swagger document (#4939) * Added identity validation expression check * Updated context passing logic to consider V2 payloads * Added missing check for simple responses and fixed validation string check * Initial swagger parsing integration testing * Added missing check for simple responses * Added checks for payload version * make format * Added template validation for swagger * make format * fix: Only print console message if enableSimpleResponses is defined (#4994) * feat: Add usage disclaimer when starting API with authorizers (#4968) * Added disclaimer for authorizer usage and updated log message for undefined authorizers * Populated message * Updated message * Updated message * feat: Add metrics for using the Lambda authorizer feature (#4942) * Added event tracking for using Lambda authorizers * Added and update unit tests * Moved event tracker to after invocation logic and added session ID passing * Updated event tracker to accept exceptions * Added tests * Updated doc string to include exception message * Addressed comments and removed old test * Added exception to __repr__ and __eq__ * chore: Removed old test file (#5006) --------- Co-authored-by: Daniel Mil <84205762+mildaniel@users.noreply.github.com> Co-authored-by: Mehmet Nuri Deveci <5735811+mndeveci@users.noreply.github.com> Co-authored-by: andrewzhan <andrewzhan8@gmail.com> Co-authored-by: Andrew Zhan <zhandr@amazon.com> Co-authored-by: David <114027923+cdavidxu-hub@users.noreply.github.com> Co-authored-by: Sriram Madapusi Vasudevan <3770774+sriram-mv@users.noreply.github.com> Co-authored-by: Qingchuan Ma <69653965+qingchm@users.noreply.github.com>

Andrew Zhan added 30 commits May 19, 2022 14:53

Added the base commands of sam list and their corresponding help mess…

d964112

…ages

Added no-args handling to base commands and added files for integrati…

5cc31a5

…on test suite

Made additions to resources integration tests

70b3ba8

Made additions to the sam list integration test suite

fc81ceb

Added to sam list integration and unit tests

97ec32d

Added integration tests to test help messages of sam list commands

3e200f2

Reformatted files

bcdf40b

Cleaned up unfinished tests

93f55cd

adding check to see what the appveyor test will produce. Trying to re…

60915f1

…solve test failure

Fixed test to check help messages

a5418f6

modified pattern matching for eliminating newlines when matching help…

a3d150a

… message

Changed the way whitespaces are handled in matching help messages

5dff9e2

Addressed PR comments, moved items into common classes

bafaea2

Made modifications based on comments, removed relative import paths, …

8e084ff

…added method annotations, fixed text formatting

Reformatted files

719cfd5

removed folder deletion

c7499e4

removed uneccessary folder creation and deletion

ee2015a

fixed errors with cwd of integration tests

2c62cd9

Added implementation and tests for the stack-outputs command

de4439b

Merge branch 'list-command' into base-command

ef1f273

Added test skips for integration tests, added unit tests, removed red…

08bf419

…undant init_client call

commit to retrigger appveyor tests

1358e61

Commmit to trigger appveyor

cf5c191

Modified client source, made fixes based on comments

3aa02da

Made fixes based on comments

5bc4247

Combined get_stack_info and stack_exists, and modified unit tests

ae4e2f0

Empty-Commit

38fc5f0

Empty-Commit

86166a7

Empty-Commit

3c8f6a1

fixed tests based on comments

3bf1d89

github-actions Bot added the stage/needs-triage Automatically applied to new issues and PRs, indicating they haven't been looked at. label Jun 30, 2022

andrew-zhan139 and others added 6 commits June 29, 2022 17:43

Merge branch 'list-command' into base-command

6d35129

Added section to avoid unused variable

defa233

Merge branch 'base-command' of github.com:andrew-zhan139/aws-sam-cli …

282c914

…into base-command

Refactored common code

e98ac15

Added tests, modified PR

562d2ad

Fixed formatting

c5d91eb

torresxb1 self-requested a review July 4, 2022 22:17

torresxb1 suggested changes Jul 6, 2022

View reviewed changes

Andrew Zhan added 7 commits July 6, 2022 18:55

Made fixes based on PR comments

7fd3179

Fixed formatting

8d9a798

Fixed typing errors

91a06a3

Reverted typing

3479342

Fixed error with typing

da1eda7

Made changes to handling optional params

4cd1506

Fixes to typing errors

0851eb3

mildaniel suggested changes Jul 8, 2022

View reviewed changes

Andrew Zhan added 6 commits July 10, 2022 22:48

Made edits based on comments

9de6afa

Fixed error

26fb5f0

Changed return type

13c7edb

Reverted return type due to make pr error

e7c17d2

Added change to fix make pr error

9ab5674

Removed translate_utils.py file

c105195

mildaniel approved these changes Jul 13, 2022

View reviewed changes

torresxb1 self-requested a review July 13, 2022 19:00

torresxb1 approved these changes Jul 13, 2022

View reviewed changes

torresxb1 merged commit 026647b into aws:list-command Jul 13, 2022

		LOG = logging.getLogger(__name__)


		def _read_sam_file(template):

		template_dict_format: Optional[Dict]
		Optional dictionary representing the sam template file to be used instead of the template file

Uh oh!

Conversation

andrew-zhan139 commented Jun 30, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue(s) does this change fix?

Why is this change necessary?

How does it address the issue?

What side effects does this change have?

Mandatory Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

torresxb1 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

andrew-zhan139 commented Jun 30, 2022 •

edited

Loading