Concept Shippable Response Plans

[xqi-splunk]

Details

This PR adds response templates to security_content repo:

1. response_templates/*.json: Response templates provided by the response templates team
2. .github/workflows/response_templates/template_script.py: Script provided by the response templates team, which provides functionality merged different versions response templates and generate manifest file of response templates.
3. .github/workflows/response_templates/validate_response_templates.py: Script to validate response templates using Draft7Validator with provided OpenAPI spec from the response templates team .github/workflows/response_templates/mcopenapi_public.yaml
4. .github/workflows/build.yml: Add upload of dist/api.
5. .github/workflows/build-response-templates.yml: Workflow to build merged response templates and validate response templates on push/PR.

Checklist

• Validate name matches <platform>_<mitre att&ck technique>_<short description> nomenclature
• CI/CD jobs passed ✔️
• Validated SPL logic.
• Validated tags, description, and how to implement.
• Verified references match analytic.
• Confirm updates to lookups are handled properly.

The update doesn't have impact to detections related. Below is the screenshot of build-response-templates workflow:

Notes For Submitters and Reviewers

• If you're submitting a PR from a fork, ensuring the box to allow updates from maintainers is checked will help speed up the process of getting it merged.
• Checking the output of the build CI job when it fails will likely show an error about what is failing. You may have a very descriptive error of the specific field(s) in the specific file(s) that is causing an issue. In some cases, its also possible there is an issue with the YAML. Many of these can be caught with the pre-commit hooks if you set them up. These errors will be less descriptive as to what exactly is wrong, but will give you a column and row position in a specific file where the YAML processing breaks. If you're having trouble with this, feel free to add a comment to your PR tagging one of the maintainers and we'll be happy to help troubleshoot it.
• Updates to existing lookup files can be tricky, because of how Splunk handles application updates and the differences between existing lookup files being updated vs new lookups. You can read more here but the short version is that any changes to lookup files need to bump the the date and version in the associated YAML file.

[pyth0n1c]

I would not mix the python files and schema files in the same directory. Inside of the response_templates directory, I would probably have:

merged_response_templates/ - exactly as it is today
either in merged_response_templates/bin or .github/workflows/response_templates/ - template_script.py, validate_response_templates.py, mcopenapi_public.yml (I would convert this to yml not yaml, for consistency with the rest of the security_content repo, even if what the team provided was yaml)
I make this recommendation as there is no other "code" in the security_content repo other than the code which lives in the .github folder for use during CI/CD

[xqi-splunk]

I would not mix the python files and schema files in the same directory. Inside of the response_templates directory, I would probably have:

merged_response_templates/ - exactly as it is today either in merged_response_templates/bin or .github/workflows/response_templates/ - template_script.py, validate_response_templates.py, mcopenapi_public.yml (I would convert this to yml not yaml, for consistency with the rest of the security_content repo, even if what the team provided was yaml) I make this recommendation as there is no other "code" in the security_content repo other than the code which lives in the .github folder for use during CI/CD

Moved template_script.py, validate_response_templates.py, mcopenapi_public.yml to .github/workflows/response_templates/ and updated mcopenapi_public.yaml to mcopenapi_public.yml for consistency.

[pyth0n1c]

A little more feedback around organization and fixing a pre-existing deficiency in the build.yml workflow.

[xqi-splunk]

Tested security_content gitlab fetching from this feature branch: https://cd.splunkdev.com/threat-research/security_content/-/pipelines/32509507

The build_response_templates job passed in the above link.

Appropriate changes have been made in a few different iterations.
Dismissing my review.