You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[AWS Bedrock Agentcore] Add support for code_interpreter_application_logs and runtime_logs events - #22209
The reason will be displayed to describe this comment to others. Learn more.
🟡 Changes recommended
Unresolved issues include multiline log truncation, duplicate YAML keys that can break policy rendering, changelog links that fail validation, and missing fixtures for new parsing paths.
Extends the Amazon Bedrock AgentCore integration with Code Interpreter application logs and runtime container logs, providing code-execution telemetry needed for the detections described in #19936.
Changes:
Adds two data streams with input configuration, field mappings, and ingest pipelines for commands, execution results, and runtime messages.
Adds pipeline fixtures, cloud-connector policy checks, and Terraform-backed system tests.
Documents the new datasets and updates the package to version 1.2.0.
Paths below are relative to packages/aws_bedrock_agentcore/.
File
Description
manifest.yml
Bumps the package version to 1.2.0.
docs/README.md
Documents datasets, example events, and exported fields.
Neither new data stream ships a pipeline-test test-common-config.yml. Add one to code_interpreter_application_logs and runtime_logs that sets the preserve_original_event tag.
Details
Each new stream's _dev/test/pipeline/ directory contains only its input fixture plus the generated output. There is no test-common-config.yml in either one. Every existing log stream in this package has one: gateway_application_logs, memory_application_logs and runtime_application_logs. Without it, the pipeline tests run without the preserve_original_event tag. The tests rubric treats a missing test-common-config.yml as HIGH.
The sibling configs also set preserve_duplicate_custom_fields, which is a prohibited pattern, so copy only the preserve_original_event tag. For runtime_logs, this file is also the natural place to inject aws.cloudwatch.log_group / aws.cloudwatch.log_stream. The default.yml branches read those fields, and nothing currently provides them in pipeline tests.
Recommendation:
Add _dev/test/pipeline/test-common-config.yml to each new data stream:
fields:
tags:
- preserve_original_event
For runtime_logs, you can also inject CloudWatch metadata here, or in per-fixture *-config.yml files:
The only runtime_logs fixture contains four InvokeAgentRuntimeCommand audit lines, so most of this pipeline is untested. Add fixtures for SDK JSON lines, legacy SDK and OTel text lines, error classification, and the log-group and log-stream branches in default.yml.
Details
pipeline-runtime-logs.yml has these parse paths:
SDK JSON lines: parse_json_line with its malformed-JSON on_failure, date_sdk_json_timestamp, the rename_sdk_json_* processors, and join_sdk_json_stack_trace.
The audit line: grok_invoke_command_audit_line.
Text lines: grok_text_line, with legacy SDK and OTel patterns. The OTel path also has the trace_id=0 / span_id=0 suppression.
Error classification: append_event_type_error.
default.yml also branches on aws.cloudwatch.log_group: the ARN vs name grok, plus grok_agent_runtime_id_and_endpoint. It branches on log.file.path too, and on aws.cloudwatch.log_stream for the runtime-logs / otel-rt-logs / spans stream types and dissect_runtime_session_id.
The only fixture, test-runtime-logs-sample.log, has 4 lines. All of them are [<epoch>] awsRequestId=... command=... audit lines, and no config injects log group or stream metadata. Every other branch therefore has no pipeline-test coverage. The system test also sends only audit lines.
Recommendation:
Add one fixture per scenario, each with a *-config.yml that injects aws.cloudwatch.log_group / aws.cloudwatch.log_stream:
SDK RequestContextFormatter JSON lines. Include one with errorType/errorMessage/stackTrace, and one malformed line that starts with {.
A legacy <ts> - <logger> - <LEVEL> - [rid:..] [sid:..] msg line, an OTel text line with real trace and span IDs, a trace_id=0 line, and a free-text line.
Records whose log_stream is otel-rt-logs and spans.
The log group in both ARN form and name form.
Then regenerate the expected output with elastic-package test pipeline --data-streams runtime_logs --generate.
The new aws-s3 template renders buffer_size (lines 74 and 86) and fips_enabled (lines 89 and 149) twice. If a user sets either var, Fleet rejects the policy with a duplicate-key error. Remove the second buffer_size block and one of the fips_enabled blocks.
Details
buffer_size (text) and fips_enabled (bool) are both declared as aws-s3 stream vars in code_interpreter_application_logs/manifest.yml (lines 195 and 236), so users can set them as advanced options. The template has two separate {{#if buffer_size}} blocks (lines 74-76 and 86-88) and two {{#if fips_enabled}} blocks (lines 89-91 and 149-151). Setting either var therefore renders the same top-level key twice.
Fleet's compileTemplate (fleet/server/services/epm/agent/agent.ts) parses the rendered template with the yaml package. That package raises DUPLICATE_KEY, which handleYamlError turns into a PackagePolicyValidationError (Duplicated key ... found in agent policy yaml). So a user enabling S3 FIPS, or tuning the buffer size, cannot save the policy.
The existing gateway, memory and runtime application-log aws-s3 templates have the same duplication. Those files are outside this PR, but the new template copies the defect.
Recommendation:
Keep a single block for each key in aws-s3.yml.hbs. For example, delete lines 86-88 (the second buffer_size) and lines 149-151 (the second fips_enabled), keeping:
code_interpreter_application_logs does not declare aws.cloudwatch.log_group, log_stream or ingestion_time, which the aws-cloudwatch input adds to every event. Declare them, as runtime_logs does, so the new aws-cloudwatch system test passes field validation.
Details
Filebeat's aws-cloudwatch input (x-pack/filebeat/input/awscloudwatch/processor.go, createEvent) adds aws.cloudwatch.log_group, aws.cloudwatch.log_stream and aws.cloudwatch.ingestion_time (a date) to every event.
The code_interpreter_application_logs field files declare none of these. fields/input.yml declares only aws.cloudwatch.message, and fields/fields.yml covers only aws.bedrock_agentcore.*. The pipeline does not remove the three fields, and the package has no validation.yml. The new _dev/test/system/test-default-config.yml runs this stream with input: aws-cloudwatch and expects 10 hits. elastic-package should therefore report these fields as undefined.
The new runtime_logs stream in this PR declares exactly these fields (runtime_logs/fields/fields.yml lines 1-12). Without a declaration, ingestion_time and log_group would also be mapped dynamically rather than as date and keyword. CI results were not checked.
Recommendation:
Declare the CloudWatch fields the input emits in code_interpreter_application_logs/fields/input.yml, matching runtime_logs:
- name: aws.cloudwatchtype: groupfields:
- name: messagetype: textdescription: | CloudWatch log message.
- name: log_grouptype: keyworddescription: The CloudWatch log group the event belongs to, as a name or an ARN depending on how the input was configured.
- name: log_streamtype: keyworddescription: Name of the CloudWatch log stream the event was read from.
- name: ingestion_timetype: datedescription: Time the event was ingested by CloudWatch Logs.
- name: log.offsettype: longdescription: Log offset
{"resource_arn":"arn:aws:bedrock-agentcore:us-east-1:111122223333:code-interpreter-custom/dataAnalysisSandbox-aB3dE5fG7h","event_timestamp":1766409783000,"account_id":"111122223333","request_id":"b1c2d3e4-f5a6-4b7c-8d9e-0f1a2b3c4d5e","tool_session_id":"01K8ZQ5R3Y8V2M6N9P4T7W1X0A","span_id":"1234567890abcdef","trace_id":"abcdefabcdefabcdefabcdefabcdef12","service_name":"AgentCoreCodeInterpreter","operation":"InvokeCodeInterpreter","resource":{"attributes":{"cloud.provider":"aws","service.name":"dataAnalysisSandbox-aB3dE5fG7h","cloud.resource_id":"arn:aws:bedrock-agentcore:us-east-1:111122223333:code-interpreter-custom/dataAnalysisSandbox-aB3dE5fG7h","cloud.platform":"aws_bedrock_agentcore"}},"attributes":{"aws.operation.name":"InvokeCodeInterpreter","aws.request.id":"b1c2d3e4-f5a6-4b7c-8d9e-0f1a2b3c4d5e","aws.account.id":"111122223333","aws.resource.type":"AWS::BedrockAgentCore::CodeInterpreterCustom","toolsession.id":"01K8ZQ5R3Y8V2M6N9P4T7W1X0A"},"timeUnixNano":1766409782800000777,"severityNumber":9,"severityText":"INFO","body":{"request_payload":{"name":"removeFiles","arguments":{"paths":["sales_data.csv"]}},"response_payload":{"content":[{"type":"text","text":"Successfully removed all 1 files"}],"isError":false}},"traceId":"abcdefabcdefabcdefabcdefabcdef12","spanId":"1234567890abcdef"}
{"resource_arn":"arn:aws:bedrock-agentcore:us-east-1:111122223333:code-interpreter-custom/dataAnalysisSandbox-aB3dE5fG7h","event_timestamp":1766409840120,"account_id":"111122223333","request_id":"c7d8e9f0-a1b2-4c3d-9e4f-5a6b7c8d9e0f","tool_session_id":"01K8ZQ5R3Y8V2M6N9P4T7W1X0A","span_id":"aabbccdd00112233","trace_id":"77889900aabbccddeeff001122334455","service_name":"AgentCoreCodeInterpreter","operation":"InvokeCodeInterpreter","resource":{"attributes":{"cloud.provider":"aws","service.name":"dataAnalysisSandbox-aB3dE5fG7h","cloud.resource_id":"arn:aws:bedrock-agentcore:us-east-1:111122223333:code-interpreter-custom/dataAnalysisSandbox-aB3dE5fG7h","cloud.platform":"aws_bedrock_agentcore"}},"attributes":{"aws.operation.name":"InvokeCodeInterpreter","aws.request.id":"c7d8e9f0-a1b2-4c3d-9e4f-5a6b7c8d9e0f","aws.account.id":"111122223333","aws.resource.type":"AWS::BedrockAgentCore::CodeInterpreterCustom","toolsession.id":"01K8ZQ5R3Y8V2M6N9P4T7W1X0A"},"timeUnixNano":1766409840000000010,"severityNumber":9,"severityText":"INFO","body":{"request_payload":{"name":"startCommandExecution","arguments":{"command":"sleep 15 && echo Task completed successfully"}},"response_payload":{"content":[{"type":"text","text":"Task started with id task-3f9a1c2b"}],"structuredContent":{"taskId":"task-3f9a1c2b","taskStatus":"submitted"},"isError":false}},"traceId":"77889900aabbccddeeff001122334455","spanId":"aabbccdd00112233"}
The code_interpreter fixture misses several pipeline branches: stopTask, the arguments.path and arguments.runtime renames, an unmapped tool name, and a non-JSON line. Add those cases, and give line 10 a unique request_id.
Details
The fixture uses 8 of the 9 tool names mapped by set_ecs_categorization. stopTask (process / end) is never exercised.
No line carries arguments.path or arguments.runtime, so rename_arguments_path_to_file_path and rename_arguments_runtime are never reached. The script's early return for an unmapped tool name is also untested.
parse_json_event_original has no ignore_failure, unlike the sibling streams. A malformed line would therefore go through the pipeline-level on_failure (event.kind: pipeline_error), but no fixture covers that path.
Line 10 reuses line 3's request_id value.
Recommendation:
Append these lines to the fixture:
a stopTask request
a request whose arguments include path and runtime
an unrecognised tool name
a truncated or non-JSON line
Give line 10 a unique request ID, then regenerate the expected output with elastic-package test pipeline --data-streams code_interpreter_application_logs --generate.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Duplicate buffer_size and fips_enabled keys cause strict YAML parsing to reject affected configurations, while newline-unsafe log patterns can lose command and traceback content or reject valid events.
The new runtime pipeline fixtures contain only four command-audit lines, so none exercises the SDK JSON branch or the legacy/OTel text parsers. The raw .log fixtures also omit the CloudWatch metadata needed to exercise runtime/session extraction and stream routing in default.yml. Please add JSON-event fixtures with representative SDK/text messages and log group/stream fields, including error/stack-trace handling and an OTel stream that should stay raw.
{"resource_arn":"arn:aws:bedrock-agentcore:us-east-1:111122223333:code-interpreter-custom/dataAnalysisSandbox-aB3dE5fG7h","event_timestamp":1766409783000,"account_id":"111122223333","request_id":"b1c2d3e4-f5a6-4b7c-8d9e-0f1a2b3c4d5e","tool_session_id":"01K8ZQ5R3Y8V2M6N9P4T7W1X0A","span_id":"1234567890abcdef","trace_id":"abcdefabcdefabcdefabcdefabcdef12","service_name":"AgentCoreCodeInterpreter","operation":"InvokeCodeInterpreter","resource":{"attributes":{"cloud.provider":"aws","service.name":"dataAnalysisSandbox-aB3dE5fG7h","cloud.resource_id":"arn:aws:bedrock-agentcore:us-east-1:111122223333:code-interpreter-custom/dataAnalysisSandbox-aB3dE5fG7h","cloud.platform":"aws_bedrock_agentcore"}},"attributes":{"aws.operation.name":"InvokeCodeInterpreter","aws.request.id":"b1c2d3e4-f5a6-4b7c-8d9e-0f1a2b3c4d5e","aws.account.id":"111122223333","aws.resource.type":"AWS::BedrockAgentCore::CodeInterpreterCustom","toolsession.id":"01K8ZQ5R3Y8V2M6N9P4T7W1X0A"},"timeUnixNano":1766409782800000777,"severityNumber":9,"severityText":"INFO","body":{"request_payload":{"name":"removeFiles","arguments":{"paths":["sales_data.csv"]}},"response_payload":{"content":[{"type":"text","text":"Successfully removed all 1 files"}],"isError":false}},"traceId":"abcdefabcdefabcdefabcdefabcdef12","spanId":"1234567890abcdef"}
{"resource_arn":"arn:aws:bedrock-agentcore:us-east-1:111122223333:code-interpreter-custom/dataAnalysisSandbox-aB3dE5fG7h","event_timestamp":1766409840120,"account_id":"111122223333","request_id":"c7d8e9f0-a1b2-4c3d-9e4f-5a6b7c8d9e0f","tool_session_id":"01K8ZQ5R3Y8V2M6N9P4T7W1X0A","span_id":"aabbccdd00112233","trace_id":"77889900aabbccddeeff001122334455","service_name":"AgentCoreCodeInterpreter","operation":"InvokeCodeInterpreter","resource":{"attributes":{"cloud.provider":"aws","service.name":"dataAnalysisSandbox-aB3dE5fG7h","cloud.resource_id":"arn:aws:bedrock-agentcore:us-east-1:111122223333:code-interpreter-custom/dataAnalysisSandbox-aB3dE5fG7h","cloud.platform":"aws_bedrock_agentcore"}},"attributes":{"aws.operation.name":"InvokeCodeInterpreter","aws.request.id":"c7d8e9f0-a1b2-4c3d-9e4f-5a6b7c8d9e0f","aws.account.id":"111122223333","aws.resource.type":"AWS::BedrockAgentCore::CodeInterpreterCustom","toolsession.id":"01K8ZQ5R3Y8V2M6N9P4T7W1X0A"},"timeUnixNano":1766409840000000010,"severityNumber":9,"severityText":"INFO","body":{"request_payload":{"name":"startCommandExecution","arguments":{"command":"sleep 15 && echo Task completed successfully"}},"response_payload":{"content":[{"type":"text","text":"Task started with id task-3f9a1c2b"}],"structuredContent":{"taskId":"task-3f9a1c2b","taskStatus":"submitted"},"isError":false}},"traceId":"77889900aabbccddeeff001122334455","spanId":"aabbccdd00112233"}
The code_interpreter system-test sample now has 13 events, but _dev/test/system/test-default-config.yml still asserts hit_count: 10. Change the assertion to 13.
Details
After this push, _dev/deploy/tf/files/code-interpreter-sample.log has 13 non-empty lines. The new lines are the getTask, IMDS-probe, stopTask, path/runtime-arguments and unknown-tool events (lines 9-13). main.tf builds code_interpreter_lines from every non-empty line of that file and sends them all in a single aws logs put-log-events call, so all 13 reach CloudWatch together. The code_interpreter ingest pipeline has no drop processor, so every event is indexed. _dev/test/system/test-default-config.yml was not touched in this push and still says hit_count: 10 on line 15. elastic-package compares the asserted hit count to the observed count exactly, so this data stream's system test will fail with an observed/expected hit-count mismatch. The runtime_logs system test is consistent (hit_count: 4 for its 4-line sample).
Recommendation:
Update the assertion in code_interpreter_application_logs/_dev/test/system/test-default-config.yml so it matches the 13 lines in the tf sample file:
assert:
hit_count: 13
Also in:packages/aws_bedrock_agentcore/data_stream/code_interpreter_application_logs/_dev/test/system/test-default-config.yml:15
This fixture puts a log-group name in aws.cloudwatch.log_group and an ARN in log.file.path. The aws-cloudwatch input never emits that combination, so the test only covers an unreachable path. Derive both fields from the same log-group identifier.
Details
In the Beats aws-cloudwatch input (x-pack/filebeat/input/awscloudwatch/processor.go, createEvent), both fields come from the same logGroupId. aws.cloudwatch.log_group is set to logGroupId, and log.file.path is set to logGroupId + "/" + logStreamName. When the log group is configured by name, log.file.path therefore starts with that name and never with an ARN. This fixture sets log_group to a name but log.file.path to an arn:aws:logs:...:log-group:... value. Its path suffix (.../2026/07/01/spans) also does not match the log_stream of spans. This combination is the only thing that makes the grok_log_file_path_account fallback in runtime_logs/elasticsearch/ingest_pipeline/default.yml fill cloud.account.id and the region. Real name-mode events never take that path: they get no account ID from either grok. No fixture covers that real outcome.
Recommendation:
Make the fixture match what the input actually emits in name mode, then regenerate the expected output with elastic-package test pipeline --data-streams runtime_logs --generate:
If the log.file.path ARN fallback is meant for a source other than the aws-cloudwatch input, name that source in the processor description. Otherwise, consider removing the fallback from default.yml.
The log-file-path fixture tests a pipeline branch that no longer exists. Nothing in the runtime_logs pipelines reads log.file.path now, so this fixture repeats the spans scenario. Remove it, or rename it after the case it actually covers.
Details
Prior finding 9bcf34835b5df7d5 suggested either making this fixture match what the input emits or removing the log.file.path fallback. The latest push did both: the fallback is gone and the config now sets a name-mode log.file.path. No file under packages/aws_bedrock_agentcore that is not a generated output mentions log.file.path any more. Without generated outputs, the only match in that tree is an unrelated file.path rename in the code_interpreter pipeline. The config now injects the same aws.cloudwatch.log_group and log_stream: spans as test-runtime-logs-spans-sample.log-config.yml. The extra log.file.path value is never read. The fixture line is free text, which the spans branch already passes through as raw message. The test's name now refers to a code path that does not exist, and it adds no distinct coverage.
Recommendation:
Delete test-runtime-logs-log-file-path-sample.log and its config and expected file. If you want to keep the free-text-on-a-spans-stream case, move the line into test-runtime-logs-spans-sample.log and drop the unused log.file.path block. Then regenerate with elastic-package test pipeline --data-streams runtime_logs --generate.
🟢 Reviewed the latest commits 72bc529 — nothing new beyond already posted comments.
⚠️ 8 comments still unresolved from earlier commits — 2 high, 3 medium, 3 low
🟠 The code_interpreter system-test sample now has 13 events, but _dev/test/system/test-default-config.yml still asserts hit_count: 10. Change the assertion to 13. (link)
🟠 Neither new data stream ships a pipeline-test test-common-config.yml. Add one to code_interpreter_application_logs and runtime_logs that sets the preserve_original_event tag. (link)
🟡 The only runtime_logs fixture contains four InvokeAgentRuntimeCommand audit lines, so most of this pipeline is untested. Add fixtures for SDK JSON lines, legacy SDK and OTel text lines, error classification, and the log-group and log-stream branches in default.yml. (link)
🟡 The new aws-s3 template renders buffer_size (lines 74 and 86) and fips_enabled (lines 89 and 149) twice. If a user sets either var, Fleet rejects the policy with a duplicate-key error. Remove the second buffer_size block and one of the fips_enabled blocks. (link)
🟡 code_interpreter_application_logs does not declare aws.cloudwatch.log_group, log_stream or ingestion_time, which the aws-cloudwatch input adds to every event. Declare them, as runtime_logs does, so the new aws-cloudwatch system test passes field validation. (link)
🔵 The log-file-path fixture tests a pipeline branch that no longer exists. Nothing in the runtime_logs pipelines reads log.file.path now, so this fixture repeats the spans scenario. Remove it, or rename it after the case it actually covers. (link)
🔵 This fixture puts a log-group name in aws.cloudwatch.log_group and an ARN in log.file.path. The aws-cloudwatch input never emits that combination, so the test only covers an unreachable path. Derive both fields from the same log-group identifier. (link)
🔵 The code_interpreter fixture misses several pipeline branches: stopTask, the arguments.path and arguments.runtime renames, an unmapped tool name, and a non-JSON line. Add those cases, and give line 10 a unique request_id. (link)
Review summary
Issues found across earlier commits afd965e — 1 low
🔵 The log-file-path fixture tests a pipeline branch that no longer exists. Nothing in the runtime_logs pipelines reads log.file.path now, so this fixture repeats the spans scenario. Remove it, or rename it after the case it actually covers. (link) (Outdated)
Issues found across earlier commits 701950e — 1 high, 1 low
🟠 The code_interpreter system-test sample now has 13 events, but _dev/test/system/test-default-config.yml still asserts hit_count: 10. Change the assertion to 13. (link)
🔵 This fixture puts a log-group name in aws.cloudwatch.log_group and an ARN in log.file.path. The aws-cloudwatch input never emits that combination, so the test only covers an unreachable path. Derive both fields from the same log-group identifier. (link) (Outdated)
Issues found across earlier commits dcd9173 — 1 high, 3 medium, 1 low
🟠 Neither new data stream ships a pipeline-test test-common-config.yml. Add one to code_interpreter_application_logs and runtime_logs that sets the preserve_original_event tag. (link)
🟡 The only runtime_logs fixture contains four InvokeAgentRuntimeCommand audit lines, so most of this pipeline is untested. Add fixtures for SDK JSON lines, legacy SDK and OTel text lines, error classification, and the log-group and log-stream branches in default.yml. (link)
🟡 The new aws-s3 template renders buffer_size (lines 74 and 86) and fips_enabled (lines 89 and 149) twice. If a user sets either var, Fleet rejects the policy with a duplicate-key error. Remove the second buffer_size block and one of the fips_enabled blocks. (link) (Outdated)
🟡 code_interpreter_application_logs does not declare aws.cloudwatch.log_group, log_stream or ingestion_time, which the aws-cloudwatch input adds to every event. Declare them, as runtime_logs does, so the new aws-cloudwatch system test passes field validation. (link)
🔵 The code_interpreter fixture misses several pipeline branches: stopTask, the arguments.path and arguments.runtime renames, an unmapped tool name, and a non-JSON line. Add those cases, and give line 10 a unique request_id. (link) (Outdated)
A new commit triggers another review — at most once every 15 minutes. I skip the PR while it's approved or has merge conflicts.
@moxarth-rathod - This PR has merge conflicts with the base branch. Please resolve them.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
moxarth-rathod
added
enhancement
![vera-review-bot[bot]](https://avatars.githubusercontent.com/in/3297022?s=60&v=4){kind=small}
qcorporation
added
the
documentation
Proposed commit message
Checklist
changelog.ymlfile.Related issues