Skip to content

[axonius][application] Add Axonius Application datastream - #16658

Merged
ShourieG merged 19 commits into
elastic:feature/axonius-0.1.0from
muskan-agarwal26:datastream-application
May 27, 2026
Merged

ShourieG merged 19 commits into
elastic:feature/axonius-0.1.0from
muskan-agarwal26:datastream-application

Conversation

@muskan-agarwal26

@muskan-agarwal26 muskan-agarwal26 commented Dec 21, 2025 •

Copy link
Copy Markdown
Contributor

Proposed commit message

The release includes application data stream and associated dashboard.

Axonius fields are mapped to their corresponding ECS fields where possible.

Test samples were derived from live data samples, which were subsequently
sanitized.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

To test the axonius package:

  • Clone integrations repo.
  • Install elastic package locally.
  • Start elastic stack using elastic-package.
  • Move to integrations/packages/axonius directory.
  • Run the following command to run tests.

elastic-package test

2025/12/21 16:45:21  INFO New version is available - v0.117.1. Download from: https://github.com/elastic/elastic-package/releases/tag/v0.117.1
Run asset tests for the package
2025/12/21 16:45:21  INFO License text found in "/root/GITHUB/integrations/LICENSE.txt" will be included in package
--- Test results for package: axonius - START ---
╭─────────┬─────────────┬───────────┬──────────────────────────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME                                                        │ RESULT │ TIME ELAPSED │
├─────────┼─────────────┼───────────┼──────────────────────────────────────────────────────────────────┼────────┼──────────────┤
│ axonius │             │ asset     │ dashboard axonius-99a89250-dd1e-4d20-be86-c7079b5c7661 is loaded │ PASS   │      1.468µs │
│ axonius │             │ asset     │ search axonius-3be4bf2c-5043-45ae-958b-9d5355f34533 is loaded    │ PASS   │        289ns │
│ axonius │             │ asset     │ search axonius-bfd7962c-9bdf-488e-9302-67164b99a8fd is loaded    │ PASS   │        511ns │
│ axonius │ application │ asset     │ index_template logs-axonius.application is loaded                │ PASS   │        281ns │
│ axonius │ application │ asset     │ ingest_pipeline logs-axonius.application-0.1.0 is loaded         │ PASS   │        246ns │
╰─────────┴─────────────┴───────────┴──────────────────────────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: axonius - END   ---
Done
Run pipeline tests for the package
--- Test results for package: axonius - START ---
╭─────────┬─────────────┬───────────┬─────────────────────────────────────────────────┬────────┬──────────────╮
│ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME                                       │ RESULT │ TIME ELAPSED │
├─────────┼─────────────┼───────────┼─────────────────────────────────────────────────┼────────┼──────────────┤
│ axonius │ application │ pipeline  │ (ingest pipeline warnings test-application.log) │ PASS   │ 740.314026ms │
│ axonius │ application │ pipeline  │ test-application.log                            │ PASS   │ 448.345683ms │
╰─────────┴─────────────┴───────────┴─────────────────────────────────────────────────┴────────┴──────────────╯
--- Test results for package: axonius - END   ---
Done
Run policy tests for the package
--- Test results for package: axonius - START ---
No test results
--- Test results for package: axonius - END   ---
Done
Run static tests for the package
--- Test results for package: axonius - START ---
╭─────────┬─────────────┬───────────┬──────────────────────────┬────────┬──────────────╮
│ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME                │ RESULT │ TIME ELAPSED │
├─────────┼─────────────┼───────────┼──────────────────────────┼────────┼──────────────┤
│ axonius │ application │ static    │ Verify sample_event.json │ PASS   │ 272.198463ms │
╰─────────┴─────────────┴───────────┴──────────────────────────┴────────┴──────────────╯
--- Test results for package: axonius - END   ---
Done
Run system tests for the package
2025/12/21 16:45:31  INFO Installing package...
2025/12/21 16:45:31  INFO License text found in "/root/GITHUB/integrations/LICENSE.txt" will be included in package
2025/12/21 16:45:33  INFO Running test for data_stream "application" with configuration 'default'
2025/12/21 16:45:41  INFO Setting up independent Elastic Agent...
2025/12/21 16:47:27  INFO Setting up service...
2025/12/21 16:47:49  INFO Validating test case...
2025/12/21 16:47:51  INFO Tearing down service...
2025/12/21 16:47:52  INFO Write container logs to file: /root/GITHUB/integrations/build/container-logs/axonius-1766315872356940429.log
2025/12/21 16:47:55  INFO Tearing down agent...
2025/12/21 16:47:56  INFO Write container logs to file: /root/GITHUB/integrations/build/container-logs/elastic-agent-1766315876429410447.log
2025/12/21 16:49:36  INFO Uninstalling package...
--- Test results for package: axonius - START ---
╭─────────┬─────────────┬───────────┬───────────┬────────┬─────────────────╮
│ PACKAGE │ DATA STREAM │ TEST TYPE │ TEST NAME │ RESULT │    TIME ELAPSED │
├─────────┼─────────────┼───────────┼───────────┼────────┼─────────────────┤
│ axonius │ application │ system    │ default   │ PASS   │ 2m17.723061976s │
╰─────────┴─────────────┴───────────┴───────────┴────────┴─────────────────╯
--- Test results for package: axonius - END   ---
Done

Related issues

Screenshots

image (5) image (6)

@andrewkroh andrewkroh added documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. Integration:axonius Axonius Crest Contributions from Crest developement team. New Integration Issue or pull request for creating a new integration package. dashboard Relates to a Kibana dashboard bug, enhancement, or modification. labels Jan 8, 2026
@botelastic

botelastic Bot commented Feb 7, 2026

Copy link
Copy Markdown

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

@botelastic botelastic Bot added the Stalled label Feb 7, 2026
@botelastic

botelastic Bot commented Mar 9, 2026

Copy link
Copy Markdown

Hi! This PR has been stale for a while and we're going to close it as part of our cleanup procedure. We appreciate your contribution and would like to apologize if we have not been able to review it, due to the current heavy load of the team. Feel free to re-open this PR if you think it should stay open and is worth rebasing. Thank you for your contribution!

@botelastic botelastic Bot closed this Mar 9, 2026
@cpascale43 cpascale43 reopened this Mar 9, 2026
@botelastic botelastic Bot removed the Stalled label Mar 9, 2026
@botelastic

botelastic Bot commented Apr 8, 2026

Copy link
Copy Markdown

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

@botelastic botelastic Bot added the Stalled label Apr 8, 2026
@cpascale43 cpascale43 removed the Stalled label Apr 8, 2026
@andrewkroh andrewkroh removed dashboard Relates to a Kibana dashboard bug, enhancement, or modification. New Integration Issue or pull request for creating a new integration package. Integration:axonius Axonius labels Apr 23, 2026
@andrewkroh andrewkroh added the New Integration Issue or pull request for creating a new integration package. label May 11, 2026
@github-actions

Copy link
Copy Markdown
Contributor

TL;DR

The Buildkite failure is caused by an auto-merge conflict during the Upload Pipeline step, not a test/runtime failure. Rebase (or merge) the PR branch onto the latest target branch and resolve the listed conflicts, then re-run CI.

Remediation

  • Rebase muskan-agarwal26:datastream-application onto feature/axonius-0.1.0 (or merge target into your branch) and resolve conflicts in:
    • packages/axonius/_dev/build/docs/README.md
    • packages/axonius/_dev/deploy/docker/files/config.yml
    • packages/axonius/changelog.yml
    • packages/axonius/docs/README.md
    • packages/axonius/kibana/dashboard/axonius-08681c8c-7071-48b8-9fb2-e2c31c38539f.json
    • packages/axonius/manifest.yml
  • Push the resolved branch and re-run Buildkite to confirm the post-checkout merge succeeds.
Investigation details

Root Cause

The failing job runs .buildkite/hooks/post-checkout, which intentionally merges the PR commit into the target branch for PR builds. That merge fails and exits non-zero in git merge --no-edit "${BUILDKITE_COMMIT}" (.buildkite/hooks/post-checkout#L47-L52), so pipeline upload stops.

Evidence

Auto-merging packages/axonius/_dev/build/docs/README.md
CONFLICT (content): Merge conflict in packages/axonius/_dev/build/docs/README.md
...
Auto-merging packages/axonius/manifest.yml
CONFLICT (content): Merge conflict in packages/axonius/manifest.yml
Automatic merge failed; fix conflicts and then commit the result.
Merge failed: 1
🚨 Error: running "repository post-checkout" shell hook: The repository post-checkout hook exited with status 1

A later pre-exit hook error (.buildkite/scripts/common.sh: line 50: TMP_FOLDER_TEMPLATE_BASE: unbound variable) appears during cleanup after the primary merge failure and is not the initiating cause of this red build.

Verification

  • Not run (workflow is read-only; analysis based on provided Buildkite logs and repository hook code).

Follow-up

  • If CI still fails after conflict resolution, share the new failed job log and I can trace the next blocker.

Note

🔒 Integrity filter blocked 2 items

The following items were blocked because they don't meet the GitHub integrity level.

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

@piyush-elastic
piyush-elastic marked this pull request as ready for review May 21, 2026 17:18
@piyush-elastic
piyush-elastic requested a review from a team as a code owner May 21, 2026 17:18
@andrewkroh andrewkroh removed the New Integration Issue or pull request for creating a new integration package. label May 21, 2026
Comment thread packages/axonius/manifest.yml Resolved
"created": "2024-12-25T22:17:25.000Z",
"kind": "event",
"original": "{\"asset_type\":\"urls\",\"internal_axon_id\":\"9c65d891077147892d0e632eb0cf6ebf\",\"adapters\":[\"okta_adapter\"],\"adapter_list_length\":1,\"event\":{\"accurate_for_datetime\":\"Thu, 13 Nov 2025 00:02:55 GMT\",\"adapter_categories\":[\"IAM\",\"SaaS Management\"],\"client_used\":\"67fd09cd782eb39db73d1af1\",\"data\":{\"domain\":\"example-domain\",\"accurate_for_datetime\":\"Thu, 13 Nov 2025 00:02:55 GMT\",\"activity_status_active\":0,\"activity_status_active_hyperlink\":[{\"compOp\":\"equals\",\"field\":\"specific_data.data.activity_status\",\"leftBracket\":0,\"logicOp\":\"and\",\"not\":false,\"rightBracket\":0,\"value\":\"Active\"}],\"activity_status_inactive\":288,\"activity_status_inactive_hyperlink\":[{\"compOp\":\"equals\",\"field\":\"specific_data.data.activity_status\",\"leftBracket\":0,\"logicOp\":\"and\",\"not\":false,\"rightBracket\":0,\"value\":\"Inactive\"}],\"app_id\":\"990f042e1d803894A3ae\",\"application_and_account_name\":\"okta/okta-demo\",\"auth_type\":[],\"created\":\"Wed, 25 Dec 2024 22:17:25 GMT\",\"extension_type\":\"SSO\",\"fetch_time\":\"Thu, 13 Nov 2025 00:02:18 GMT\",\"first_fetch_time\":\"Mon, 14 Apr 2025 13:27:09 GMT\",\"first_seen\":\"Mon, 22 Apr 2024 21:22:25 GMT\",\"from_last_fetch\":true,\"grant_types\":[],\"id\":\"cd2f40fe8670900112ab\",\"id_raw\":\"okta_adapter_0!990f042e1d803894A3ae\",\"integration_type\":\"Admin Managed\",\"is_admin\":16,\"is_admin_hyperlink\":[{\"compOp\":\"true\",\"field\":\"specific_data.data.permissions.is_admin\",\"leftBracket\":0,\"logicOp\":\"and\",\"not\":false,\"rightBracket\":0}],\"is_fetched_from_adapter\":true,\"is_identity\":0,\"is_identity_hyperlink\":[{\"compOp\":\"true\",\"field\":\"specific_data.data.permissions.is_identity\",\"leftBracket\":0,\"logicOp\":\"and\",\"not\":false,\"rightBracket\":0}],\"is_operational\":true,\"last_fetch_connection_id\":\"67fd09cd782eb39db73d1af1\",\"last_fetch_connection_label\":\"okta-demo\",\"last_seen\":\"Mon, 07 Apr 2025 13:05:30 GMT\",\"last_used\":\"Mon, 07 Apr 2025 13:05:30 GMT\",\"name\":\"Cloudflare\",\"not_fetched_count\":0,\"permissions\":[{\"is_admin\":true,\"name\":\"allow_api_access_to_account\",\"users_amount\":9}],\"redirect_uris\":[],\"related_vendor_name\":\"Cloudflare\",\"scope_tag_calendar\":0,\"scope_tag_calendar_hyperlink\":[{\"compOp\":\"equals\",\"field\":\"specific_data.data.app_id\",\"leftBracket\":0,\"logicOp\":\"and\",\"not\":false,\"rightBracket\":0,\"value\":\"990f042e1d803894A3ae\"}],\"scope_tag_drive\":0,\"scope_tag_drive_hyperlink\":[{\"compOp\":\"equals\",\"field\":\"specific_data.data.permissions.scope_tag\",\"leftBracket\":0,\"logicOp\":\"and\",\"not\":false,\"rightBracket\":0,\"value\":\"Drive\"}],\"scope_tag_mail\":0,\"scope_tag_mail_hyperlink\":[{\"compOp\":\"equals\",\"field\":\"specific_data.data.permissions.scope_tag\",\"leftBracket\":0,\"logicOp\":\"and\",\"not\":false,\"rightBracket\":0,\"value\":\"Mail\"}],\"source_application\":\"Okta\",\"type\":\"Extensions\",\"urls\":[],\"user_count\":288,\"user_count_link\":[{\"bracketWeight\":0,\"compOp\":\"IN\",\"field\":\"specific_data.data.mail\",\"leftBracket\":0,\"logicOp\":\"\",\"not\":false,\"rightBracket\":0,\"value\":\"ronald.mays@demo.local,justin.baugh@demo.local,brian.williamskaren.cox@demo.local,bill.mcnay@demo.local,billy.woodruff@demo.local,hazel.contreras@demo.local,eduardo.mandeville@demo.local,karen.neal@demo.local,troy.hooper@demo.local,andreas.rice@demo.local,janis.henry@demo.local,mary.tavares@demo.local,patrick.rowe@demo.local,kenneth.gardner@demo.local,paul.hendricks@demo.local,tanya.wolf@demo.local,kathleen.arroliga@demo.local,robert.blunt@demo.local,charles.duncan@demo.local,margarita.zapata@demo.local,charles.paredes@demo.local,floyd.conrad@demo.local,tammy.hawkins@demo.local,sam.chavez@demo.local,francis.rivera@demo.local,brandon.lilly@demo.local,matthew.wiley@demo.local,lacey.smith@demo.local,tracy.white@demo.local\"}],\"username_formats\":[],\"users_amount\":288},\"initial_plugin_unique_name\":\"okta_adapter_0\",\"plugin_name\":\"okta_adapter\",\"plugin_type\":\"Adapter\",\"plugin_unique_name\":\"okta_adapter_0\",\"quick_id\":\"okta_adapter_0!cd2f40fe8670900112ab\",\"type\":\"entitydata\"}}"
},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing event.category, event.type. Can you please add them?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its still missing for some of the pipelines.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is because no other category fits for the other missing pipelines.

@kcreddy kcreddy May 25, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use configuration and info for the event.category and event.type respectively for the rest.
https://www.elastic.co/docs/reference/ecs/ecs-allowed-values-event-category#ecs-event-category-configuration

- urls
- application_services
- application_resources
- secrets

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have test fixtures for all of them added? What about the pipelines, I see for some of them but not for all.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have logs for these asset types (not even any schema), hence not added in test logs.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there any plans to get some documents? This is a concern.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We had connected earlier with the Axonius team for logs, they had provided for few assets only.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thin we should only add the ones we have the samples for, because only those are mapped correctly. The rest can be added as enhancements later if we get samples.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The common fields in default.yml will get populated in other asset types as well, hence I think we shouldn't remove support of those types.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are serious problems with shipping asset types you have no samples for:

  1. No field mappings. The fields.yml has 1018 lines of field definitions, but these were clearly derived from the asset types that have samples. For the 11 types without samples (admin_managed_extensions, user_initiated_extensions, application_addons, admin_managed_extension_instances, user_initiated_extension_instances, application_addon_instances, application_keys, urls, application_services, application_resources, secrets), we have no idea what fields those responses contain. Any type-specific fields will either fail mapping (wrong types) or fall through to dynamic mapping, creating untested index mappings.

  2. No type conversion. The sub-pipelines exist specifically to do type conversions (convert to long/boolean/double, date parsing) and ECS mappings for type-specific fields. Without a sub-pipeline, those fields stay as raw strings from the JSON parse — a long field getting a string value, a date field staying as a human-readable date string. This creates mapping conflicts or silent data quality issues.

  3. No test coverage. You can't validate what you can't see. Shipping untested asset types means any change to the default pipeline could silently break ingestion for those types without anyone noticing. The pipeline test fixture only covers 7-8 asset types. If something breaks for secrets or application_keys, there's no test to catch it.

  4. False sense of coverage. Users see 18 asset types listed and assume the integration handles them all correctly. In reality, only ~7 are properly mapped, typed, and tested. That's worse than listing 7 types honestly and adding the rest when they're ready.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After checking with the client again on this, got few more logs so adding them in test files, now only one asset type needed to be removed: application_services.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Let me know when it's ready to be reviewed again.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added the missing types, please review @kcreddy

@muskan-agarwal26
muskan-agarwal26 requested a review from kcreddy May 24, 2026 14:38
@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@kcreddy kcreddy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pending on #16658 (comment)

name: '{{ IngestPipeline "pipeline_application_settings" }}'
tag: pipeline_application_settings
if: >-
ctx.axonius?.application?.asset_type != null && ctx.axonius.application.asset_type == 'application_settings'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ctx.axonius?.application?.asset_type != null && ctx.axonius.application.asset_type == 'application_settings'
ctx.axonius?.application?.asset_type == 'application_settings'

That should work

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for the rest

"created": "2024-12-25T22:17:25.000Z",
"kind": "event",
"original": "{\"asset_type\":\"urls\",\"internal_axon_id\":\"9c65d891077147892d0e632eb0cf6ebf\",\"adapters\":[\"okta_adapter\"],\"adapter_list_length\":1,\"event\":{\"accurate_for_datetime\":\"Thu, 13 Nov 2025 00:02:55 GMT\",\"adapter_categories\":[\"IAM\",\"SaaS Management\"],\"client_used\":\"67fd09cd782eb39db73d1af1\",\"data\":{\"domain\":\"example-domain\",\"accurate_for_datetime\":\"Thu, 13 Nov 2025 00:02:55 GMT\",\"activity_status_active\":0,\"activity_status_active_hyperlink\":[{\"compOp\":\"equals\",\"field\":\"specific_data.data.activity_status\",\"leftBracket\":0,\"logicOp\":\"and\",\"not\":false,\"rightBracket\":0,\"value\":\"Active\"}],\"activity_status_inactive\":288,\"activity_status_inactive_hyperlink\":[{\"compOp\":\"equals\",\"field\":\"specific_data.data.activity_status\",\"leftBracket\":0,\"logicOp\":\"and\",\"not\":false,\"rightBracket\":0,\"value\":\"Inactive\"}],\"app_id\":\"990f042e1d803894A3ae\",\"application_and_account_name\":\"okta/okta-demo\",\"auth_type\":[],\"created\":\"Wed, 25 Dec 2024 22:17:25 GMT\",\"extension_type\":\"SSO\",\"fetch_time\":\"Thu, 13 Nov 2025 00:02:18 GMT\",\"first_fetch_time\":\"Mon, 14 Apr 2025 13:27:09 GMT\",\"first_seen\":\"Mon, 22 Apr 2024 21:22:25 GMT\",\"from_last_fetch\":true,\"grant_types\":[],\"id\":\"cd2f40fe8670900112ab\",\"id_raw\":\"okta_adapter_0!990f042e1d803894A3ae\",\"integration_type\":\"Admin Managed\",\"is_admin\":16,\"is_admin_hyperlink\":[{\"compOp\":\"true\",\"field\":\"specific_data.data.permissions.is_admin\",\"leftBracket\":0,\"logicOp\":\"and\",\"not\":false,\"rightBracket\":0}],\"is_fetched_from_adapter\":true,\"is_identity\":0,\"is_identity_hyperlink\":[{\"compOp\":\"true\",\"field\":\"specific_data.data.permissions.is_identity\",\"leftBracket\":0,\"logicOp\":\"and\",\"not\":false,\"rightBracket\":0}],\"is_operational\":true,\"last_fetch_connection_id\":\"67fd09cd782eb39db73d1af1\",\"last_fetch_connection_label\":\"okta-demo\",\"last_seen\":\"Mon, 07 Apr 2025 13:05:30 GMT\",\"last_used\":\"Mon, 07 Apr 2025 13:05:30 GMT\",\"name\":\"Cloudflare\",\"not_fetched_count\":0,\"permissions\":[{\"is_admin\":true,\"name\":\"allow_api_access_to_account\",\"users_amount\":9}],\"redirect_uris\":[],\"related_vendor_name\":\"Cloudflare\",\"scope_tag_calendar\":0,\"scope_tag_calendar_hyperlink\":[{\"compOp\":\"equals\",\"field\":\"specific_data.data.app_id\",\"leftBracket\":0,\"logicOp\":\"and\",\"not\":false,\"rightBracket\":0,\"value\":\"990f042e1d803894A3ae\"}],\"scope_tag_drive\":0,\"scope_tag_drive_hyperlink\":[{\"compOp\":\"equals\",\"field\":\"specific_data.data.permissions.scope_tag\",\"leftBracket\":0,\"logicOp\":\"and\",\"not\":false,\"rightBracket\":0,\"value\":\"Drive\"}],\"scope_tag_mail\":0,\"scope_tag_mail_hyperlink\":[{\"compOp\":\"equals\",\"field\":\"specific_data.data.permissions.scope_tag\",\"leftBracket\":0,\"logicOp\":\"and\",\"not\":false,\"rightBracket\":0,\"value\":\"Mail\"}],\"source_application\":\"Okta\",\"type\":\"Extensions\",\"urls\":[],\"user_count\":288,\"user_count_link\":[{\"bracketWeight\":0,\"compOp\":\"IN\",\"field\":\"specific_data.data.mail\",\"leftBracket\":0,\"logicOp\":\"\",\"not\":false,\"rightBracket\":0,\"value\":\"ronald.mays@demo.local,justin.baugh@demo.local,brian.williamskaren.cox@demo.local,bill.mcnay@demo.local,billy.woodruff@demo.local,hazel.contreras@demo.local,eduardo.mandeville@demo.local,karen.neal@demo.local,troy.hooper@demo.local,andreas.rice@demo.local,janis.henry@demo.local,mary.tavares@demo.local,patrick.rowe@demo.local,kenneth.gardner@demo.local,paul.hendricks@demo.local,tanya.wolf@demo.local,kathleen.arroliga@demo.local,robert.blunt@demo.local,charles.duncan@demo.local,margarita.zapata@demo.local,charles.paredes@demo.local,floyd.conrad@demo.local,tammy.hawkins@demo.local,sam.chavez@demo.local,francis.rivera@demo.local,brandon.lilly@demo.local,matthew.wiley@demo.local,lacey.smith@demo.local,tracy.white@demo.local\"}],\"username_formats\":[],\"users_amount\":288},\"initial_plugin_unique_name\":\"okta_adapter_0\",\"plugin_name\":\"okta_adapter\",\"plugin_type\":\"Adapter\",\"plugin_unique_name\":\"okta_adapter_0\",\"quick_id\":\"okta_adapter_0!cd2f40fe8670900112ab\",\"type\":\"entitydata\"}}"
},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its still missing for some of the pipelines.

@muskan-agarwal26

Copy link
Copy Markdown
Contributor Author

Its still missing for some of the pipelines.

@kcreddy , I didn't understand what is missing?

@muskan-agarwal26
muskan-agarwal26 requested review from efd6 and kcreddy May 25, 2026 12:25
- audit_activities (endpoint: `/api/v2/assets/audit_activities`)
- business_applications (endpoint: `/api/v2/assets/business_applications`)
- urls (endpoint: `/api/v2/assets/urls`)
- application_services (endpoint: `/api/v2/assets/application_services`)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove application_services?

Comment on lines +82 to +89
- name: preserve_duplicate_custom_fields
required: false
title: Preserve duplicate custom fields
description: Preserve axonius.application.* fields that were copied to Elastic Common Schema (ECS) fields.
type: bool
multi: false
show_user: false
default: false

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using preserve_duplicate_custom_fields is a legacy pattern now. We should plan to remove this before GA.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this change needs to be done in all the rest datastreams of Axonius and this one is a last datatsream PR pending for merge, one this gets merged, will have to create another PR, so will do this change there for all.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes before GA is the intention.

@kcreddy kcreddy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@ShourieG

Copy link
Copy Markdown
Contributor

/test

@ShourieG
ShourieG enabled auto-merge (squash) May 27, 2026 07:40
@elasticmachine

Copy link
Copy Markdown

💚 Build Succeeded

History

@ShourieG
ShourieG merged commit c0dae6c into elastic:feature/axonius-0.1.0 May 27, 2026
5 checks passed
@muskan-agarwal26 muskan-agarwal26 changed the title [axonius][application] Add Axonius Application datastream #16603 [axonius][application] Add Axonius Application datastream Jun 1, 2026
@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

Package axonius - 0.1.0 containing this change is available at https://epr.elastic.co/package/axonius/0.1.0/

@andrewkroh andrewkroh added dashboard Relates to a Kibana dashboard bug, enhancement, or modification. Integration:axonius Axonius labels Jun 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Crest Contributions from Crest developement team. dashboard Relates to a Kibana dashboard bug, enhancement, or modification. documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. Integration:axonius Axonius

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants