Skip to content

Segments created in the same batch have the same created_date entry & rename metric - #15977

Merged
abhishekrb19 merged 3 commits into
apache:masterfrom
abhishekrb19:created_time_insert
Feb 27, 2024
Merged

abhishekrb19 merged 3 commits into
apache:masterfrom
abhishekrb19:created_time_insert

Conversation

@abhishekrb19

@abhishekrb19 abhishekrb19 commented Feb 27, 2024 •

Copy link
Copy Markdown
Contributor

Summary of changes:

  1. ebc2a0f: Materialize created_date outside the for loop, so all the segments created in a batch have the same created_date entry. In the absence of a group_id column, this metadata would allow us to easily reason about and troubleshoot ingestion-related issues.

  2. aa1bee5: Address review comment from Refactor and add tests and metric to KillUnusedSegments duty (auto-kill) #15941 (comment) in. The metric name and description along with the code references are updated.

This PR has:

  • been self-reviewed.
  • been tested in a test Druid cluster.

In the absence of a group_id column, this metadata would allow us to easily
reason about and troubleshoot ingestion-related issues.
@abhishekrb19
abhishekrb19 requested a review from kfaraz February 27, 2024 07:07
@abhishekrb19 abhishekrb19 changed the title Segments created in the same batch have the same created_date entry Segments created in the same batch have the same created_date entry & rename metric Feb 27, 2024

// Each unused segment interval returned above has a 1:1 correspondence with an unused segment. So we can assume
// these are candidate segments eligible for deletion by the kill task. After the umbrella interval is computed
// these are eligible segments for deletion by the kill task. After the umbrella interval is computed

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// these are eligible segments for deletion by the kill task. After the umbrella interval is computed
// these are segments eligible for deletion by the kill task. After the umbrella interval is computed

@abhishekrb19
abhishekrb19 merged commit beccc40 into apache:master Feb 27, 2024
@abhishekrb19
abhishekrb19 deleted the created_time_insert branch February 27, 2024 11:58
@adarshsanjeev adarshsanjeev added this to the 30.0.0 milestone May 6, 2024
riovic918data pushed a commit to riovic918data/druid that referenced this pull request Jun 12, 2026
… & rename metric (apache#15977)

* All segments stored in the same batch have the same created_date entry.

In the absence of a group_id column, this metadata would allow us to easily
reason about and troubleshoot ingestion-related issues.

* Rename metric name and code references to eligibleUnusedSegments.

Address review comment from apache#15941 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants