Skip to content

Make directory operations methods on Directory - #11945

Merged
JosiahWI merged 14 commits into
apache:masterfrom
JosiahWI:refactor/extract-directory
Jun 16, 2025
Merged

Make directory operations methods on Directory#11945
JosiahWI merged 14 commits into
apache:masterfrom
JosiahWI:refactor/extract-directory

Conversation

@JosiahWI

@JosiahWI JosiahWI commented Jan 8, 2025

Copy link
Copy Markdown
Contributor

These operations are working with directory data. Right now, they operate on data of type Stripe or StripeSM, but that was because those data structures contained the directory data they needed until recently. Most of the other data there is irrelevant.

This PR moves those operations (see commits for the list) into the Directory structure to reduce the dependence on the irrelevant parts of Stripe and StripeSM.

The StripeSMs are needed in some cases for debugging, metrics, and an intriguing invocation of Stripe::dir_valid from dir_probe; the Directory structure will need to be adjusted to have access to a few things for those purposes, but the details of that are still a work in progress.

@JosiahWI JosiahWI self-assigned this Jan 8, 2025
@JosiahWI JosiahWI added this to the 10.1.0 milestone Jan 8, 2025
@JosiahWI
JosiahWI force-pushed the refactor/extract-directory branch 2 times, most recently from dee18ce to 3695b7a Compare January 8, 2025 20:39
@JosiahWI
JosiahWI force-pushed the refactor/extract-directory branch from 3695b7a to 7959b78 Compare March 4, 2025 20:34
@cmcfarlen cmcfarlen modified the milestones: 10.1.0, 10.2.0 Mar 10, 2025
@bryancall

Copy link
Copy Markdown
Contributor

[approve ci autest 2]

@JosiahWI
JosiahWI force-pushed the refactor/extract-directory branch from 7959b78 to 388d7ab Compare May 26, 2025 22:40
@JosiahWI

Copy link
Copy Markdown
Contributor Author

Rebased on master.

@JosiahWI
JosiahWI marked this pull request as ready for review May 26, 2025 22:41
@JosiahWI

Copy link
Copy Markdown
Contributor Author

I have not removed the dependency of Directory on StripeSM or Stripe, but after looking at this multiple times, I'm not sure it's a good idea to try to force that. It would be nice to have a cleaner dependency structure, but we should aim to simplify things, not add complexity. This PR cleans up some call sites and makes it clearer what data is required to carry out each operation. Therefore, I think it is a good change as-is.

@JosiahWI
JosiahWI requested a review from masaori335 June 2, 2025 22:49
Comment thread src/iocore/cache/Cache.cc
{
CACHE_TRY_LOCK(lock, stripe->mutex, mutex->thread_holding);
if (!lock.is_locked() || (od = stripe->open_read(key)) || dir_probe(key, stripe, &result, &last_collision)) {
if (!lock.is_locked() || (od = stripe->open_read(key)) || stripe->directory.probe(key, stripe, &result, &last_collision)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note for later refactoring: this can be like this?

stripe->dir_probe(key, &result, &last_collision)

// Note : abuse of the token bit in dir entries
int
dir_bucket_loop_fix(Dir *start_dir, int s, Stripe *stripe)
dir_bucket_loop_fix(Dir *start_dir, int s, Directory *directory)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make this function a member of Directory too?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely, I missed this opportunity. Very easy improvement for further refactoring. Thanks for catching it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think dir_init_segment can be a method of Directory as well.


inline void
unlink_from_freelist(Dir *e, int s, Stripe *stripe)
unlink_from_freelist(Dir *e, int s, Directory *directory)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to above. Would you share me what's the policy of which function should be moved to Directory or not here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked for the functions that were operations on the directory, and I might have missed this one.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this should be moved. I see what happened. I only moved the public interface, and not any helper functions. Further refactoring should make all these helper functions private methods on Directory.

@masaori335 masaori335 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This just moves Dir operators into Directory for cleanup. Probably, we need more refactoring in this area, but this seems good to me.

@JosiahWI
JosiahWI merged commit 9ff9476 into apache:master Jun 16, 2025
@JosiahWI
JosiahWI deleted the refactor/extract-directory branch June 16, 2025 15:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants