Skip to content

FYI: related project fscacher/question #241

Open

Description

@yarikoptic

Just ran into this library which is of interest, yet to try/explore more. But it resonated with me as potentially (?) to replace our home-brewed joblib based https://github.com/con/fscacher where the goal is to provide easy means to cache results of compute on files (think e.g. extraction of metadata or even checksums). Does exca provide such a facility (caching based on the state of the file)?

Activity

  1. kingjr commented on Jun 20, 2026

    @kingjr
    Contributor

    Thanks for the pointer. I didn't quite get what functionality you're looking for, would you have an example use case?

  2. yarikoptic commented on Jun 20, 2026

    @yarikoptic
    ContributorAuthor

    taking on the example in your README could be smth like

      class MyTask(pydantic.BaseModel):
          path: Path = xk.Path
          infra: xk.TaskInfra = xk.TaskInfra()
    
          @infra.apply
          def process_file(self) -> str:
              return hashlib.sha256(self.path.read_bytes()).hexdigest()

    so the point here is that if path content changes (according to change in some fingerprint e.g. size, mtime, inode) - function is recomputed but if not -- result is cached/reused

  3. kingjr commented on Jun 21, 2026

    @kingjr
    Contributor

    I see. I think this should can covered with pydantic api:

    from pathlib import Path
    import hashlib
    import pydantic
    
    class MyTask(pydantic.BaseModel):
        path_uid: str
        infra: xk.TaskInfra = xk.TaskInfra()
    
        @pydantic.field_validator("path_uid", mode="before")
        @classmethod
        def hash_path(cls, v):
            return hashlib.sha256(Path(v).read_bytes()).hexdigest() # potentially concatenate it to `v` 
    
        @infra.apply
        def run(self) -> tp.Any:
            # your main compute here
            return 'foo'

    The downside is that it'd check at the initialization, whereas exca usually check for cache at execution.

  4. yarikoptic commented on Jun 21, 2026

    @yarikoptic
    ContributorAuthor

    what in above would take care about invalidating the cache when file fingerprint (e.g. mtime or size) changes?

  5. kingjr commented on Jun 22, 2026

    @kingjr
    Contributor

    At init, my snippet gets the hexdigest of the file and keep it as param in the class.

    The exca infra uses the non-default param to compute the unique id, against which the infra is checked. If that unique id isn't already cached, then the method will be executed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions