Skip to content

chore: refactor endpoints to use same inferencing path, add automatic retrial mechanism in case of errors#9029

Merged
mudler merged 2 commits intomasterfrom
chore/refactor-retry
Mar 16, 2026
Merged

chore: refactor endpoints to use same inferencing path, add automatic retrial mechanism in case of errors#9029
mudler merged 2 commits intomasterfrom
chore/refactor-retry

Conversation

@mudler
Copy link
Copy Markdown
Owner

@mudler mudler commented Mar 16, 2026

Description

This PR does refactoring around duplicated code in anthropic/openresponses endpoint, and make consistent usage of ComputeChoice across the code.

Here we also introduce a retrial mechanism directly in ComputeChoice instead of scattering it across callers in the code, to cover also streaming mode.

Reasoning handling has been extracted as well to its own structure to make the code more maintenable

Notes for Reviewers

Signed commits

  • Yes, I signed my commits.

@netlify
Copy link
Copy Markdown

netlify bot commented Mar 16, 2026

Deploy Preview for localai ready!

Name Link
🔨 Latest commit 92494c5
🔍 Latest deploy log https://app.netlify.com/projects/localai/deploys/69b83e9ebf630b000858a793
😎 Deploy Preview https://deploy-preview-9029--localai.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@mudler mudler force-pushed the chore/refactor-retry branch from b70d3b7 to a6d80ed Compare March 16, 2026 17:10
… retrial mechanism in case of errors

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler mudler force-pushed the chore/refactor-retry branch from a6d80ed to 9553243 Compare March 16, 2026 17:12
@mudler mudler merged commit ee96e5e into master Mar 16, 2026
36 of 37 checks passed
@mudler mudler deleted the chore/refactor-retry branch March 16, 2026 20:31
localai-bot pushed a commit to localai-bot/LocalAI that referenced this pull request Mar 18, 2026
… retrial mechanism in case of errors (mudler#9029)

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: localai-bot <localai-bot@users.noreply.github.com>
localai-bot pushed a commit to localai-bot/LocalAI that referenced this pull request Mar 20, 2026
… retrial mechanism in case of errors (mudler#9029)

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: localai-bot <bot@localai.com>
localai-bot pushed a commit to localai-bot/LocalAI that referenced this pull request Mar 20, 2026
… retrial mechanism in case of errors (mudler#9029)

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Signed-off-by: localai-bot <bot@localai.com>

Signed-off-by: localai-bot <localai-bot@localai.io>
localai-bot pushed a commit to localai-bot/LocalAI that referenced this pull request Mar 20, 2026
… retrial mechanism in case of errors (mudler#9029)

Signed-off-by: localai-bot <localai-bot@localai.io>
localai-bot pushed a commit to localai-bot/LocalAI that referenced this pull request Mar 20, 2026
… retrial mechanism in case of errors (mudler#9029)

Signed-off-by: localai-bot <localai-bot@localai.io>
localai-bot pushed a commit to localai-bot/LocalAI that referenced this pull request Mar 25, 2026
… retrial mechanism in case of errors (mudler#9029)

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant