Repository navigation
fix: check pinecone upsert response - #3561
IsmaelMartinez merged 2 commits into
Conversation
PR Summary by QodoFail similar-issue indexing on Pinecone batch errors
AI Description
Diagram
High-Level Assessment
Files changed (2)
|
Code Review by Qodo
1.
|
f03f2c9 to
4c240b9
Compare
|
Code review by qodo was updated up to the latest commit 4c240b9 |
4c240b9 to
5025ee2
Compare
|
Code review by qodo was updated up to the latest commit 5025ee2 |
IsmaelMartinez
left a comment
There was a problem hiding this comment.
Thanks for the same-morning turnaround on #3544 and for folding both Qodo findings in. This holds on the real pinecone 10.0.0 client: a failed batch now raises with the count logged, and the LSN poll replaces the fixed sleep.
One ask inline: the SDK keeps each failed batch's error message, so log it beside the count, or the operator sees "100 vectors" and no cause.
#3560 landed first this afternoon, so this needs a rebase: the repository namespace now goes on your upsert and on the readiness fetch.
| get_logger().error( | ||
| "Pinecone upsert failed", | ||
| artifact={"failed_item_count": failed_item_count}, | ||
| ) |
There was a problem hiding this comment.
The SDK keeps each failed batch's error_message; without it this log says how many vectors failed and nothing about why.
| get_logger().error( | |
| "Pinecone upsert failed", | |
| artifact={"failed_item_count": failed_item_count}, | |
| ) | |
| errors = getattr(response, "errors", response_dict.get("errors")) or [] | |
| get_logger().error( | |
| "Pinecone upsert failed", | |
| artifact={"failed_item_count": failed_item_count, | |
| "errors": [_get_value(error, "error_message") for error in errors]}, | |
| ) |
| with pytest.raises(RuntimeError, match="100"): | ||
| tool._update_index_with_issues([_make_issue(7)], "example-repo", upsert=True) | ||
|
|
||
| assert logged == [{"failed_item_count": 100}] |
There was a problem hiding this comment.
Matching expectation for the artifact above.
| assert logged == [{"failed_item_count": 100}] | |
| assert logged == [{"failed_item_count": 100, "errors": []}] |
5025ee2 to
f872889
Compare
|
Code review by qodo was updated up to the latest commit f872889 |
IsmaelMartinez
left a comment
There was a problem hiding this comment.
Thanks for extending this to a full readiness check and folding in both Qodo findings, this needs one change before merge. The core fix holds: reverting your source while keeping your tests turns 6 of 20 green tests red, and the LSN wait terminates cleanly under both fast and stuck reconciliation. One inline ask: the wait loop calls fetch before checking its deadline, so a slow fetch can push the wait past the configured 30 seconds; suggestion attached, applied to your head and tests still pass. Full suite is green (9184 passed, no regressions), cleanly rebased on #3560's namespace change.
| while True: | ||
| fetch_response = pinecone_index.fetch(ids=[vector_id], namespace=namespace) |
There was a problem hiding this comment.
The wait loop must check its deadline before calling fetch, not after; a slow fetch can otherwise push the total wait past the configured 30 seconds.
| while True: | |
| fetch_response = pinecone_index.fetch(ids=[vector_id], namespace=namespace) | |
| while True: | |
| if time.monotonic() >= deadline: | |
| raise TimeoutError(f"Pinecone upsert was not query-ready after {PINECONE_UPSERT_READY_TIMEOUT_SECONDS}s") | |
| fetch_response = pinecone_index.fetch(ids=[vector_id], namespace=namespace) |
There was a problem hiding this comment.
Applied on the head commit a1c587a: the loop now raises when time.monotonic() >= deadline before calling fetch, so a slow fetch can no longer stretch the wait past the configured timeout. Added a regression test that pins the loop stops polling once the deadline is passed.
|
Code review by qodo was updated up to the latest commit a1c587a |
IsmaelMartinez
left a comment
There was a problem hiding this comment.
Thanks for the quick fix, this is ready to merge. The wait loop now checks its deadline before calling fetch, so a slow request can no longer push past the 30 second budget: reverting your source while keeping the tests turns 7 of 14 red, and a timed harness with a slow fake fetch cuts worst case overshoot from about 97% to under 4%. Full suite is green at 9193 passed, exactly baseline plus your tests, and CI is all green.
Summary
Closes #3544
Tests