Skip to content

Retries not working in azure.storage.blob response streaming when a ServiceResponseError occurs #44668

Description

@meliache
  • Package Name: azure-storage-blob
  • Package Version: 12.28.0
  • Operating System: macOS 15.7.3
  • Python Version: 3.13.11

Describe the bug
When downloading a blob with

    buffer = io.BytesIO()
    with ContainerClient.from_container_url(container_url) as cc_in:
        cc_in.download_blob(blob_key).readinto(buffer)

in production we once got a requests.exceptions.ConnectionError during the response processing. It seems that currently, on IncompleteReadError, HttpResponseError and DecodeError are retried in the _download_chunk function here

                try:
                    chunk_data = process_content(response, offset[0], offset[1], self.encryption_options)
                    retry_active = False
                except (IncompleteReadError, HttpResponseError, DecodeError) as error:
                    retry_total -= 1
                    if retry_total <= 0:
                        raise HttpResponseError(error, error=error) from error
                    time.sleep(1)

I found that in the latest azure-storage-blob as of PR #43201, connection errors are now wrapped as ServiceResponseError in the StreamDownloadGenerator.__next__ method here. It's just when we first saw this in production, we had an azure-storage-blob version which didn't wrap connection errors yet.

Now that during process_content, a connection error wrapped as a ServiceResponseError can occur, I think that ServiceResponseError should be added to the list of exceptions that are retried in the except clause above. I saw that previously, retries had been added for requests.exceptions.ConnectionError in PR #18164, but this was removed later to get rid of the requests requirement #25017 by @Stevenjin8, so if ServiceResponseError is not wrapped this seems like a degradatio to me.

Expected behavior
If process_content raises a ServiceResponseError due to a connection error during downloading of chunks, the download should be retried, consistent with the indended behaviour of #18164

Metadata

Metadata

Labels

Service AttentionWorkflow: This issue is responsible by Azure service team.StorageStorage Service (Queues, Blobs, Files)customer-reportedIssues that are reported by GitHub users external to the Azure organization.needs-team-attentionWorkflow: This issue needs attention from Azure service team or SDK teamquestionThe issue doesn't require a change to the product in order to be resolved. Most issues start as that

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions