What would you like to happen?
Currently, many exception handlers throughout the Python SDK re-raise exceptions without using raise ... from, which loses the original traceback context. This makes debugging significantly harder because developers can't see the root cause when exceptions are wrapped.
Example Problem:
# Current code (loses context)
try:
connection = self._engine.connect()
result = connection.execute(text(query))
except Exception as e:
raise RuntimeError(f"Database operation failed: {e}")
When this fails, you only see:
RuntimeError: Database operation failed: connection timeout
But you don't see the original ConnectionTimeoutError with its full traceback.
Desired Solution:
# With exception chaining (preserves context)
try:
connection = self._engine.connect()
result = connection.execute(text(query))
except Exception as e:
raise RuntimeError(f"Database operation failed: {e}") from e
Now you see:
RuntimeError: Database operation failed: connection timeout
The above exception was the direct cause of the following exception:
ConnectionTimeoutError: Failed to connect within 30 seconds
[full original traceback here]
I would like to systematically add exception chaining throughout the codebase. This involves:
- Identifying all exception re-raises - Find all places where exceptions are caught and re-raised
- Adding
from e clause - Update raise NewException(...) to raise NewException(...) from e
- Enabling pylint warning - Enable the
raise-missing-from warning in .pylintrc (currently disabled with TODO)
- Adding tests - Ensure exception chains are preserved in error scenarios
- Incremental approach - Start with high-traffic modules (transforms, runners, IO)
This enhancement will significantly improve debuggability by preserving the full exception context, making it easier to identify root causes of failures in production pipelines.
Note: This is already tracked in .pylintrc line 139:
raise-missing-from, #TODO(https://github.com/apache/beam/issues/21169) Enable and fix warnings
Issue Priority
Priority: 3 (P3 - Code quality improvement)
Issue Components
What would you like to happen?
Currently, many exception handlers throughout the Python SDK re-raise exceptions without using
raise ... from, which loses the original traceback context. This makes debugging significantly harder because developers can't see the root cause when exceptions are wrapped.Example Problem:
When this fails, you only see:
But you don't see the original
ConnectionTimeoutErrorwith its full traceback.Desired Solution:
Now you see:
I would like to systematically add exception chaining throughout the codebase. This involves:
from eclause - Updateraise NewException(...)toraise NewException(...) from eraise-missing-fromwarning in.pylintrc(currently disabled with TODO)This enhancement will significantly improve debuggability by preserving the full exception context, making it easier to identify root causes of failures in production pipelines.
Note: This is already tracked in
.pylintrcline 139:Issue Priority
Priority: 3 (P3 - Code quality improvement)
Issue Components