Failure: PR #1015 Stress test (amd_debug) - Segmentation fault (signal 11, NULL pointer) during SELECT shardNum()

[CarlosFelipeOR]

Version: 25.8.9.20448.altinityantalya

Description

A segmentation fault occurred during the execution of the stress tests in workflow run
PR #1015, specifically in the job “Stress test (amd_debug)”, which was triggered by Pull Request #1161.
The failure was detected by the ClickHouse server, which terminated with signal 11 (Segmentation fault) caused by a NULL pointer read during query processing.

The crash happens while executing the following query:

SELECT shardNum() AS _shard_num, __table1.key
FROM test_1.dist_2 AS __table1
ORDER BY shardNum(), __table1.key

Main Error Extract

<Fatal> BaseDaemon: Received signal 11
<Fatal> BaseDaemon: Signal description: Segmentation fault
<Fatal> BaseDaemon: Address: NULL pointer. Access: read. Address not mapped to object.

The stack trace indicates the failure occurs inside the AST type promotion / casting logic, referencing the following files:

Common/typeid_cast.h
Common/TypePromotion.h
Interpreters/executeQuery.cpp
Server/TCPHandler.cpp

Access job report to check the full error message

---

CI Run

https://github.com/Altinity/ClickHouse/actions/runs/19671147070/job/56352115318

Artifacts and Reports

• Workflow report: https://s3.amazonaws.com/altinity-build-artifacts/PRs/1161/6b0768017f4eb215c8db1e95bb43bdcd367d191e/19671147070/ci_run_report.html
• Job report: https://altinity-build-artifacts.s3.amazonaws.com/json.html?PR=1161&sha=6b0768017f4eb215c8db1e95bb43bdcd367d191e&name_0=PR&name_1=Stress%20test%20%28amd_debug%29

[CarlosFelipeOR]

Upstream Error Database Analysis

I queried the upstream error database to check whether this crash pattern had been seen before.
Using the following query:

SELECT * FROM "default"."checks" 
WHERE test_name LIKE '%Fatal message in clickhouse-server.log%'
AND check_name LIKE 'Stress test%'
AND test_context_raw LIKE '%TypePromotion%'
ORDER BY check_start_time DESC
LIMIT 100

I found 46 historical occurrences across different ClickHouse versions and different queries. The most recent upstream occurrence was recorded on 2025-04-29 in version 25.5.1.1. Based on this, similar issues appear to have been previously observed on upstream, but are not present in the most recent upstream versions.

[CarlosFelipeOR]

Altinity stress test history

I also checked the Altinity internal database for recent stress test runs.

Considering the last 80 executions across all four ClickHouse build variants
(amd_msan, amd_ubsan, amd_tsan, amd_debug) in both MasterCI and PR workflows:

• The stress test failed 11 times.
• Only 2 of these failures showed the same TypePromotion-related error on the
  SELECT shardNum() query — and both occurred recently, on 2025-11-25.
• After these two occurrences, the stress test has been executed again multiple times in CI/CD
  and passed successfully, which indicates that this is not a new persistent or reproducible failure.

Most of the remaining failures were caused by unrelated issues, such as:

• Code: 48. DB::Exception: Disk does not support WriteMode::Append. (NOT_IMPLEMENTED)
• Aborted executions (signal 6) in other subsystems (e.g., Parquet / S3 read paths)

Conclusion:
The TypePromotion / shardNum() crash appears to be a rare intermittent failure, not a consistent regression.