Connection timeout does not cause origin to be marked down

[vuori]

As far as I understand from the documentation, hosts that are unresponsive are supposed to be marked as "down" in hostdb and retried according to the connect_attempts_max_retries_down_server setting. However with the following steps this does not seem to be happening:

1. Attempt to make a forward-proxied request to a host that is not responding at all (error is ETIMEDOUT).
2. Note in trace log that connect_attempts_max_retries attempts are made.
3. Try the same request again.
4. connect_attempts_max_retries attempts are still made, the count never goes down to connect_attempts_max_retries_down_server.

In particular if the latter is set to 0, ATS still always tries to connect and the trace message Not connecting to the server because it is marked down from HttpSM.cc never appears.

Is this a bug or am I misunderstanding how hostdb is supposed to work?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. Marking it stale to flag it for further consideration by the community.

[bryancall]

This was a real defect, and it is now fixed. Before the fix, none of the three origin connect-attempt retry settings were applied correctly: an unresponsive origin kept receiving the full connect_attempts_max_retries attempts on every request, the host never transitioned to the down state, and connect_attempts_max_retries_down_server (even set to 0) never took effect, so the "Not connecting to the server because it is marked down" trace never appeared.

PR #13102 ("Fix connect attempt retries") reworks the retry path so it honors the HostDB state of the origin: a recovering (suspect) origin gets the limited probe budget, and an origin that is fully down gets zero connect attempts. It also fixes the failure-count threshold used to mark a host down and timestamps each connect attempt distinctly so a host actually reaches the down state, and it adds tests for the down/recovering behavior. As part of the change the setting was renamed from proxy.config.http.connect_attempts_max_retries_down_server to proxy.config.http.connect_attempts_max_retries_suspect_server to match the recovering (suspect) state it actually governs, with the old name kept working via a deprecation warning.

This change is on master and first ships in the 10.2.0 release. Closing as fixed. Please reopen if you still see a fully unresponsive origin getting more than the configured attempts on 10.2.0.