[dns-01] Idea: actively wait for DNS "propagation"

[JulienPalard]

The code in dns_common.py tells:

# DNS updates take time to propagate and checking to see if the update has occurred is not
# reliable (the machine this code is running on might be able to see an update before
# the ACME server). So: we sleep for a short amount of time we believe to be long enough.

but I do not agree: there's no such thing as "DNS propagation" there's "DNS cache invalidation".

So I bet the only fix this option provides is to wait for the providers to actually setup their DNS after their receive the order from their API. For example I use OVH in France, and it can take like 110s±100s to setup in their side, so either I set the wait time to 300s, either I have random failures (it's not a cache invalidation issue, it's just the time it take for their script to update their DNS configuration).

If that's true, active waiting should work, but we should not send the DNS requests to the locally configured resolver (with cache, and thus invalidation again), we should send them directly to the right name server, dig-speaking it would not look like:

dig _acme-challenge.mdk.fr TXT +short

but:

dig @$(dig mdk.fr NS +short | head -n 1) _acme-challenge.mdk.fr TXT +short

(I know a Python POC would be nicer than a bash/dig POC, but I bet the dig one is shorter?)

Am I wrong?

[alexzorin]

To clarify one thing: Certbot doesn't perform any DNS queries while performing its role in a DNS-01 challenge.

Certbot merely signals to the Certificate Authority that "OK, I've set up the challenge, please check it for me". The propagation-seconds option in Certbot allows users to delay that signal, so that the authoritative DNS host has time to reload their nameservers, do all the AXFRs, propagate it through their anycast network, invalidate any transparent DNS caches etc.

Regarding recursive resolver caches and querying authoritative nameservers directly: this is the responsibility of the Certificate Authority and the exact policies will vary between them.

To the best of my understanding, Let's Encrypt minimize their resolver cache to quite an extreme level, so practically speaking, stale records will never be seen, especially if you have a propagation-seconds delay of at least 60 seconds. There's a lot more written about that at https://community.letsencrypt.org/.

I'm going to close this as I don't think there's anything actively relevant to the Certbot project itself, but feel free to open a discussion at https://community.letsencrypt.org or comment back if you feel there's something that should change in Certbot.

[JulienPalard]

I understand certbot don't do DNS requests, what I'm proposing is to make certbot do them instead of blindly waiting for the authoritative DNS host to do their thing, we could actively probe them by sending DNS requests, it would go like:

ask_host_to_set_dns()
while not host_has_set_dns(): # Does the request just to see if it's done or not
    sleep(1)
tell_ca_the_challenges_are_setup()

instead of:

ask_host_to_set_dns()
sleep(300)
tell_ca_the_challenges_are_setup()

Idea is my host has a very random setup time, it can range from 10s to more than 200s, I don't really know yet, which is unconfortable: I started with the default of 60s if was failing randomly, I doubled it, it still fails randomly, I doubled it again in the hope it'll be OK this time, but I bet I'll still get very rare random failures.

By actively probing I hope it would have worked without trial-and-error/guesswork.

[alexzorin]

If the defaults sleeps in Certbot for certain DNS plugins are not reliably long enough, they should be changed. It's possible that the value that was chosen for OVH 5 years ago is no longer suitable.

Doing recursive queries from Certbot could be worth exploring. I think acme.sh, lego and cert-manager do something like that, though I do recall helping multiple users disable that preflight check because its behavior just wasn't correct in some situations. Certbot's current approach of sleeping seems like a good balance between reliability and complexity.

I'll reopen this for further discussion if anybody else is interested in having this.

[JulienPalard]

Certbot's current approach of sleeping seems like a good balance between reliability and complexity.

That's very true, it's like a simple line of code vs hundreds of lines (proably in a new dependency like dnspython).

I think, if we implement the active polling, we should keep the good old constant sleep, (which could be reduced to near 0 by default if active polling works well), so users with issues in active polling (not being able to do DNS queries for reasons) would gracefully fallback to constant time sleep.

[JulienPalard]

I'm doddling around the idea here: https://github.com/JulienPalard/dnswait too see where it goes first.

[JulienPalard]

So, my dnswait demo works, but I see no activity in the issue.

Maybe there's nobody in this thread just because everyone think DNS propagation is a thing and waiting a constant time is normal?

[osirisinferi]

I disagree there is no such thing as DNS propogation. Usually, DNS providers have multiple DNS servers for a single IP address using anycast around the world. If you set a certain DNS resource record at location X it can take a certain amount of time for this RR to propogate to all DNS servers world wide, even if you directly query the authorative DNS servers. This process to me can perfectly be called "propogation", as the RR propogates around the world.

Actively requesting a RR on authorative server at location X might be succesful, while the same RR request might fail on the authorative server at location Y. Therefore simply querying the authorative DNS servers at a single location as dnswait does might not be enough.

Also, I think many users are fine with just waiting 🙂 The single thumbs up on your OP might resemble this, so I would encourage any user also wanting this feature to give a thumbs up.

[JulienPalard]

it can take a certain amount of time for this RR to propogate to all DNS servers world wide, even if you directly query the authorative DNS servers.

You're right (that's just not what people think about when they speak about DNS propagation), to avoid ambiguity I'd call it a "setup time" maybe?

But you raise a very interesting point: it's not because my dnswait tells it's OK that it'll be OK from letsencrypt point of view (potentially hitting another server), so a constant time have to be added anyway to cover this.

Also, I think many users are fine with just waiting

Yes, as it's executed most of the time from a cron, you're right.

My initial idea was because I had to use a low cost provider that can randomly take from a few seconds to around 3 minutes to setup their DNS, so I had to bump my --dns-ovh-propagation-seconds multiple times to have it pass consistently.

Bumping the value for the 3rd time I though "we could do better, like actively waiting instead of hardcoding the value" (but it means adding code, so adding possible other failures...).

I don't like open dormant issues, so I'm closing this: if you're reading this in the future and think it's a good idea, please tell.

[osirisinferi]

I think that in many situations querying the authorative DNS servers and waiting an extra, but smaller amount of time might be working for many DNS providers, but there are absolutely no guarantees. I.e.: YMMV.

Without guarantees and probably a relative significant effort to include this feature into Certbot, I think this isn't something implemented anytime soon even if the Certbot team would agree on adding it.

Users can always revisit this feature request in the future, might the need become high.