Letsencrypt certificate not used after renewal

[globo]

Hi, I have juste renewed my SSL-certificate. No errors.

Code: Select all

certbot certificates

shows my new valid certificate.

However, lighttpd still serves the old (expired) certificate.
Looking at the lighttpd config file, there are no SSL entries.

Where does lighttpd look for the certificates? /etc/letsencrypt/live ?
That's where the new ones I generated are.

Confused...

[globo]

Found a solution. Only two files are generated. You need to combine them.

Code: Select all

 cat /etc/letsencrypt/live/yourdomain/privkey.pem /etc/letsencrypt/live/yourdomain/fullchain.pem > /etc/letsencrypt/live/yourdomain/combined.pem

[baz123]

I created a small script to do this automatically https://tech.borpin.co.uk/2019/03/22/le ... -lighttpd/

[MichaIng]

@globo @baz123
This combining step is actually already implemented if you setup the certificate with dietpi-letsencrypt. Check:

Code: Select all

cat /etc/systemd/system/certbot.service.d/dietpi-lighttpd.conf

If your certificate does not renew automatically, so you e.g. need to run certbot renew manually, then I guess it is due to the old authentication module we used that can only run when Lighttpd service is disabled. To solve this (hence have certs renewed automatically), adjust the method:

Code: Select all

certbot renew --force-renewal --webroot -w /var/www

[baz123]

This combining step is actually already implemented if you setup the certificate with dietpi-letsencrypt.

Great. From what version was this implemented?

@MichaIng unless something has changed, I'm not aware you can setup the DNS-01 challenge method of getting certificates via dietpi-letsencrypt.

[MichaIng]

I am a bid lazy currently to find the exact version, but it's in for more than 1 year, AFAIK.

Yeah dietpi-letsencrypt currently does not support DNS challenges. There are requests for this already, since this is the only way to receive wildcard certificates, but the implementation is not thaat easy since it cannot be fully automated, AFAIK. It requires parsing some string from the challenge output and creating a related file in webroot. And I am not sure how reliable we can get this string automatically, due to possible output changes, multi line output and magic characters. Currently one can do this challenge manually, then use dietpi-letsencrypt to apply webserver settings and renewal tasks, e.g. combining key+cert for Lighttpd automatically on renewal. If a certificate already exists, dietpi-letsencrypt will only call certbot renew, so will not change any challege settings.

[baz123]

Yeah dietpi-letsencrypt currently does not support DNS challenges.

then use dietpi-letsencrypt to apply webserver settings and renewal tasks

Ah OK. The initial setting up of the DNS challenge can be done via a plugin for some DNS hosts (I use cloudflare and the right plugin).

What I understand from what you have said, is that if the initial certificates are issued - no matter how that was done by certbot, then running dietpi-letsencrypt would pick up the fact the certificates already exist and use them to setup Lighttpd correctly?

The other advantage to DNS challenge is that the machine in question is not actually exposed to the internet for the challenge and any subdomain does not even need to exist on the DNS. Ideal for internal web based systems that Chrome complains are not HTTPS!

There are now quite a number of DNS plugins now available https://certbot.eff.org/docs/using.html#dns-plugins. It seems to be a rapidly evolving ecosystem. The real issue for yourself is that the format of the credentials files are all different!

[MichaIng]

Yeah I know about the DNS challenge plugins, although this is nothing we can easily use/automate due to the wide range of CAs and plugins.

Hmm, just read a bid further into it, DNS TXT file edit/creating cannot be automated, as long as the provider does not have an API. And the APIs vary (so the plugins). So actually we cannot do much more than automate some options based on input mask, print the DNS TXT output of Certbot and ask the user to add it via whatever method provided by the provider, so basically follow Certbot instructions which already waits for user input to continue.

And the big problem is that this method does not allow automated cert renewal of course, since the same manual steps need to be done every time...

Okay looks like we need to stick with webroot/browser methods, any only allow optional manual DNS method with the hint that also renewal needs to be done manually. As said implementing a bunch of known DNS APIs that in case need to be maintained/updated and still will not cover all existing providers, is not something we gonna do.

---

Ah there is also a Python-based automation script available: https://github.com/EnigmaBridge/certbot-external-auth
pip install certbot-external-auth, requires Python-pip, but worth having a look at. Probably it has all known APIs included.

Although longterm I either want to add or replace certbot with acme.sh, which is MUCH more lightweight and does not require Python: https://github.com/Neilpang/acme.sh
But the problem that there is no standard API to automate, is the same of course.

[baz123]

Yeah I know about the DNS challenge plugins, although this is nothing we can easily use/automate due to the wide range of CAs and plugins.

Yes I suspected so.

If we are able to reliable automate it, I think we can make this the default method, so only one method that allows unused and wildcard certificates.

So would my idea work with the existing setup? The user could use certbot to create the initial certificate directly, then run dietpi-letsencrypt to use and manage the certificate for the installed web server?

[MichaIng]

I updated text above, read a bid more into it.

Yeah your plan will work, at least with Lighttpd. For the other webservers we use the Certbot internal installer method (via Nginx and Apache plugin), so creating certs manually and run dietpi-letsencrypt afterwards, will not install those certs to the webservers.
So it makes sense that we integrate it partly:
- Using the dietpi-letsencrypt mask to enter info.
- Allow to choose DNS challenge for wildcard certs and subdomains which do not resolve to the local webserver.
- In case of DNS challenge selected, show an additional input field to add those additional domains. It makes sense to have a "main" domain field separatly, which is used to give the webserver a server name.
- In case of DNS challenge then inform the user to follow Certbot instructions and update the DNS record manually before continuing.

In case of Nginx and Apache, Certbot as well allows to separate authentication and install methods, so we can stay with Apache/Nginx automated cert+key+SSL config install and use DNS challenge for authentication only.

---

Hmm, acme.sh has a very large amount of DNS APIs included: https://github.com/Neilpang/acme.sh/tree/master/dnsapi
This would be much easier to implement natively, since we can simply scrape the directory and list the files. Their names should allow users to select the correct one for their provider, otherwise select manual. Certbot plugins are much less available and we would need to call APT to scan for available plugins... and as far as I could see, the required option to call the plugin + required input fieds are not consistent as well.