Koozali.org: home of the SME Server

Solved - letsencrypt challenge not completing

Offline mercyh

  • *
  • 824
  • +0/-0
    • http://mercyh.org
Solved - letsencrypt challenge not completing
« on: June 10, 2017, 07:00:31 PM »
Good Morning,
I have been unsuccessful in getting my serve to complete the challenge for letsencrypt. Here is the output of dehydrated -c

Code: [Select]
ERROR: Challenge is invalid! (returned: invalid) (result: {
  "type": "http-01",
  "status": "invalid",
  "error": {
    "type": "urn:acme:error:unauthorized",
    "detail": "Invalid response from http://www.mercyh.org/.well-known/acme-challenge/FMTvp_V7itHCGh1yK9AjZJv2rmGawZYArY-5r3DpTX4: \"\u003c!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\"\u003e\n\u003chtml\u003e\u003chead\u003e\n\u003ctitle\u003e403 Forbidden\u003c/title\u003e\n\u003c/head\u003e\u003cbody\u003e\n\u003ch1\u003eForbidden\u003c/h1\u003e\n\u003cp\"",
    "status": 403
  },
  "uri": "https://acme-staging.api.letsencrypt.org/acme/challenge/FgNcud1f0aqdQ7akFN1NyrEz6BkKLGFTHN_HlxC7cUE/43197390",
  "token": "FMTvp_V7itHCGh1yK9AjZJv2rmGawZYArY-5r3DpTX4",
  "keyAuthorization": "FMTvp_V7itHCGh1yK9AjZJv2rmGawZYArY-5r3DpTX4.266BsZK-dHl_Lk8qUZQa6lxP_cNRbxz8lP3lFEP_1Rs",
  "validationRecord": [
    {
      "url": "https://www.mercyh.org/.well-known/acme-challenge/FMTvp_V7itHCGh1yK9AjZJv2rmGawZYArY-5r3DpTX4",
      "hostname": "www.mercyh.org",
      "port": "443",
      "addressesResolved": [
        "63.245.178.234"

the httpd error_log file simply gives the following entry:

Code: [Select]
(13)Permission denied: access to /.well-known/acme-challenge/fxiFyqyaNhfd7DQVFHfl2LgUdOQr-AZOyCuC10UCK-Q denied
(13)Permission denied: access to /.well-known/acme-challenge/cVQNMIHnlXVdH0qUUpcMyQ8xlbU-4eMmT-pr6GFE7FE denied
(13)Permission denied: access to /.well-known/acme-challenge/GdfqjRvzhE4viN5pY7xQzdcEJan-veIi9FOvBxkPz68 denied

can anyone give me a pointer toward what I am doing wrong?
« Last Edit: December 10, 2019, 08:53:38 AM by TerryF »

Offline DanB35

  • ****
  • 764
  • +0/-0
    • http://www.familybrown.org
Re: letsencrypt challenge not completing
« Reply #1 on: June 10, 2017, 09:03:01 PM »
What are the permissions on /home/e-smith/files/ibays/Primary/html/.well-known and /home/e-smith/files/ibays/Primary/html/.well-known/acme-challenge?
......

Offline Jean-Philippe Pialasse

  • *
  • 2,746
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: letsencrypt challenge not completing
« Reply #2 on: June 10, 2017, 11:25:47 PM »
Also are you running the commands as root ? If not try as root.

Offline mercyh

  • *
  • 824
  • +0/-0
    • http://mercyh.org
Re: letsencrypt challenge not completing
« Reply #3 on: June 11, 2017, 12:03:57 AM »
I knew it was simple. I had no domains hosted on Primary and somewhere down the line, the permissions on primary got skewed. I never even considered that the Primary Ibay was used here.

correcting the permissions made the challenge work perfectly.

THANKS,

Royce

guest22

Re: letsencrypt challenge not completing
« Reply #4 on: June 11, 2017, 01:26:44 AM »
I never even considered that the Primary Ibay was used here.


In my opinion (and some others) that remains a problem, letsencrypt demanding http access to the Primary domain.

Offline DanB35

  • ****
  • 764
  • +0/-0
    • http://www.familybrown.org
Re: letsencrypt challenge not completing
« Reply #5 on: June 11, 2017, 01:29:18 AM »
In my opinion (and some others) that remains a problem, letsencrypt demanding http access to the Primary domain.
If you don't want to give http access to your SME server (HTTPS access is OK, because SME implements a HTTP -> HTTPS redirect), you can always use DNS validation instead--but you'll have to script it yourself.
......

guest22

Re: letsencrypt challenge not completing
« Reply #6 on: June 11, 2017, 01:34:57 AM »
AFAIK and in my experience (see also comments from Stafan) http is required by letsencrypt. e.g. having forced the Primary ibay inot httpS only, letsencrypt will fail.

Offline DanB35

  • ****
  • 764
  • +0/-0
    • http://www.familybrown.org
Re: letsencrypt challenge not completing
« Reply #7 on: June 11, 2017, 01:48:42 AM »
AFAIK and in my experience (see also comments from Stafan) http is required by letsencrypt. e.g. having forced the Primary ibay inot httpS only, letsencrypt will fail.
From my own experience, this is incorrect.  When an ibay (including Primary) is set to HTTPS-only on SME, the system implements a HTTP -> HTTPS redirect--if you connect to http://yourhostname, it's redirected to https://yourhostname.  The Let's Encrypt servers will follow this redirect and authenticate without issue.  Now, if you have a firewall blocking access on port 80, the LE servers won't be able to validate, but that isn't an SME setting.  It also isn't generally a good idea--even if you want your server to be SSL-only, people are going to (attempt to) connect from the outside using HTTP.

But, again, if you don't want to open your SME server to web access, you can use DNS validation instead.  You'll need to write some custom template fragments for that, though.  See https://github.com/lukas2511/dehydrated/blob/master/docs/dns-verification.md and https://github.com/lukas2511/dehydrated/wiki/Examples-for-DNS-01-hooks for more information on implementing DNS validation with dehydrated.

Edit:  There's a third option, if the hostname for which you're seeking the cert resolves to a different machine: deploy the challenge file automatically to the appropriate host.  In the simplest case, the contrib can handle the necessary automation.  See https://wiki.contribs.org/Letsencrypt#Obtaining_certificates_for_a_private_SME_Server.  More complicated cases will probably need custom template fragments.
« Last Edit: June 11, 2017, 04:02:34 AM by DanB35 »
......

Offline mercyh

  • *
  • 824
  • +0/-0
    • http://mercyh.org
Re: letsencrypt challenge not completing
« Reply #8 on: June 11, 2017, 02:36:15 AM »
I actually set the server to force SSL while I was trying to troubleshoot. The logs showed letsencrypt following the redirect but then failing with the permissions error. I made the assumption that it was following the domain direct on the server to the ibay where the site is hosted. I never considered that it was accessing the primary ibay and the paths in both the letsencrypt reply and the httpd logs were not complete. Thanks again for pointing me in the right direction!!!

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: letsencrypt challenge not completing
« Reply #9 on: June 11, 2017, 02:33:12 PM »
Letsencrypt effectively HAS to follow a http redirect or logically you could not disable http access.

Remember that (if you are sensible) http would be disabled once you have your certs installed, but the auto renew still works....

All my boxes with letsencrypt have http disabled/redirected and I have zero issues.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline DanB35

  • ****
  • 764
  • +0/-0
    • http://www.familybrown.org
Re: letsencrypt challenge not completing
« Reply #10 on: June 11, 2017, 08:55:33 PM »
To be completely clear, every time Let's Encrypt issues a certificate (which will be about every 60 days if you're following their recommendations), they must validate that you control every hostname for which you're seeking a cert.  Their servers support three methods of validation:
  • Look for a specific file under http://$FQDN/.well-known/acme-challenge
  • Look for a specific TLS certificate to be served when connecting to https://$FQDN
  • Look for specific DNS TXT records for each hostname sought to be validated

Dehydrated, the client that @ReetP's contrib works with, supports only the first and third challenges, and the contrib supports only the first--it wouldn't be possible to come up with a consistent set of instructions for the third, since we're all using different DNS hosts.  That means that, any time you're seeking to issue or renew a certificate, the Let's Encrypt servers must be able to connect to your hostname on port 80.  They'll happily follow a redirect, but they must still be able to connect on port 80.  If you don't want port 80 open to the Internet, yes, this is a problem for you.
......

Offline Jean-Philippe Pialasse

  • *
  • 2,746
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: letsencrypt challenge not completing
« Reply #11 on: June 11, 2017, 11:05:23 PM »
However HF might have a point here, why put .well-known in the Primary ibay, it could be somewhere else and still work.

We could move it to /var/www/.well-known and make the httpd.conf template fragment to point http://mydomain.com/.well-known to it.
This would avoid customized chmod or chown on the Primary ibay to prevent apache to read the content of .well-known.
Also it would keep it away from the eyes of users not knowing what this is.

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: letsencrypt challenge not completing
« Reply #12 on: June 11, 2017, 11:14:27 PM »
If you have more than domain they may point to different ibays....

I need to check my configs to demonstrate this but I think this proposal may break some stuff for me at least.

I think it is the due to a situation Tony showed me some while back regarding domains but need to check my configs.

Also if we start moving stuff to /var/www then either it should all go in there, or stay the way we are. Splitting locations could confuse some just as easily as helping others
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline DanB35

  • ****
  • 764
  • +0/-0
    • http://www.familybrown.org
Re: letsencrypt challenge not completing
« Reply #13 on: June 11, 2017, 11:26:30 PM »
why put .well-known in the Primary ibay,
Because the Primary ibay is the default web root of the SME Server, and therefore the logical place to put anything that's going to be served out of the web root.  It doesn't have to go there, of course--neither dehydrated nor the Let's Encrypt servers care where on the filesystem the challenge file goes, as long as it's served in response to a request for /.well-known/acme-challenge/whatever--but it does seem like the most logical place to put it.  Why not put it there?
......

Offline Jean-Philippe Pialasse

  • *
  • 2,746
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: letsencrypt challenge not completing
« Reply #14 on: June 11, 2017, 11:37:52 PM »
If you have more than domain they may point to different ibays....
if a domain point to an ibay else than Primary, then it is already not pointing tothe Primary and hence there is already a fragment making the url well-known to point to the folder in the ibay, it could then point to somewhere else.


I need to check my configs to demonstrate this but I think this proposal may break some stuff for me at least.
go ahead, indeed i feel some tweaks might, at least with use of the hook script migh break

I think it is the due to a situation Tony showed me some while back regarding domains but need to check my configs.

Also if we start moving stuff to /var/www then either it should all go in there, or stay the way we are. Splitting locations could confuse some just as easily as helping others

well here we have a software indepedant from user data, and with nothing to backup from it, its place should be naturally out of /home/. Another example is horde that sit for a while in /home, but should be moved to a standard path, and has been for 10. Phpmyadmin is not either in /home....

So user accessible web data are kept in /home/e-smith, while software path, not needed to be accessible by the user are out of it.

Because the Primary ibay is the default web root of the SME Server, and therefore the logical place to put anything that's going to be served out of the web root.  It doesn't have to go there, of course--neither dehydrated nor the Let's Encrypt servers care where on the filesystem the challenge file goes, as long as it's served in response to a request for /.well-known/acme-challenge/whatever--but it does seem like the most logical place to put it.  Why not put it there?
Because this does not need to be accessible by the user. It is only for automatized access for cert renewal. I think it was placed in the Primary  ibay by John in the first time only because it was conveniant while developing the contribution and did avoid to play with the handling of virtual path vs absolute server path, but when it was time to handle also domain pointing to other ibays he had to play with alias or other means to point to the current absolute path of .well-known.