Koozali.org: home of the SME Server

How well would NextCloud work in an i-bay?

Offline Mace

  • **
  • 65
  • +0/-0
How well would NextCloud work in an i-bay?
« on: October 04, 2021, 11:20:25 AM »
Last week I realized that my letsencrypt certs' auto-renew was failing. I had to set my Nextcloud domain (referred to as nc.mymain.domain in this post) to temporarily not use letsencrypt to get the others updated (including mymain.domain).

For the better part of a week now I've been trying to renew my letsencrypt cert for my Nextcloud domain renewed but can't get it to succeed. It consistently fails with
Code: [Select]
+ Handling authorization for nc.mymain.domain
 + 1 pending challenge(s)
 + Deploying challenge tokens...
 + Responding to challenge for nc.mymain.domain authorization...
 + Cleaning challenge tokens...
 + Challenge validation has failed :(
ERROR: Challenge is invalid! (returned: invalid) (result: {
  "type": "http-01",
  "status": "invalid",
  "error": {
    "type": "urn:ietf:params:acme:error:unauthorized",
    "detail": "Invalid response from https://nc.mymain.domain/login [redacted external IP]: \"\u003c!DOCTYPE html\u003e\\n\u003chtml class=\\\"ng-csp\\\" data-placeholder-focus=\\\"false\\\" lang=\\\"en\\\" data-locale=\\\"en\\\" \u003e\\n\\t\u003chead\\n data-requesttoken=\\\"Jwm6\"",
    "status": 403
  },
  "url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/redacted",
  "token": "redacted",
  "validationRecord": [
    {
      "url": "http://nc.mymain.domain/.well-known/acme-challenge/redacted",
      "hostname": "nc.mymain.domain",
      "port": "80",
      "addressesResolved": [
        "redacted external IP"
      ],
      "addressUsed": "redacted external IP"
    },
    {
      "url": "https://nc.mymain.domain/",
      "hostname": "nc.mymain.domain",
      "port": "443",
      "addressesResolved": [
        "redacted external IP"
      ],
      "addressUsed": "redacted external IP"
    },
    {
      "url": "https://nc.mymain.domain/login",
      "hostname": "nc.mymain.domain",
      "port": "443",
      "addressesResolved": [
        "redacted external IP"
      ],
      "addressUsed": "redacted external IP"
    }
  ],
  "validated": "2021-10-04T09:04:07Z"
})

I had the same trouble with it when I originally set it up. I looked at all my command history but the solution has escaped me now. :sad: It seems it may be related to it looking for the response on http (port 80) instead of https (port 443). I apologize for my increasing ignorance on all this, but could Nextcloud work well in an i-bay? I wondered if it may be easier since mymain.domain cert did successfully renew, but i still get a certificate error on nc.mymain.domain, which will not successfully renew, and is served from /usr/share/nextcloud.
« Last Edit: October 04, 2021, 11:34:11 AM by Sterling »

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: How well would NextCloud work in an i-bay?
« Reply #1 on: October 04, 2021, 10:14:15 PM »
Read this for why you should fix the original issue properly and not try and bodge your way around it.

https://xyproblem.info

IIRC it does need 80. And it should just work.

So start here:

Quote
Invalid response from https://nc.mymain.domain/login

Hmm.

Better tell us how you have your domains & server set up.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline Mace

  • **
  • 65
  • +0/-0
Re: How well would NextCloud work in an i-bay?
« Reply #2 on: October 10, 2021, 03:13:39 AM »
Read this for why you should fix the original issue properly and not try and bodge your way around it.

https://xyproblem.info

IIRC it does need 80. And it should just work.

So start here:

Hmm.

Better tell us how you have your domains & server set up.

I believe I should probably use my backup and start fresh install. Had an incident happen where a lot of my recollection from the last few months is fragmented and I don't know what setup to post here. AFAIK my setup is by the book with no custom templates other than the temporary fix for https://bugs.koozali.org/show_bug.cgi?id=11641 which seems to be resolved now, and /etc/e-smith/templates-custom/etc/dar/DailyBackup.dcf/41go-into for extra locations to back up, and /etc/e-smith/templates-custom/usr/share/horde/ingo/config/backends.local.php/100Ingo which i assume was from a contrib. I had just successfully updated to nextcloud version 20.0.13.1 via the web interface before noticing the expired certs, so I don't know if that update could have anything to do with it or not.

Oh, I did enable "pretty urls" as soon as I got nextcloud running a few months ago - https://docs.nextcloud.com/server/20/admin_manual/installation/source_installation.html?#pretty-urls

Offline Jean-Philippe Pialasse

  • *
  • 2,747
  • +11/-0
  • aka Unnilennium
    • http://smeserver.pialasse.com
Re: How well would NextCloud work in an i-bay?
« Reply #3 on: October 10, 2021, 05:17:02 AM »
try disabling the pretty url

normally there are element to prevent redirection to validate the domain using theedicated space, but it could happen that a httpd directive will override this....

Alias /.well-known/acme-challenge/ /home/e-smith/files/ibays/Primary/html/.well-known/acme-challenge/



see https://serverfault.com/questions/966675/apache-2-4-mod-alias-mod-rewrite-mod-proxy-execution-order
« Last Edit: October 10, 2021, 05:19:53 AM by Jean-Philippe Pialasse »

Offline Mace

  • **
  • 65
  • +0/-0
Re: How well would NextCloud work in an i-bay?
« Reply #4 on: October 10, 2021, 07:28:53 PM »
try disabling the pretty url

normally there are element to prevent redirection to validate the domain using theedicated space, but it could happen that a httpd directive will override this....

Alias /.well-known/acme-challenge/ /home/e-smith/files/ibays/Primary/html/.well-known/acme-challenge/



see https://serverfault.com/questions/966675/apache-2-4-mod-alias-mod-rewrite-mod-proxy-execution-order

Tried disabling pretty urls, but when i removed the overwrite.cli.url line from nextcloud's config.php, or set overwrite.cli.url to "" i get
Code: [Select]
[root@sme nextcloud]# occ maintenance:update:htaccess
Error updating .htaccess file, not enough permissions or "overwrite.cli.url" set to an invalid URL?

I verified it's not a permissions issue, so it seems now it doesn't want to rewrite the .htaccess without the overwrite.cli.url being defined with some valid url.

<edit>
Here is my current /usr/share/nextcloud/.htaccess for what it's worth.
Code: [Select]
[root@sme nextcloud]# cat .htaccess
<IfModule mod_headers.c>
  <IfModule mod_setenvif.c>
    <IfModule mod_fcgid.c>
       SetEnvIfNoCase ^Authorization$ "(.+)" XAUTHORIZATION=$1
       RequestHeader set XAuthorization %{XAUTHORIZATION}e env=XAUTHORIZATION
    </IfModule>
    <IfModule mod_proxy_fcgi.c>
       SetEnvIfNoCase Authorization "(.+)" HTTP_AUTHORIZATION=$1
    </IfModule>
  </IfModule>

  <IfModule mod_env.c>
    # Add security and privacy related headers

    # Avoid doubled headers by unsetting headers in "onsuccess" table,
    # then add headers to "always" table: https://github.com/nextcloud/server/pull/19002
    Header onsuccess unset Referrer-Policy
    Header always set Referrer-Policy "no-referrer"

    Header onsuccess unset X-Content-Type-Options
    Header always set X-Content-Type-Options "nosniff"

    Header onsuccess unset X-Download-Options
    Header always set X-Download-Options "noopen"

    Header onsuccess unset X-Frame-Options
    Header always set X-Frame-Options "SAMEORIGIN"

    Header onsuccess unset X-Permitted-Cross-Domain-Policies
    Header always set X-Permitted-Cross-Domain-Policies "none"

    Header onsuccess unset X-Robots-Tag
    Header always set X-Robots-Tag "none"

    Header onsuccess unset X-XSS-Protection
    Header always set X-XSS-Protection "1; mode=block"

    SetEnv modHeadersAvailable true
  </IfModule>

  # Add cache control for static resources
  <FilesMatch "\.(css|js|svg|gif|png|jpg|ico)$">
    Header set Cache-Control "max-age=15778463"
  </FilesMatch>

  # Let browsers cache WOFF files for a week
  <FilesMatch "\.woff2?$">
    Header set Cache-Control "max-age=604800"
  </FilesMatch>
</IfModule>
<IfModule mod_php7.c>
  php_value mbstring.func_overload 0
  php_value default_charset 'UTF-8'
  php_value output_buffering 0
  <IfModule mod_env.c>
    SetEnv htaccessWorking true
  </IfModule>
</IfModule>
<IfModule mod_rewrite.c>
  RewriteEngine on
  RewriteCond %{HTTP_USER_AGENT} DavClnt
  RewriteRule ^$ /remote.php/webdav/ [L,R=302]
  RewriteRule .* - [env=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
  RewriteRule ^\.well-known/host-meta /public.php?service=host-meta [QSA,L]
  RewriteRule ^\.well-known/host-meta\.json /public.php?service=host-meta-json [QSA,L]
  RewriteRule ^\.well-known/webfinger /public.php?service=webfinger [QSA,L]
  RewriteRule ^\.well-known/nodeinfo /public.php?service=nodeinfo [QSA,L]
  RewriteRule ^\.well-known/carddav /remote.php/dav/ [R=301,L]
  RewriteRule ^\.well-known/caldav /remote.php/dav/ [R=301,L]
  RewriteRule ^remote/(.*) remote.php [QSA,L]
  RewriteRule ^(?:build|tests|config|lib|3rdparty|templates)/.* - [R=404,L]
  RewriteCond %{REQUEST_URI} !^/\.well-known/(acme-challenge|pki-validation)/.*
  RewriteRule ^(?:\.|autotest|occ|issue|indie|db_|console).* - [R=404,L]
</IfModule>
<IfModule mod_mime.c>
  AddType image/svg+xml svg svgz
  AddEncoding gzip svgz
</IfModule>
<IfModule mod_dir.c>
  DirectoryIndex index.php index.html
</IfModule>
AddDefaultCharset utf-8
Options -Indexes
<IfModule pagespeed_module>
  ModPagespeed Off
</IfModule>
#### DO NOT CHANGE ANYTHING ABOVE THIS LINE ####

ErrorDocument 403 /
ErrorDocument 404 /
<IfModule mod_rewrite.c>
  Options -MultiViews
  RewriteRule ^core/js/oc.js$ index.php [PT,E=PATH_INFO:$1]
  RewriteRule ^core/preview.png$ index.php [PT,E=PATH_INFO:$1]
  RewriteCond %{REQUEST_FILENAME} !\.(css|js|svg|gif|png|html|ttf|woff2?|ico|jpg|jpeg|map|webm|mp4|mp3|ogg|wav)$
  RewriteCond %{REQUEST_FILENAME} !core/img/favicon.ico$
  RewriteCond %{REQUEST_FILENAME} !core/img/manifest.json$
  RewriteCond %{REQUEST_FILENAME} !/remote.php
  RewriteCond %{REQUEST_FILENAME} !/public.php
  RewriteCond %{REQUEST_FILENAME} !/cron.php
  RewriteCond %{REQUEST_FILENAME} !/core/ajax/update.php
  RewriteCond %{REQUEST_FILENAME} !/status.php
  RewriteCond %{REQUEST_FILENAME} !/ocs/v1.php
  RewriteCond %{REQUEST_FILENAME} !/ocs/v2.php
  RewriteCond %{REQUEST_FILENAME} !/robots.txt
  RewriteCond %{REQUEST_FILENAME} !/updater/
  RewriteCond %{REQUEST_FILENAME} !/ocs-provider/
  RewriteCond %{REQUEST_FILENAME} !/ocm-provider/
  RewriteCond %{REQUEST_URI} !^/\.well-known/(acme-challenge|pki-validation)/.*
  RewriteCond %{REQUEST_FILENAME} !/richdocumentscode(_arm64)?/proxy.php$
  RewriteRule . index.php [PT,E=PATH_INFO:$1]
  RewriteBase /
  <IfModule mod_env.c>
    SetEnv front_controller_active true
    <IfModule mod_dir.c>
      DirectorySlash off
    </IfModule>
  </IfModule>
</IfModule>
« Last Edit: October 10, 2021, 07:38:47 PM by Sterling »

Offline Mace

  • **
  • 65
  • +0/-0
Re: How well would NextCloud work in an i-bay?
« Reply #5 on: October 16, 2021, 09:19:47 PM »
I think I'll try running nextcloud exclusively on Rocky in another small KVM VM and switch my caldav/carddav services to horde on SME.

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: How well would NextCloud work in an i-bay?
« Reply #6 on: October 16, 2021, 10:58:27 PM »
So that's because you can't figure out certificates... which was the original issue?

I have had nextcloud running via the contrib for nearly a year now with no issues.

As I said before...

Quote
Read this for why you should fix the original issue properly and not try and bodge your way around it

All you are probably going to do is install it on Rocky and then go and annoy devs there because you won't understand the issue you encounter.

And if you think you can run it safely on your own setup on Rocky as opposed to the secure environmemt that is SME then "Chapeaux"......

And then mix it it SME Horde.

Really?

Please, read this, again, have a cup of tea (the solution to all IT issues) and start again.

https://xyproblem.info/

Fix your original issue. Don't bodge your way round it. Because you'll save us all hours of time when you come back with more issues you have created for yourself.

Thanks.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline Mace

  • **
  • 65
  • +0/-0
Re: How well would NextCloud work in an i-bay?
« Reply #7 on: October 17, 2021, 12:20:40 AM »
I sincerely apologize, I had an an accident that has significantly affected my memory. I'm honestly not trying to bodge my way around the issue. I read through that link you provided, but I will do so again.

I still have a feeling it may be the command "occ maintenance:update:htaccess" refusing to rewrite .htaccess when the overwrite.cli.url option is removed from nextcloud's config.php, thus breaking the
"Alias /.well-known/acme-challenge/ /home/e-smith/files/ibays/Primary/html/.well-known/acme-challenge/"
directive.
« Last Edit: October 17, 2021, 12:37:19 AM by Sterling »

Offline Mace

  • **
  • 65
  • +0/-0
Re: How well would NextCloud work in an i-bay?
« Reply #8 on: October 18, 2021, 10:30:23 PM »
I verified my VirtualHost sections for nextcloud look identical Jean-Philippe's in https://forums.contribs.org/index.php/topic,54520.msg285658.html#msg285658
My "db domains show mycloud.mydomain.com" result is identical also, and "config show nextcloud" is the same except I don't have PHPBaseDir or updater.secret defined.

Where else besides .htaccess could the "Alias /.well-known/acme-challenge/ /home/e-smith/files/ibays/Primary/html/.well-known/acme-challenge/" be being overridden? This does seem to be most likely what's happening.

I apologize for all the bother, I have been doing my best to pinpoint the root cause of the issue and address it properly, even before my OP. I know how applying "band-aids" to issues can explode badly down the road.

<edit>
I forgot to mention http://nc.mydomain.com/.well-known/acme-challenge/ instantly redirects back to my nextcloud dashboard instead of showing the empty /.well-known/acme-challenge/ directory like all my regular domains do.
« Last Edit: October 18, 2021, 10:40:28 PM by Sterling »

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: How well would NextCloud work in an i-bay?
« Reply #9 on: October 19, 2021, 10:15:04 AM »
The root cause was fixing letsencrypt which would probably have been easy.

At a guess and with no logs, the likely cause there was you probably didn't keep up to date and swap API v1 to v2

Everything after was an XY problem that has just made it worse.

You have not documented a clear path of actions so it is hard to follow a patch work quilt of guesses.

You probably ought to go right back, strip everything out, sort out letsencrypt, and start again. The only way to fix things is slowly, methodically, and with copious notes so we can repeat if necessary.

You probably ought to post the output of
Code: [Select]
/sbin/e-smith/audittools/newrpms
Code: [Select]
/sbin/e-smith/audittools/templates

Then a nice accurate list of things you have done from the start.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: How well would NextCloud work in an i-bay?
« Reply #10 on: October 19, 2021, 10:18:43 AM »
PS for simplicity using the contrib will probably save you grief.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline Mace

  • **
  • 65
  • +0/-0
Re: How well would NextCloud work in an i-bay?
« Reply #11 on: October 22, 2021, 03:50:26 AM »
The root cause was fixing letsencrypt which would probably have been easy.

At a guess and with no logs, the likely cause there was you probably didn't keep up to date and swap API v1 to v2

Everything after was an XY problem that has just made it worse.

You have not documented a clear path of actions so it is hard to follow a patch work quilt of guesses.

You probably ought to go right back, strip everything out, sort out letsencrypt, and start again. The only way to fix things is slowly, methodically, and with copious notes so we can repeat if necessary.

You probably ought to post the output of
Code: [Select]
/sbin/e-smith/audittools/newrpms
Code: [Select]
/sbin/e-smith/audittools/templates

Then a nice accurate list of things you have done from the start.

Thanks for the suggestions. I think I will follow the recommendation to start again, as I do have good daily backups. As for an accurate list of things I have done from the start, that's where I cannot remember well enough, due to an incident, to be accurate enough for any useful level of troubleshooting. I believe this will also be a good time to set up Proxmox. Would the qemu-guest-agent package would be useful in SME, or is SME too customized?

Another thing I just realized is that I did have the SME Server - contribs repo enabled in addition to the default selected ones in the Software installer settings from the beginning.

[Edited to post requested command output]

/sbin/e-smith/audittools/newrpms (pastebinned due to total post size being greater than 20k characters)
https://controlc.com/5c910d26

Code: [Select]
[root@sme ~]# /sbin/e-smith/audittools/templates
/etc/e-smith/templates-custom/etc/dar/DailyBackup.dcf/41go-into: MANUALLY_ADDED, ADDITION
/etc/e-smith/templates-custom/usr/share/horde/ingo/config/backends.local.php/100Ingo: MANUALLY_ADDED, OVERRIDE
« Last Edit: October 22, 2021, 04:27:26 AM by Sterling »

Offline ReetP

  • *
  • 3,722
  • +5/-0
Re: How well would NextCloud work in an i-bay?
« Reply #12 on: October 22, 2021, 11:59:57 AM »
Quote
As for an accurate list of things I have done from the start, that's where I cannot remember well enough, due to an incident, to be accurate enough for any useful level of troubleshooting.

Hence it is even more important to write things down as you do them.

It means that we can follow the steps and try to repeat an issue.

Without them even people without memory issues easily forget what they did.

Quote
/sbin/e-smith/audittools/newrpms
nextcloud.noarch                          20.0.1-1.el7.sme          @smecontribs
smeserver-nextcloud.noarch                1.2.0-11.el7.sme          @smecontribs

Ahhhh so that is still installed then. You can't use the contrib AND install in an ibay..... no wonder you are in a mess (well, technically you could but don't go there).

Quote
I believe this will also be a good time to set up Proxmox

Yes, but if you have not used it before you will need a bit of practice.

Is your hardware up to it?

Quote
Would the qemu-guest-agent package would be useful in SME, or is SME too customized?

SME isn't 'customized' in the way you think. It is a templating system, rather than specialist binaries.

We try and leave it as close to vanilla OS as possible, and then add tools that make configuration easier.

Personally I have never run the Guest Agent and suffered no ill effects. Someone else may have experience.

Here's how if required:

https://wiki.koozali.org/Qemu_guest_agent

I am also building a little rpm to aid installation. Just need SeƱor Fage to test it.

I'll post back when it is in the smecontribs repo.

...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline ReetP

  • *
  • 3,722
  • +5/-0
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline Mace

  • **
  • 65
  • +0/-0
Re: How well would NextCloud work in an i-bay?
« Reply #14 on: October 22, 2021, 01:16:50 PM »
Yeah, I am definitely going to start keeping track of what I (un)do. I haven't attempted to set up a nextcloud ibay yet, so that's why the contrib is still installed. It's still working well, I've just had to accept the security exception.

I'm playing around with Proxmox on a spare machine i had sitting around. I did figure out the basics and created a temp server with which I was able to grab a LE certificate for my nextcloud domain so I think that should work for now if I copy it over to my SME install. I'm running SME in a KVM/QEMU vm on my ubuntu server at the moment so it should be easy to import it to Proxmox if i decide to start using that.

I have a R9 3900X CPU sitting here that no one seems to want to buy lol. I think it will be a great CPU for a small hypervisor box, and if it works out i can sell/donate the older boxes I have set up and free up some space and potentially save electricity at the same time.