Koozali.org formerly Contribs.org

Upgraded SME9.0 -> 9.2 (new install, copied files, db, cloned config) SSI broken

more precisely, SSI directives in php output not interpreted (appear as comments in html) with V9.2, using same config and ibays as V9.0 (which did work). SSI in .shtml files OK.

have checked this:
All apache help articles / post re SSI. Tried all suggestions.
Diff'd httpd.conf's. Apart from php version and site name aliases, same
Insured no templates-custom are from V9.0, conflicting with V9.2

To be concise: I need the text/html rendered by php to go through the SSI filter, which worked on V9.0

# To use server-parsed HTML files
AddType text/html .shtml

<Directory />
    Options FollowSymLinks
    AllowOverride None
    order deny,allow
    deny from all
    allow from none
    AddOutputFilter includes .php
</Directory>

<Directory /home/e-smith/files/ibays/problemibay/html>
    Options None
    Options +Indexes
    Options +Includes
    AllowOverride None
    order deny,allow
    deny from all
    allow from all
</Directory>

Note that apache versions identical
V9.0 (working) php: LoadModule php5_module modules/libphp54-php5.so
V9.2 (not working) php: LoadModule php5_module modules/libphp5.so

Any suggestions?

Thanks;
Bill

Offline TerryF

  • grumpy old man
  • *
  • 1,577
At its most basic have you enabled Execution of dynamic content (CGI, PHP, SSI) for an iBay in the Server Manager, found under

 Information bays, select ibay to modify, setting is found there



Added: default is disabled
--
qui scribit bis legit

yes; dynamic content enabled.

something has changed, I have confirmed that V9.0 and V9.2 configs are identical, have exhausted that path.

Now, treating this as a problem to be debugged. Here's where I am at:

LoadModule filter_module modules/mod_filter.so
AddOutputFilterByType includes text/html
OR
    #FilterDeclare SSI
    #FilterProvider SSI INCLUDES "%{CONTENT_TYPE} =~ m|^text/html|"
    #FilterChain SSI

...both of which are not working.

Regards;
Bill

Offline ReetP

  • *
  • 3,034
Note that apache versions identical
V9.0 (working) php: LoadModule php5_module modules/libphp54-php5.so
V9.2 (not working) php: LoadModule php5_module modules/libphp5.so

Hmmm - I don't think so.

libphp54-php5.so

vs

libphp5.so

So, what else have you got installed that you haven't told us about?

My guess is you installed PHP5.4 somehow before and that's got lost somewhere??

Did you take a list of extra RPMs installed on your v9.0 box ?

Any particular reason why did you do a new install ?
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Hi ReetP;

"Any particular reason why did you do a new install ?"

Yes: V9.0 was cumulative upgrade of Older -> V7.x -> V8.x - V9.0
hacks upon hacks, working around legacy problems, lost track / trust of the band-aids on band-aids
site becoming unmaintainable. Not only Sme, but CMS (Xoops).
Needed to clean everything up.

"So, what else have you got installed that you haven't told us about?

My guess is you installed PHP5.4 somehow before and that's got lost somewhere??"

Ok, here's what's installed extra, both systems:
php-scl, python-scl, phpmyadmin

On working site (V9.0) ibay: php-scl version: disabled, php -v = PHP 5.3.3 (cli) (built: Mar 22 2017 12:27:09)
On broken site (V9.2) ibay: php-scl version: php54 (no scl-53 available)

Correction: On Working (V9.0) site php-scl version is Php54 (globally)

so, apparently both use same php version
"My guess is you installed PHP5.4 somehow before and that's got lost somewhere??"
If I were to guess, its php-5.3 that's awol.

"Did you take a list of extra RPMs installed on your v9.0 box ?"

here (V9.0): https://www.rossco.org/Downloads/OpenWrt/rpms_9.0.txt
here (V9.2): https://www.rossco.org/Downloads/OpenWrt/rpms.txt

PS: Thanks for taking an interest...

PPSS: Site Capta verification a repetitive annoyance...

Regards;
Bill
« Last Edit: August 15, 2018, 06:54:36 PM by rossbcan »

Offline ReetP

  • *
  • 3,034
Can't be quite right as php 5.3.x is default on all CentOS 6 based boxes so all versions of SME v9.

From your file lists you had PHP SCL installed.

I'd suggest you go back, read this page and follow instructions.

https://wiki.contribs.org/PHP_Software_Collections

I'd also suggest you don't use 5.4 as it is EOL. Minimum 5.6 (note 5.3 is EOL as far as PHP is concerened but still has security maintenance from RH)

The contrib adds a control panel to set PHP per ibay or overall.


...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Have php-scl installed on both boxes.

The working site uses php-scl54 globally
The non-working site uses php-scl54 for the ibay having problems.
For this particular site, stuck with php54 else, have to re-write a bunch of third party xoops (CMS) php modules.

I am beginning to think it is a fluke that (working box) that php output is filtered for SSI. Is this what Sme does by default, and I somehow broke it?

So, I am focusing on explicitly filtering all text/html for SSI:

deltas (httpd.conf):
global: LoadModule filter_module modules/mod_filter.so
directory:
    FilterDeclare SSI CONTENT_SET
    FilterProvider SSI INCLUDES resp=Content-Type $text/html
    FilterChain SSI
    FilterTrace SSI 1

error_log:
cat /var/log/httpd/error_log | grep mod_filter.c
[Wed Aug 15 13:50:34 2018] [debug] mod_filter.c(117): [client 192.168.1.27] SSI
[Wed Aug 15 13:50:34 2018] [debug] mod_filter.c(122): [client 192.168.1.27] SSI: type: FILE, length: 199
[Wed Aug 15 13:50:34 2018] [debug] mod_filter.c(122): [client 192.168.1.27] SSI: type: EOS, length: 0
[Wed Aug 15 13:50:34 2018] [debug] mod_filter.c(117): [client 192.168.1.27] SSI
[Wed Aug 15 13:50:34 2018] [debug] mod_filter.c(122): [client 192.168.1.27] SSI: type: FILE, length: 147
[Wed Aug 15 13:50:34 2018] [debug] mod_filter.c(122): [client 192.168.1.27] SSI: type: EOS, length: 0
...

Seems that "something" is being filtered, yet view source still has:

<div class="article-text"><!--#include virtual="/Articles/SomeFile.htp"-->

<!--#include virtual="/Files/AboutBR.htp"-->   

and, the virtual content (from included files) is missing

I no longer believe this is a Sme-Server issue, but a custom httpd config issue.

Anyway, if any suggestions / thoughts why filter invoked but content not served, love to hear them.

Thanks;
Bill

Offline ReetP

  • *
  • 3,034
I believe there are differences between runnning per ibay and runnning globally.

Personally I'd go to 5.6 globally (5.4 being EOL) and then start debugging your httpd confs from there.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Thanks 4 your suggestion.
Cannot use 5.6 globally, breaks legacy site.
I will seek help (for SSI / filter problem) on apache focussed sites.

Regards;
Bill

Offline ReetP

  • *
  • 3,034
I think most people will politely tell you to take a hike if you aren't running a supported version of PHP (and they won't include 5.3.x in that)

You really need to address your legacy issues and stop burying them... you have to face it one day, so it may as well be now.....

I know it isn't what you want to hear, but it doesn't make it any less true.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Hi rossbcan and ReetP,

Quote
I believe there are differences between runnning per ibay and runnning globally.

I have a WordPress site on SME-8.0 upgraded to SME-9.1 to 9.2 and WordPress always updated.

I have installed PHP-SCL and installed other sites on same SME-9.2. If I set PHP globally, the original site in Primary doesn't work anymore.

I set the PHP of new sites to PHP-5.6 and PHP-7.1 per i-bay and all is working fine.

Also, if you change MemoryLimit, PostMaxSize, and UploadMaxFilesize you have to respect the relation of: MemoryLimit > PostMaxSize > UploadMaxFilesize.

Original site: original PHP

Code: [Select]
#  db accounts show Primary
Primary=ibay
    AllowOverride=all
    CgiBin=enabled
    FollowSymLinks=enabled
    Group=shared
    Indexes=disabled
    MemoryLimit=512M
    Modifiable=no
    Name=Primary i-bay
    PHPBaseDir=/home/e-smith/files/ibays/Primary:/tmp
    PasswordSet=no
    Passwordable=no
    PostMaxSize=52M
    PublicAccess=global
    Removable=no
    UploadMaxFilesize=50M
    UserAccess=wr-admin-rd-group

New site: PHP-5.6
Code: [Select]
#  db accounts show mn-101
mn-101=ibay
    AllowOverride=all
    CgiBin=enabled
    FollowSymLinks=enabled
    Gid=5088
    Group=admin
    Indexes=disabled
    Name=Micronator-101.com
    PHPBaseDir=/home/e-smith/files/ibays/mn-101:/tmp
    PasswordSet=no
    PhpVersion=php56
    PublicAccess=global
    SSL=disabled
    Uid=5088
    UserAccess=wr-admin-rd-group

New site: PHP-7.1
Code: [Select]
#  db accounts show wiki
wiki=ibay
    AllowOverride=all
    CgiBin=enabled
    FollowSymLinks=enabled
    Gid=5092
    Group=admin
    Indexes=disabled
    Name=Pour MediaWiki
    PHPBaseDir=/home/e-smith/files/ibays/wiki:/tmp
    PasswordSet=yes
    PhpVersion=php71
    PublicAccess=global-pw
    SSL=enabled
    Uid=5092
    UserAccess=wr-admin-rd-group

Michel-André



I think the php version is a dead end, not the problem.
The problem is apache/Sme9.2 AddOutputFilter is not working, perhaps in general:

<Directory />
    ...
    AddOutputFilter includes .php
</Directory>

Above is default Sme httpd.conf

https://httpd.apache.org/docs/2.2/mod/mod_mime.html#addoutputfilter

clearly states that all output from .php files should go through the include filter for SSI processing (as it does on V9.0)

Make a simple testcase file: test.php and place in docroot
#contents:
<html>
<head>
<title>SSI Test</title>
</head>
<body>
<!--#echo var="DATE_LOCAL" -->
</body>
</html>
#end contents

try it: site_url/test.php

On SmeV9.0, date is displayed
On SmeV9.2, nothing is displayed. View source, will see the SSI code

I think it is important to determine whether this is a Sme V9.2 problem, or just mine

Regards;
Bill
« Last Edit: August 15, 2018, 11:08:21 PM by rossbcan »

Offline ReetP

  • *
  • 3,034
Hi rossbcan and ReetP,

I have a WordPress site on SME-8.0 upgraded to SME-9.1 to 9.2 and WordPress always updated.

I have installed PHP-SCL and installed other sites on same SME-9.2. If I set PHP globally, the original site in Primary doesn't work anymore.


Unless you believe that this is directly related here then either open a new thread or post a bug. You will just confuse things otherwise.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline ReetP

  • *
  • 3,034
I think the php version is a dead end, not the problem.

Quite possibly. However my advice above is still the same, and still correct :-)

Quote
The problem is apache/Sme9.2 AddOutputFilter is not working, perhaps in general:

Possibly, but we haven't actually yet ascertained where it is, so hard to say if it is one thing or or the other.

Next thing to suggest checking.

Can you diff your old httpd.conf and your new one and check if there are any difference sat all?

e.g are there any difference in templates/setup that may be causing a wriggle.

Note that I have checked the change log in e-smith-apache which has the httpd templates.

The last updates were in Jan 2016 and involved disabling SSL v2, v3, and TLSv1, v1.1

I can't remember when 9.2 was released so not sure how relevant that is as you may well have got the latest version with SME v9.0

Also check the mod_deflate module is loaded I guess

Code: [Select]
apachectl -M | grep deflate
or

Code: [Select]
apachectl -M | grep deflate
My test box doesn't by default. Neither do I have any AddOutputFilter lines anywhere.

Code: [Select]
grep -rn AddOutputFilter /etc/httpd/conf/httpd.conf
Code: [Select]
grep -rn AddOutputFilter /etc/httpd/conf/*
Neither can I find any references in the standard templates

Code: [Select]
cd /etc/e-smith/templates/etc/httpd/conf/httpd.conf
grep -rn AddOutputFilter *

So presumably you added a template to load mod_deflate and a template for the filter?
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

I have already done most of what you suggested, due diligence B4 bothering everyone, just to not be a lazy noob.

It is definitely not a php issue 'cuz the test.php file does not invoke php, just apache.

You are correct: /etc/e-smith/templates-custom/etc/httpd/conf/httpd.conf/85DefaultAccess
# First, we configure the "default" to be a very restrictive set of
# permissions.

<Directory />
    Options FollowSymLinks Includes
    AllowOverride None
    order deny,allow
    deny from all
    allow from none
    AddOutputFilter includes .php
</Directory>

not using mod_deflate

Is it possible to add the filter to config and do the testcase? May also have to add "Options + Includes". I really need to know whether this is a V9.2 problem implying apache update.

Regards;
Bill

« Last Edit: August 16, 2018, 12:24:19 AM by rossbcan »

Offline ReetP

  • *
  • 3,034
I have already done most of what you suggested, due diligence B4 bothering everyone, just to not be a lazy noob.

Yes, but as you haven't said that, and we are not at your terminal watching, we have absolutely no idea of what you have or haven't done. Better to go back to the start and check properly. Otherwise we end up with a http://xyproblem.info/ problem.

Quote
It is definitely not a php issue 'cuz the test.php file does not invoke php, just apache.

Most likely, but again, it is just a case of elimination and so we know where we stand. See above.....

Quote
You are correct: /etc/e-smith/templates-custom/etc/httpd/conf/httpd.conf/85DefaultAccess

And that is why it is important to explain EXACTLY what you have done. you know in your head. We don't.

Quote
Is it possible to add the filter to config and do the testcase? May also have to add "Options + Includes". I really need to know whether this is a V9.2 problem implying apache update.


As and when I get 5 minutes free I will try and take a look. Note that if it is an issue with apache, then it isn't per se a SME problem as we only run with what CentOS throw us usually.

It will be an Apache bug and that will have to be bugged upstream, though you should enter one on the bug tracker here and refer upstream.

Regarding your comment on Options + Indexes I'd suggest you give it a try yourself and see if it works.

From reading the manual it would seem that this is probably the case. You may want to do a bit more reading and testing:

https://httpd.apache.org/docs/2.2/en/mod/core.html#options

Quote
Options Directive
Includes
    Server-side includes provided by mod_include are permitted.

https://httpd.apache.org/docs/2.2/mod/mod_include.html

Quote
Enabling Server-Side Includes

Server Side Includes are implemented by the INCLUDES filter. If documents containing server-side include directives are given the extension .shtml, the following directives will make Apache parse them and assign the resulting document the mime type of text/html:

AddType text/html .shtml
AddOutputFilter INCLUDES .shtml

The following directive must be given for the directories containing the shtml files (typically in a <Directory> section, but this directive is also valid in .htaccess files if AllowOverride Options is set):

Options +Includes

For backwards compatibility, the server-parsed handler also activates the INCLUDES filter. As well, Apache will activate the INCLUDES filter for any document with mime type text/x-server-parsed-html or text/x-server-parsed-html3 (and the resulting output will have the mime type text/html).

For more information, see our Tutorial on Server Side Includes https://httpd.apache.org/docs/2.2/howto/ssi.html.

...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline ReetP

  • *
  • 3,034
I meant to paste this too

Syntax:   AddOutputFilter filter[;filter...] extension [extension] ...

E.g

AddOutputFilter INCLUDES .shtml

Yours has no Filter set??

Let us know how you get along.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

Offline mmccarn

  • *
  • 2,568
Your test file displays the date on my SME 9.2 server if I put it in .../Primary/html, make these changes to httpd.conf, and restart httpd-e-smith:

Code: [Select]
# diff -u /etc/httpd/conf/httpd.conf /etc/httpd/conf/httpd.conf.works
--- /etc/httpd/conf/httpd.conf 2018-08-16 08:20:24.345693239 -0400
+++ /etc/httpd/conf/httpd.conf.works 2018-08-16 08:20:16.794359181 -0400
@@ -1133,8 +1133,8 @@
 
 
 <Directory /home/e-smith/files/ibays/Primary/html>
-    AddType application/x-httpd-php .php .php3 .phtml
-    AddType application/x-httpd-php-source .phps
+    AddType text/html .php
+    AddHandler server-parsed .php
     php_admin_value open_basedir /home/e-smith/files/ibays/Primary/
 </Directory>

* If I don't delete the existing "AddType application/x-httpd-php .php..." directive, the file downloads as a text file instead of rendering in the browser

* It also worked to put the new lines in the global config instead of the <Directory...> section, as long as the .php "AddType" directive is removed from the <Directory...>

Offline mmccarn

  • *
  • 2,568
I also got the file to work (in my Primary ibay) by doing this:
Code: [Select]
db accounts setprop Primary AllowOverride All
expand-template /etc/httpd/conf/httpd.conf
sv t httpd-e-smith

Then adding these two lines to /home/e-smith/files/ibays/Primary/html/.htaccess:
Code: [Select]
AddType text/html .php
AddHandler server-parsed .php

[edit]
NOTE: I did not need to make any custom changes in /etc/httpd/conf/httpd.conf for this to work -- that is, I did not have to delete the application/x-httpd-php directive from the <Directory...> section
« Last Edit: August 16, 2018, 02:39:08 PM by mmccarn »

I have also had some success:

CONFIRMED: This is not a Sme-Server bug.
What I did:
Disabled remi-php (set to default) for dysfunctional ibay and globally. Ie; use Sme-Server default php version.
php -v: PHP 5.3.3 (cli) (built: Mar 22 2017 12:27:09)
Copyright (c) 1997-2010 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2010 Zend Technologies

This (native php version) is NOT a mistake. Did a Sme-server netinstall (about 2 months ago), applied all updates yesterday. The only php related changes were installing scl-php.

My best guess as to what was happening is that when apache reconciles all config, swapping in php-remi redirection, php related settings such as "AddOutputFilter includes .php" are not passed to remi php.

So, this is NOT a php version problem, but an attribute if how php-scl does its redirection and not accepting previous apache related configuration.

Since native Sme-Server php is adequate for my legacy sites, I will not pursue this further until Sme updates (php) break things.

Once again, thanks for your help. I found this "problem" to be extremely counter-intuitive and, your due diligence suggestions to be extremely necessary (although annoying - my hubris).

I am not sure if you (Sme team) want to follow up with the php-scl team with what I consider a bug.

Regards;
Bill

Offline ReetP

  • *
  • 3,034
It's annoying to not get to the bottom of the issue (and nothing is confirmed until it is confirmed !). Just realised I made a post quoting something you had already said (reading/replying on mobile is a PITA) so apologies for that.

You can check what the php-scl contrib actually does to httpd.conf

There are 3 added templates for httpd.conf (rpm -ql smeserver-php-scl)

/etc/e-smith/templates/etc/httpd/conf/httpd.conf
80phpscl-cgi
90php_rhscl

/etc/e-smith/templates-custom/etc/httpd/conf/httpd.conf (not sure why this one is custom)
20LoadModule80PHP

There ARE differences between running PHP per ibay and globally (you originally said the the working server had it globally and the non working per ibay)

Try swapping between the standard, global, and per ibay settings and diff up your httpd.conf file for starters.

Note for PHP SCL I have found I also need to add various configs per ibay or I get some complaints eg

db accounts....

PHPBaseDir=/home/e-smith/files/ibays/products/html/:/usr/share/pear/:/usr/share/pear-addons/:/tmp/:/opt/remi/php71/root/usr/share/pear/

PHPUploadDir=/home/e-smith/files/ibays/products/html/tmp/

Again it is unlikely to affect you though.

I think the issue is more likely to be how apache is configured for what you want it to do - you just need the right magic sauce to configure it correctly.

If you do think it is a bug then you should try and read upstream bug reports searching for your issue first - it is very unlikely that you would be the first to experience it. Second, add one to our bug tracker, and greater minds than mine can take a look.

Also note the comments from @mmccarn - he knows a lot more about it than me.

Ok, back to playing Samba 4 on docker....
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation

gonna have to not follow up with php-scl. Tight schedule(s) to meet.
this thread should help those with similar issues

Fyi, U helped me, thanks

[Edit: Ad links not allowed and removed]

Regards;
Bill
« Last Edit: August 17, 2018, 10:23:42 AM by ReetP »

Offline ReetP

  • *
  • 3,034
gonna have to not follow up with php-scl. Tight schedule(s) to meet.

It may surprise you that most of us have day jobs too, and SME isn't the only thing in our lives.

Quote
this thread should help those with similar issues

Unlikely as it fixes nothing, and leaves no real solution.

Unfortunately the distro can't progress when people are 'too busy' to help fix these things.

What if we were 'too busy' working to actually write code for SME?

We do our best, and when issues like this arrive we spend time (that we could be coding or earning money or spending with our families) trying to help out, but we can't do it all by ourselves.

I hope in future you can find time in your busy schedule to help out, and put back something into the software that helps earn you a crust.
...
1. Read the Manual
2. Read the Wiki
3. Don't ask for support on Unsupported versions of software
4. I have a job, wife, and kids and do this in my spare time. If you want something fixed, please help.

Bugs are easier than you think: http://wiki.contribs.org/Bugzilla_Help

If you love SME and don't want to lose it, join in: http://wiki.contribs.org/Koozali_Foundation