Koozali.org formerly Contribs.org

[SOLVED] Squid has vanished! SME8.1.

[SOLVED] Squid has vanished! SME8.1.
« on: October 09, 2014, 07:35:41 AM »
Hi.

I noticed some windows updates were taking longer than normal to download.
As I had configured squid to cache the updates a long time ago, I didn't understand why the delay.
So... I logged into the server-manager and looked for the squid log files.
There weren't any....:(

So, I logged in via console and ran "ps aux | less", which didnt show squid running at all.
I had a look in  /service/ for squid, and it wasn't there...

I tried to flush the cache using:
Code: [Select]
sv d /service/squid
echo "" > /var/spool/squid/swap.state
sv u /service/squid

it returned:
"unable to change to service directory: file does not exist"

However, if I run "squid" from the console without arguments it does nothing... ie doesn't throw a "command not found" error.

I haven't changed any settings on this server for months, so I'm not sure what has changed.
However, this could have occurred ages ago and I may not have noticed.

i had a look through the messages log. it had this:
Quote
Oct  9 15:05:55 webserver squid[3353]: Squid Parent: child process 3355 started
Oct  9 15:05:56 webserver squid[3355]:    Failed to verify one of the swap directories, Check cache.log    for

details.  Run 'squid -z' to create swap directories    if needed, or if running Squid for the first time.
Oct  9 15:05:56 webserver squid[3353]: Squid Parent: child process 3355 exited due to signal 6
Oct  9 15:05:59 webserver squid[3353]: Squid Parent: child process 3360 started
Oct  9 15:06:00 webserver squid[3360]:    Failed to verify one of the swap directories, Check cache.log    for

details.  Run 'squid -z' to create swap directories    if needed, or if running Squid for the first time.
Oct  9 15:06:00 webserver squid[3353]: Squid Parent: child process 3360 exited due to signal 6
Oct  9 15:06:03 webserver squid[3353]: Squid Parent: child process 3363 started
Oct  9 15:06:03 webserver squid[3363]:    Failed to verify one of the swap directories, Check cache.log    for

details.  Run 'squid -z' to create swap directories    if needed, or if running Squid for the first time.
Oct  9 15:06:03 webserver squid[3353]: Squid Parent: child process 3363 exited due to signal 6
Oct  9 15:06:06 webserver squid[3353]: Squid Parent: child process 3366 started
Oct  9 15:06:06 webserver squid[3366]:    Failed to verify one of the swap directories, Check cache.log    for

details.  Run 'squid -z' to create swap directories    if needed, or if running Squid for the first time.
Oct  9 15:06:06 webserver squid[3353]: Squid Parent: child process 3366 exited due to signal 6
Oct  9 15:06:09 webserver squid[3353]: Squid Parent: child process 3369 started
Oct  9 15:06:10 webserver squid[3369]:    Failed to verify one of the swap directories, Check cache.log    for

details.  Run 'squid -z' to create swap directories    if needed, or if running Squid for the first time.
Oct  9 15:06:10 webserver squid[3353]: Squid Parent: child process 3369 exited due to signal 6
Oct  9 15:06:10 webserver squid[3353]: Exiting due to repeated, frequent failures

so, I ran squid -z, as suggested, and it reported "creating swap directories" or words to that effect.

I rebooted, and the issue remains.
I tried looking at cache.log, but it also doesnt seem to exist - at least not in the server-manager logs.

I'm no expert with SME server, so I am aware that this could be a very serious case of user fault but...where has squid gone?

And if it actually has gone, what is the best way to reinstall it?

I seem to remember some page in the server manager that referred to squid settings and options to force users to use squid, or to bypass it... but after going through all the pages, I cannot find it.
Perhaps this has something to do with it?

I installed a few updates today including one that referred to squid, after I noticed this issue, just to see if that helped.
After a reboot I rechecked and issue remains.

Thanks for any advice you can provide.

Running sme8.1 in server-gateway mode...
Currently has all software updates as of today.
« Last Edit: October 10, 2014, 05:50:26 AM by joshAU »

guest22

Re: Squid has vanished! SME8.1.
« Reply #1 on: October 09, 2014, 08:25:43 AM »
JoshAU,

I'm sure people need more info. Have you examined other system log files? e.g. /var/log/messages. Yum log etc.

Have you used RPM e.g. rpm -q squid to see if the RPM is still installed etc etc

guest

Re: Squid has vanished! SME8.1.
« Reply #2 on: October 09, 2014, 08:32:15 AM »
Hi RequestedDeletion, and thanks for your advice.

i edited my original post while you posted your reply, and have included some info I found in the messages log there.

I just ran, as per your suggestion rpm -q squid, and it returned:

Quote
squid-2.6.STABLE21-7.e15_10

I had a look in /var/log/squid and found the cache.log.
It had this inside:
Quote
2014/10/09 15:05:55| Starting Squid Cache version 2.6.STABLE21 for i386-redhat-linux-gnu...
2014/10/09 15:05:55| Process ID 3355
2014/10/09 15:05:55| With 1024 file descriptors available
2014/10/09 15:05:55| Using epoll for the IO loop
2014/10/09 15:05:55| Performing DNS Tests...
2014/10/09 15:05:55| Successful DNS name lookup tests...
2014/10/09 15:05:55| DNS Socket created at 0.0.0.0, port 52092, FD 6
2014/10/09 15:05:55| Adding domain XXXXXX from /etc/resolv.conf
2014/10/09 15:05:55| Adding nameserver 10.0.0.1 from /etc/resolv.conf
2014/10/09 15:05:55| User-Agent logging is disabled.
2014/10/09 15:05:55| Referer logging is disabled.
2014/10/09 15:05:55| Unlinkd pipe opened on FD 11
2014/10/09 15:05:55| Swap maxSize 102400 + 8192 KB, estimated 0 objects
2014/10/09 15:05:55| Target number of buckets: 425
2014/10/09 15:05:55| Using 8192 Store buckets
2014/10/09 15:05:55| Max Mem  size: 8192 KB
2014/10/09 15:05:55| Max Swap size: 102400 KB
2014/10/09 15:05:55| Local cache digest enabled; rebuild/rewrite every 3600/3600 sec
2014/10/09 15:05:55| /var/spool/squid/00: (2) No such file or directory
FATAL:  Failed to verify one of the swap directories, Check cache.log


Both access.log and store.log are empty.

As I mentioned, I'm not very experienced in sme server, so apologies for the lack of initial information.
Thanks again.

« Last Edit: October 09, 2014, 08:37:34 AM by joshAU »

guest22

Re: Squid has vanished! SME8.1.
« Reply #3 on: October 09, 2014, 08:36:20 AM »
The first suggestion in the logs is:

"Failed to verify one of the swap directories, Check cache.log"

Have you done that?

Re: Squid has vanished! SME8.1.
« Reply #4 on: October 09, 2014, 08:39:02 AM »
Geez you're fast. :)
Sorry, I just edited my above post to include the relevant info from cache.log.
(Had to find it, as I couldnt see it in the server manager logs page....

Re: Squid has vanished! SME8.1.
« Reply #5 on: October 09, 2014, 08:40:47 AM »
/var/spool/squid/00
does actually exist, and has many two character folders within it.

can I just delete the contents of:
/var/spool/squid/

and then run:

squid -z

?

I'm obviously not concerned about the contents of the cache...
« Last Edit: October 09, 2014, 08:43:32 AM by joshAU »

guest22

Re: Squid has vanished! SME8.1.
« Reply #6 on: October 09, 2014, 08:51:46 AM »
No, don't delete anything manually. You want to know the root cause, and then fix it, not the other way around.

Try this:

http://wiki.contribs.org/Useful_Commands#Refresh_Squid_Cache

Re: Squid has vanished! SME8.1.
« Reply #7 on: October 09, 2014, 08:57:45 AM »
Fair call. :)

OK.

I'll have a look at it tomorrow morning... I was meant to be out of work 1/2 an hour ago.

Thanks again for your suggestions, I'll try what you suggested and post back tommorrow.


Offline janet

  • ****
  • 4,812
Re: Squid has vanished! SME8.1.
« Reply #8 on: October 09, 2014, 09:24:40 AM »
joshAU

Quote
...I had configured squid to cache the updates a long time ago....

It might be useful to tell us what you did, as that may be related to the problem you now have.
Please search before asking, an answer may already exist.
The Search & other links to useful information are at top of Forum.

Re: Squid has vanished! SME8.1.
« Reply #9 on: October 09, 2014, 06:02:41 PM »
Sounds to me like you may have a corrupted file system, since you seem to have missing directories and symlinks.

What do you see with:

rpm -V squid
rpm -V e-smith-proxy

Re: Squid has vanished! SME8.1.
« Reply #10 on: October 10, 2014, 01:05:00 AM »
Hi again RequestedDeletion.

OK, I tried running:
sv d /service/squid

it returned unable to change to service directory: file does not exist

I tried running:
echo "" > /var/spool/squid/swap.state

which did not throw any errors, or anything for that matter.

The line:
sv u /service/squid

threw the original error.

As did:
sv s /service/squid

There is no squid folder within the /service folder. :(

@ janet: thanks for your input.
The changes I did I did a long time ago, and were fairly standard...
I followed the procedure for it on:

 http://wiki.contribs.org/Squid

ie...
Quote
Create the template fragment

mkdir -p /etc/e-smith/templates-custom/etc/squid/squid.conf
nano  -w /etc/e-smith/templates-custom/etc/squid/squid.conf/05refreshpattern

paste lines below

hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY
cache_mem 16 MB
maximum_object_size 1280096 KB
cache_dir ufs /var/spool/squid 3000 16 256

# caching windows update various windows versions
refresh_pattern http://.*\.windowsupdate\.microsoft\.com/ 0 80% 20160 reload-into-ims
refresh_pattern http://.*\.update\.microsoft\.com/ 0 80% 20160 reload-into-ims
refresh_pattern http://download\.microsoft\.com/ 0 80% 20160 reload-into-ims
refresh_pattern http://windowsupdate\.microsoft\.com/ 0 80% 20160 reload-into-ims
refresh_pattern http://office\.microsoft\.com/ 0 80% 20160 reload-into-ims

Execute:

expand-template /etc/squid/squid.conf
squid -k reconfigure

I have checked the squid logs in the past and had not noticed anything wrong.

@CharlieBrady: Thanks also for your input.
your suggestion re corrupt file system I suspect may be correct.

The first line you suggested trying:
Code: [Select]
rpm -V squid
paused for a few seconds and then returned nothing.

The second line:
Code: [Select]
rpm -V e-smith-proxy
returned "package e-smith-proxy is not installed"

Not sure where to go from from here.
Thanks again for all your input, and any advice you can provide.



Offline Stefano

  • *
  • 10,805
  • Skype account: maghissimo
    • Smeserver italian community
Re: Squid has vanished! SME8.1.
« Reply #11 on: October 10, 2014, 01:25:24 AM »
Code: [Select]
history | grep rpm
Consulente di Smeserver.it -  Soluzioni e supporto su Sme server in Italia

Re: Squid has vanished! SME8.1.
« Reply #12 on: October 10, 2014, 01:52:13 AM »
Hi Stefano, and thanks for your input too.

I ran that line:
Code: [Select]
history | grep rpm
it returned:
Quote
273  rpm -q squid
308  rpm -V squid
309  rpm squid
310  rpm -v squid
311  rpm -V squid
312  rpm -V e-smith-proxy
313  rpm -V squid
314  history | grep rpm
319  history | grep rpm

I hope that output means more to you than it does too me. :)
Thanks.

Re: Squid has vanished! SME8.1.
« Reply #13 on: October 10, 2014, 02:02:46 AM »
I hope that output means more to you than it does too me. :)

It means you bash history doesn't show the removal from the command line of any rpms. There is still a mystery as to why 'e-smith-proxy' appears not to be installed on your system. If 'e-smith-proxy' is not installed on your system, that would explain why squid is not running.

I'd suggest that you run:

yum install e-smith-proxy -y

Re: Squid has vanished! SME8.1.
« Reply #14 on: October 10, 2014, 04:38:28 AM »
OK.
I ran:
yum install e-smith-proxy -y

and it installed successfully.
I rebooted, and lost internet... hence the delay...

However, squid logs are now listed in the server manager screen, etc so it seems to have fixed some issues...
Also, the "proxy settings" tab reappeared in the server manager, so I was able to disable the proxy from there and get back on the internet....:)
albeit without a proxy...

Looking through the messages.log I see this repeatedly:
Quote
Oct 10 12:28:02 webserver squid[8148]: Starting Squid Cache version 2.6.STABLE21 for i386-redhat-linux-gnu...
Oct 10 12:28:02 webserver squid[8148]: Process ID 8148
Oct 10 12:28:02 webserver squid[8148]: With 1024 file descriptors available
Oct 10 12:28:02 webserver squid[8148]: Using epoll for the IO loop
Oct 10 12:28:02 webserver squid[8148]: DNS Socket created at 0.0.0.0, port 45476, FD 6
Oct 10 12:28:02 webserver squid[8148]: Adding domain XXXXXXXXX from /etc/resolv.conf
Oct 10 12:28:02 webserver squid[8148]: Adding nameserver 10.0.0.1 from /etc/resolv.conf
Oct 10 12:28:02 webserver squid[8148]: User-Agent logging is disabled.
Oct 10 12:28:02 webserver squid[8148]: Referer logging is disabled.
Oct 10 12:28:03 webserver squid[8148]: Unlinkd pipe opened on FD 11
Oct 10 12:28:03 webserver squid[8148]: Swap maxSize 102400 + 8192 KB, estimated 0 objects
Oct 10 12:28:03 webserver squid[8148]: Target number of buckets: 1843
Oct 10 12:28:03 webserver squid[8148]: Using 8192 Store buckets
Oct 10 12:28:03 webserver squid[8148]: Max Mem  size: 8192 KB
Oct 10 12:28:03 webserver squid[8148]: Max Swap size: 102400 KB
Oct 10 12:28:03 webserver squid[8148]: Local cache digest enabled; rebuild/rewrite every 3600/3600 sec
Oct 10 12:28:03 webserver squid[8148]: /var/spool/squid/swap.state: (13) Permission denied
Oct 10 12:28:03 webserver squid[8148]: storeUfsDirOpenSwapLog: Failed to open swap log.

Should I try to change permissions on /var/spool/squid/* ?

If so, does anyone know what the default permissions should be?

If I ls- al swap.state I get this returned:
Quote
-rw-r--r-- 1 root root 1 Oct 10 09:45 swap.state

Does that seem like the right permissions?
I'm guessing not...

Thanks again for any help you can provide.

Re: Squid has vanished! SME8.1.
« Reply #15 on: October 10, 2014, 05:15:08 AM »
if I do a "ls -al" on the /var/spool/squid folder, it returns:
Quote
drwxr-x---  18 squid squid 4096 Oct  9 17:02 .
drwxr-xr-x  14 root  root  4096 Aug 24 15:25 ..
drwxr-x--- 258 squid squid 4096 Oct  9 17:01 00
drwxr-x--- 258 squid squid 4096 Oct  9 17:01 01
drwxr-x--- 258 squid squid 4096 Oct  9 17:01 02
drwxr-x--- 258 squid squid 4096 Oct  9 17:01 03
drwxr-x--- 258 squid squid 4096 Oct  9 17:01 04
drwxr-x--- 258 squid squid 4096 Oct  9 17:01 05
drwxr-x--- 258 squid squid 4096 Oct  9 17:01 06
drwxr-x--- 258 squid squid 4096 Oct  9 17:01 07
drwxr-x--- 258 squid squid 4096 Oct  9 17:01 08
drwxr-x--- 258 squid squid 4096 Oct  9 17:01 09
drwxr-x--- 258 squid squid 4096 Oct  9 17:01 0A
drwxr-x--- 258 squid squid 4096 Oct  9 17:01 0B
drwxr-x--- 258 squid squid 4096 Oct  9 17:01 0C
drwxr-x--- 258 squid squid 4096 Oct  9 17:01 0D
drwxr-x--- 258 squid squid 4096 Oct  9 17:01 0E
drwxr-x--- 258 squid squid 4096 Oct  9 17:01 0F
-rw-r--r--   1 root  root     1 Oct 10 09:45 swap.state

Am I right in assuming that the swap.state file should have the same permissions as the rest of those files do...ie. drwxr-x--- 258 squid squid 4096?

[SOLVED] Re: Squid has vanished! SME8.1.
« Reply #16 on: October 10, 2014, 05:48:41 AM »
OK, think its been sorted...

I ended up changing the ownership to squid...
Code: [Select]
chown -R squid.squid swap.state
and that seemed to fix it.
checked logs, and squid seems happy, re-enabled the cache and internet works. tried going to the same page and saw that TCP HITS were happening.

Not sure of the cause... but happy its up and running again.

Thank you all for all your help.
I appreciate it.