Topic: Dansguardian making web page look messed up

[steve288]

I have just installed Dansguardian.
I have a simple installation of DG.
 Just set the sme server as severgateway. I have it set as the gateway of my computer.

Now when I go my mail box on yahoo the page is all screwed up. Its hard to describe but most of the text is down the left side and its missing most of its color, There are no columns sorting the mail nicely across, The info is basically there but alll the formating is removed basically.
This is server that eventually will filter web sites for users. I need to get it right so others dont have the same issue.
I asume that something is being blocked. Java?? or something. Strangely all other sites that I can see are fine. Only my email (of course something I care about.)  I went to gmail account and it seems fine.

Can anyone provide any tips on this problem.

PS. Im modifying my message as I thought I would look in the DG.logs I see this message which May be the problem?? (or not?) not sure what to do about it if it is.

2012.3.21 10:34:29 - 10.1.0.87 http://prod1.rest-notify.msg.yahoo.com/v1/pushchannel/myname@myisp.com?seq=7&cb=52i3xu7d&format=json&idle=110&imtoken=2tXoIeR3KQvqMXO2NvpKGz8wrOh_5oMwO_Sue1Kf1fMi6nTF1_GvtLfoQBXNznpr0aLJ4tw7Cixs3W8A-%7C5Lb1w7AXXgfvoi4E3fdjlg--&sid=cWDhH47s3BkmmjvmwuUxv4F2IdOGLjwEIc2Y&c=wwqu3N%2FOYWq&msgrAppId=mim&cache=1332340460275 *DENIED* Banned extension: .com GET 0 0 Banned extension 1 403 application/json   -

Regards.

[janet]

steve288

2012.3.21 10:34:29 - 10.1.0.87 http://prod1.rest-notify.msg.yahoo.com/v1/pushchannel/myname@myisp.com?seq=7&cb=52i3xu7d&format=json&idle=110&imtoken=2tXoIeR3KQvqMXO2NvpKGz8wrOh_5oMwO_Sue1Kf1fMi6nTF1_GvtLfoQBXNznpr0aLJ4tw7Cixs3W8A-%7C5Lb1w7AXXgfvoi4E3fdjlg--&sid=cWDhH47s3BkmmjvmwuUxv4F2IdOGLjwEIc2Y&c=wwqu3N%2FOYWq&msgrAppId=mim&cache=1332340460275 *DENIED* Banned extension: .com GET 0 0 Banned extension 1 403 application/json   -

Well the obvious thing to do would be to remove the ban on that extension (.com), restart DG and see if the problem still exists.

[steve288]

Thank you Mary for your suggestion.

I will provide the details of how I resolved this issue. I'm coming up to speed on this software, so like any software there  is so much to learn, and form me to do wrong. If I have made a mis step please kindly indicate it.  (like I could stop you) I hope that this information will help others not just with this error but it should help them to resolve other DENIED issues, if they extrapolate the information. (ok I just wanted to use the word extrapolate)

The important thing here is the log message with the phrase ..

*DENIED* Banned extension: .com GET 0 0 Banned extension 1 403 application/json   -

Taking some of my knowledge from Watchguard firewalls, really what needs to be allowed is the application/json type. I'm not really sure what the .com really means, happy if someone tells me. I tried to do alot of reading and really could not find much. So I just experimented. I found that if I went to the file ,
 #/etc/dansguardian/lists/exceptionmimetypelist
and then added the following mimetype to the list of allowed mime types,

application/json

The problem was prety much solved. I say prety much because the site was better but still not perfect. So I went to the trusty  /var/dangsguardian/allow.log  and saw,

*DENIED* Weighted phrase limit of 50 : 60 ((pink, lips)+(proxy, block)+(proxy, filter)+-main+-transparent+-tumor) GET 115503 60 Proxies, Pornography 1 403 text/css   -

Dansguardian uses a weighted phrase system to filter web sites. If it saw many many bad words it would block the site. In this case it saw the words pink and lips on the web site. And then blocked it, at least the way I read it. Now you have two options to resolve this. The 50 : 60 number is very significant.
What it is saying is that your default naughtynesslimit is set to 50. Any score from a site that is over your naughtynesslimit will be blocked. In this case the words pink, lips has been given a score of 60. The DG security guard jumps into way and is saying No Stop I will not allow this because your limit is 50 but this site is 60.
There are two ways to fix this.
1. Because my install is brand new install of DG my naughtynesslimit was set to 50. Thats really restrictive. (if you want that, then great but for an office setting I think its a bit high) I have set mine to 135. You do this by going to /etc/dansguardian/dansguardianf1.conf, looking for a line that says
 naughtynesslimit = ##
The ## should be a number. In my case it was naughtynesslimit = 50. I changed this to 135, then ran  /etc/init.d/dansguardian restart. This downgrades all the scores. BTW I chose 135 because when I went to the playboy site, ( I just read articles honest) I found that the score was about 143. If in my mind playboy is the mildest site for porn then I use that as my gage and set my  naughtynesslimit to 135. Blocking that site and everything worse. Thats my thinking anyway.

2. Second method. Maybe you already have your global naughtynesslimit set to a setting you like and dont want to change it. Then you can change the actual score for the bad words. In my case, and you will have to do your own research on this, I found that the reported bad words pink lips were coming from a file
 /etc/dansguardian/lists/phraselists/pornography/weighted
(just do a find for the words in the /lists sub directorys)

I found in this file the following line <pink>,<lips><50>

Just change the number 50 to something less, you can do trial and error. Keep looking at the logs to see what the score is and then lower it to below that. Of course that will affect all future scores for that phrase. But these scores are not perfect. They are writen with the best intentions. I have found in some situations, like health organizations they need to allow certain street phrases because they do research into odd things.
Of course after any changes run /etc/init.d/dansguardian restart , try your web site and if its still not right change the score again. And keep and eye on the logs as they will be your best indication of whats happening and why things are blocked.

Hope it helps.

[janet]

steve288

Well all of what you reported or advised or discovered is already in two of the Dansguardian wiki articles.
Did you read them first, you may have saved yourself some time and effort.

http://wiki.contribs.org/Dansguardian
and
http://wiki.contribs.org/Dansguardian/ConfigFiles

[steve288]

Hello Mary,

Oddly I share my experiences from the other side of the problem. Here is the problem eg DENIED web sites, here is the message, and then how to resolve them. Perhaps a few will find that direction helpfull. Real world examples. I hope people will be able to search on my reports or errors and then it will help them identify the various configuaration settings that need change. I hope it gives them yet another resource to give them the answers when frustrated. I don't think some of the  specifices I mention are in the wiki. I was trying to show my self discovery. It's the way I learn. I'm certainly glad for those who learn differently.

When one approaches new software it's hard sometimes to understand what is ment in the wiki's. It's also hard to absorb all the nuances of the software and settings that perhaps later in hind site (at least for me) I say "OH of course thats what they mean."  I'm just trying to do the best I can. However I'm somtimes frustrated in the instructions that are provided. That they are not clear to me (maybe I'm the only one). I will of course admit I like things a little pedantic. But when it's new software to me, thats really what I need. For others who are smarter than me, it's hard to see it again like a novice, because now you carry intelectual understanding, that can't be undone. This is really why I try in my EXTREMELY small way to be on the "Wiki docs" team to be a part of the solution instead me just complaining. This is NOT to say that I'm not thankfull for those who have layed down what has already been written in their own way. But if it was perfectly writen down then there would be no need for the concept of the Wiki.

So I’m going have to apologize a head of time, I’m probably going to have to ask more dumb questions.

By the way Mary thank you for putting the links in your message above, they are good links with of course lots of helpfull info.

As always
Regards.

[janet]

steve288

I understand what you are saying, every "new" piece of software has a "learning curve" before one is comfortably competent with it, and instructional syntax can often be difficult to initially grasp.

I don't think some of the  specifics I mention are in the wiki.

If you feel that is the case, then please add to the existing DG wiki articles, as it will help future users more readily understand the concepts.
It could be added to the particular section, or perhaps in a "Troubleshooting" section at the end of the article.
I am very partial to "real world examples", I feel they they should be obligatory in every technical article or set of instructions.

This is really why I try in my EXTREMELY small way to be on the "Wiki docs" team to be a part of the solution instead me just complaining. This is NOT to say that I'm not thankfull for those who have layed down what has already been written in their own way. But if it was perfectly writen down then there would be no need for the concept of the Wiki.

That's why the wiki is there, for all to edit and improve upon, under peer review & interaction to prevent major blunders.
The DG wiki articles SHOULD be sufficiently well written for a novice user to be able to follow and setup DG, and move on to more advanced configuration after having mastered the initial setup of DG.

Please add where you think there are improvements needed.

[steve288]

Thanks Mary,

Sometimes I will put my thoughts in the Newsgroup before putting into the Wiki. That way if Im making a mistake in my assumptions then its not in a wiki which should be accurate. The Newsgroups need to be read carefully, as sometimes mistakes are written, the Wiki I try to be very cautious as it needs to be accurate. I think I have a guilt complex because every time I make a change to a Wiki Im frightened that someone is going to say to me “You numbskull that’s completely wrong and you should be deleted from the board for leading people astray.” Boy I must have been dropped on my head as a child.

Yes Troubleshooting would be a good topic.  And when you say

"I am very partial to "real world examples", I feel they they should be obligatory in every technical article or set of instructions."

Yes yes yes. Nicely put. I will end this discussion on a humorous note. When I worked for one company that developed hardware and software, the head hardware engineer had a sign in his office.
“If you make it so idiots can use it, they probably will.” Knowing electrical engineers as I do, I think he interpreted this to mean, don’t make it so easy so Idiots can use it. Being a bit of an idiot myself I take it to mean, make it easy enough for me to use. (and um sorry to all the electrical engineers out there.
Regards.