Koozali.org: home of the SME Server

testers wanted: htbwondershaper

Offline hanscees

  • *
  • 267
  • +0/-0
    • nl.linkedin.com/in/hanscees/
testers wanted: htbwondershaper
« on: November 11, 2005, 10:29:09 PM »
Hi,
I announced a wondershaper script for sme7 here before.

I have tested it and it works for me.
I would like to ask people with adsl/ broadband to test the script.
This wondershaper script does traffic-shaping, and so it should enhance your latency for instance.


I would like to know:
- Do you understand the script enough to use it?
- Does it work for your situation
- Can you come up with ideas to enhance it?

If you already use a wondershaper script, would you be so kind to compare it with this one? Do you see
differences in latency for instance?

Please also measure latency (use ping and surf)  while you are downloading/uploading a lot etc.

The script lives here:
http://www.hanscees.com/sme7/HTBWondershaper
The page describing it is here:
http://www.hanscees.com/sme7/wondershaperbeefedup.html

The script is relatively easily extendable to do traffic shaping like:
- limit downloads from some internal ip-address/ tcp-port etc.

Hope to have many testers! If all goes well it might grow into a regular sme package.

Hans-Cees
nl.linkedin.com/in/hanscees/

Offline psoren

  • *
  • 371
  • +0/-0
Re: testers wanted: htbwondershaper
« Reply #1 on: November 13, 2005, 01:03:11 AM »
Quote from: "hanscees"
Hi,
I announced a wondershaper script for sme7 here before.

I have tested it and it works for me.
I would like to ask people with adsl/ broadband to test the script.
This wondershaper script does traffic-shaping, and so it should enhance your latency for instance.


I would like to know:
- Do you understand the script enough to use it?
- Does it work for your situation
- Can you come up with ideas to enhance it?

If you already use a wondershaper script, would you be so kind to compare it with this one? Do you see
differences in latency for instance?

Please also measure latency (use ping and surf)  while you are downloading/uploading a lot etc.

The script lives here:
http://www.hanscees.com/sme7/HTBWondershaper
The page describing it is here:
http://www.hanscees.com/sme7/wondershaperbeefedup.html

The script is relatively easily extendable to do traffic shaping like:
- limit downloads from some internal ip-address/ tcp-port etc.

Hope to have many testers! If all goes well it might grow into a regular sme package.

Hans-Cees


Hi, i would love to test it... But i don't even know where to put it :hammer:
The reason for me to use it would be to priorytize VoIP traffic. I have looked a little on the script but i don't fully understand yet. I have read that there has to be some things enabled in the kernel for HTB, is that already the case? I'm sure a lot off people want's to get VoIP to work better with this script, so maybe we could get a few pointers how to do that.

Per

Offline hanscees

  • *
  • 267
  • +0/-0
    • nl.linkedin.com/in/hanscees/
Re: testers wanted: htbwondershaper
« Reply #2 on: November 13, 2005, 07:21:42 PM »
see below, this went wrong..
nl.linkedin.com/in/hanscees/

Offline hanscees

  • *
  • 267
  • +0/-0
    • nl.linkedin.com/in/hanscees/
Re: testers wanted: htbwondershaper
« Reply #3 on: November 13, 2005, 07:24:48 PM »
Quote from: "hanscees"
Quote from: "psoren"

Hi, i would love to test it... But i don't even know where to put it
The reason for me to use it would be to priorytize VoIP traffic. I have looked a little on the script but i don't fully understand yet. I have read that there has to be some things enabled in the kernel for HTB, is that already the case? I'm sure a lot off people want's to get VoIP to work better with this script, so maybe we could get a few pointers how to do that.

Per

Hi,
-HTB is enabled in sme7. I do not know about sme6.
Have a look with lsmod and see if:


sch_sfq          
sch_htb          

are there. The htb thing is htb.

What traffic do you have that is voip? You will need to mark it with iptables. To be able to do that you will need to know what tcp/udp-ports it uses and what flows take place from internal to internet and back.
So question is:
- what kind of traffic are you aiming at exactly with voip here?

If you can answer that I can see if that is possible.


You can save the script and just run it as a script. You should firstly adjust the liner-speed to your speed, so adjust "CEIL" "DOWNLINK" and "EXTDEV" and "LANDEV" if needed.

Does this answer help? Or are still other things unclear?

Hans-Cees
nl.linkedin.com/in/hanscees/

Offline psoren

  • *
  • 371
  • +0/-0
Re: testers wanted: htbwondershaper
« Reply #4 on: November 18, 2005, 10:53:56 AM »
Quote from: "hanscees"
Quote from: "hanscees"
Quote from: "psoren"

Hi, i would love to test it... But i don't even know where to put it
The reason for me to use it would be to priorytize VoIP traffic. I have looked a little on the script but i don't fully understand yet. I have read that there has to be some things enabled in the kernel for HTB, is that already the case? I'm sure a lot off people want's to get VoIP to work better with this script, so maybe we could get a few pointers how to do that.

Per

Hi,
-HTB is enabled in sme7. I do not know about sme6.
Have a look with lsmod and see if:


sch_sfq          
sch_htb          

are there. The htb thing is htb.

What traffic do you have that is voip? You will need to mark it with iptables. To be able to do that you will need to know what tcp/udp-ports it uses and what flows take place from internal to internet and back.
So question is:
- what kind of traffic are you aiming at exactly with voip here?

If you can answer that I can see if that is possible.


You can save the script and just run it as a script. You should firstly adjust the liner-speed to your speed, so adjust "CEIL" "DOWNLINK" and "EXTDEV" and "LANDEV" if needed.

Does this answer help? Or are still other things unclear?

Hans-Cees


Hans-Cees,

I am running SME 6.
I can do "modprobe sch_htb" and i can se it with lsmod, but sch_sfq says "can't locate module".

I need ports UDP 10000-20000 (incoming) for the voice traffic and UDP 5060 to connect to the VoIP provider (don't think that would need priority) but i don't know what ports is used outgoing. Is there a way to see that?

Where do i physically locate the script on the server and how do i actually activate it?

I hope you can help a little, i think this is interesting for all the people using VoIP and all the people implementing asterisk on the SME.

Cheers
Per

Offline hanscees

  • *
  • 267
  • +0/-0
    • nl.linkedin.com/in/hanscees/
Re: testers wanted: htbwondershaper
« Reply #5 on: November 18, 2005, 10:03:22 PM »
Quote from: "psoren"

Hans-Cees,

I am running SME 6.
I can do "modprobe sch_htb" and i can se it with lsmod, but sch_sfq says "can't locate module".

I need ports UDP 10000-20000 (incoming) for the voice traffic and UDP 5060 to connect to the VoIP provider (don't think that would need priority) but i don't know what ports is used outgoing. Is there a way to see that?

Where do i physically locate the script on the server and how do i actually activate it?

I hope you can help a little, i think this is interesting for all the people using VoIP and all the people implementing asterisk on the SME.

Cheers
Per


Question one is howto run it, that is relatively easy:

Ok, I think you only need sch_htb, so that would be fine.

You can runs the script from anywhere you like. So for instance do:

go to your sme server with ssh.
mkdir /root/scripts
cd /root/scripts
wget http://web.inter.nl.net/users/hanscees/sme7/HTBWondershaper

chmod +x HTBWondershaper

and run it like this:
/root/scripts/HTBWondershaper

Than look with
/root/scripts/HTBWondershaper status

if you see all kinds of lines with information about htb and iptables chains. So far for question one.

Question two is about traffic shaping and what you need. This is more difficult.

We need to determine what traffic you need to prioritise exactly. That question is not simple I am afraid.

There is a chance the script will improve your traffic
as long as you set the CEIL and other variables correctly. By default it can improve your latency. It also improves the flow for traffic where the TOS bits are set. If for instance asteriks sets those tos bits, the script might do good as it is by default.

If that is not the case we will have to look more into the traffic your voip will generate.

Is it correct that you want to look at using this script with an asteriks on your sme?

Hans-Cees
nl.linkedin.com/in/hanscees/

Offline psoren

  • *
  • 371
  • +0/-0
Re: testers wanted: htbwondershaper
« Reply #6 on: November 19, 2005, 11:40:00 PM »
Quote from: "hanscees"

Question one is howto run it, that is relatively easy:

Ok, I think you only need sch_htb, so that would be fine.

There is a chance the script will improve your traffic
as long as you set the CEIL and other variables correctly. By default it can improve your latency. It also improves the flow for traffic where the TOS bits are set. If for instance asteriks sets those tos bits, the script might do good as it is by default.

If that is not the case we will have to look more into the traffic your voip will generate.

Is it correct that you want to look at using this script with an asteriks on your sme?

Hans-Cees


Hans-Cees,

OK, i got the script running but it did give me a few errors:

Unknown qdisc "htb", hence option "default" is unparsable
+ tc class add dev eth1 parent 1: classid 1:1 htb rate 384kbit ceil 384kbit
Error: Qdisc "htb" is classless.
+ tc class add dev eth1 parent 1:1 classid 1:10 htb rate 100kbit ceil 200kbit prio 0
Error: Qdisc "htb" is classless.
+ tc class add dev eth1 parent 1:1 classid 1:11 htb rate 200kbit ceil 384kbit prio 1
Error: Qdisc "htb" is classless.
+ tc class add dev eth1 parent 1:1 classid 1:12 htb rate 50kbit ceil 384kbit prio 2
Error: Qdisc "htb" is classless.
+ tc class add dev eth1 parent 1:1 classid 1:13 htb rate 100kbit ceil 307kbit prio 3
Error: Qdisc "htb" is classless.
+ tc qdisc add dev eth1 parent 1:10 handle 100: sfq perturb 10

Is that bad?

I use VoIP in tree different ways:
 1. My Sipura 2002 adapter set up to my VoIP provider. Simple but no fun... Is what i use at the moment.

 2. I have an older server running asterisk@home and my sipura set up with local extensions. Good fun but not without problems.

3. A basic asterisk installation on the SME server. Works well.

If SME 7 and asterisk@home gets well integrated, then i will go for that in the long run. But ideally, the traffic shaping should wor regardless if the VoIP is happening on the SME serv or through it.

I will try running the script now for a while and see (hear  :-D ) how it works.

Thanks
Per

Offline hanscees

  • *
  • 267
  • +0/-0
    • nl.linkedin.com/in/hanscees/
Re: testers wanted: htbwondershaper
« Reply #7 on: November 21, 2005, 07:12:41 PM »
Quote from: "psoren"


OK, i got the script running but it did give me a few errors:

Unknown qdisc "htb", hence option "default" is unparsable
+ tc class add dev eth1 parent 1: classid 1:1 htb rate 384kbit ceil 384kbit
Error: Qdisc "htb" is classless.
+ tc class add dev eth1 parent 1:1 classid 1:10 htb rate 100kbit ceil 200kbit prio 0
Error: Qdisc "htb" is classless.
+ tc class add dev eth1 parent 1:1 classid 1:11 htb rate 200kbit ceil 384kbit prio 1
Error: Qdisc "htb" is classless.
+ tc class add dev eth1 parent 1:1 classid 1:12 htb rate 50kbit ceil 384kbit prio 2
Error: Qdisc "htb" is classless.
+ tc class add dev eth1 parent 1:1 classid 1:13 htb rate 100kbit ceil 307kbit prio 3
Error: Qdisc "htb" is classless.
+ tc qdisc add dev eth1 parent 1:10 handle 100: sfq perturb 10

Is that bad?

I will try running the script now for a while and see (hear  :-D ) how it works.

Thanks
Per


I am afraid the script will not work at all with those errors. Sorry, I can't help you.

Hans-Cees
nl.linkedin.com/in/hanscees/

Offline psoren

  • *
  • 371
  • +0/-0
Re: testers wanted: htbwondershaper
« Reply #8 on: November 21, 2005, 07:24:50 PM »
Quote from: "hanscees"

I am afraid the script will not work at all with those errors. Sorry, I can't help you.

Hans-Cees


Hmmm.. I was afraid so... But your help so far is very apreciated, thanks.
Guess i have to go for 7 soon....

Per

BeatJunkie

Re: testers wanted: htbwondershaper
« Reply #9 on: November 24, 2005, 06:35:18 PM »
Quote from: "psoren"
OK, i got the script running but it did give me a few errors:

Unknown qdisc "htb", hence option "default" is unparsable
+ tc class add dev eth1 parent 1: classid 1:1 htb rate 384kbit ceil 384kbit
Error: Qdisc "htb" is classless.
+ tc class add dev eth1 parent 1:1 classid 1:10 htb rate 100kbit ceil 200kbit prio 0
Error: Qdisc "htb" is classless.
+ tc class add dev eth1 parent 1:1 classid 1:11 htb rate 200kbit ceil 384kbit prio 1
Error: Qdisc "htb" is classless.
+ tc class add dev eth1 parent 1:1 classid 1:12 htb rate 50kbit ceil 384kbit prio 2
Error: Qdisc "htb" is classless.
+ tc class add dev eth1 parent 1:1 classid 1:13 htb rate 100kbit ceil 307kbit prio 3
Error: Qdisc "htb" is classless.
+ tc qdisc add dev eth1 parent 1:10 handle 100: sfq perturb 10

Is that bad?

I had the same problem running this on SME 6 too.  I found a post on the LARTC mailing list archive, where someone downloaded a tarball for HTB3 and used the version of tc contained in it.  I suppose the version of tc included with SME 6 is broken, or is the wrong version for use with HTB.

The post that I read is here: http://mailman.ds9a.nl/pipermail/lartc/2004q4/013799.html

The HTB3 tarball is available here: http://luxik.cdi.cz/~devik/qos/htb/v3/htb3.6-020525.tgz

Extract this tarball to a directory of your choice, then copy the file "tc" into /sbin to replace the original one.  The HTBWonderShaper script should work after that.

Cheers!

BeatJunkie

Re: testers wanted: htbwondershaper
« Reply #10 on: November 24, 2005, 07:06:03 PM »
Quote from: "hanscees"
I would like to know:
- Do you understand the script enough to use it?
- Does it work for your situation
- Can you come up with ideas to enhance it?

The script lives here:
http://www.hanscees.com/sme7/HTBWondershaper
The page describing it is here:
http://www.hanscees.com/sme7/wondershaperbeefedup.html

Hi Hans,

Bug report!  :)

I noticed that in many of your tc and iptables lines, there are direct references to eth1 where $EXTDEV should be used instead.

It occurs mostly in your examples, but it also occurs in some of your live tc commands.  Your script will work fine as-is for firewalls using eth1 as the external interface, but not for firewalls using another device as its external interface (in my case, ppp0 for my PPPoE DSL connection).

The following lines:
Code: [Select]
tc qdisc add dev eth1 root handle 1: htb default 13
tc class add dev eth1 parent 1: classid 1:1 htb rate ${CEIL}kbit ceil ${CEIL}kbit
should read:
Code: [Select]
tc qdisc add dev $EXTDEV root handle 1: htb default 13
tc class add dev $EXTDEV parent 1: classid 1:1 htb rate ${CEIL}kbit ceil ${CEIL}kbit

And these lines:
Code: [Select]
tc filter add dev eth1 parent 1:0 protocol ip prio 1 handle 1 fw classid 1:10
tc filter add dev eth1 parent 1:0 protocol ip prio 2 handle 2 fw classid 1:11
tc filter add dev eth1 parent 1:0 protocol ip prio 3 handle 3 fw classid 1:12
tc filter add dev eth1 parent 1:0 protocol ip prio 4 handle 4 fw classid 1:13
Should read:
Code: [Select]
tc filter add dev $EXTDEV parent 1:0 protocol ip prio 1 handle 1 fw classid 1:10
tc filter add dev $EXTDEV parent 1:0 protocol ip prio 2 handle 2 fw classid 1:11
tc filter add dev $EXTDEV parent 1:0 protocol ip prio 3 handle 3 fw classid 1:12
tc filter add dev $EXTDEV parent 1:0 protocol ip prio 4 handle 4 fw classid 1:13

I modified those lines on my copy, and it appears to load correctly.  I'll have to do some further tweakage and testing.

Cheers!

Offline psoren

  • *
  • 371
  • +0/-0
Re: testers wanted: htbwondershaper
« Reply #11 on: November 24, 2005, 07:31:26 PM »
Quote from: "BeatJunkie"
I had the same problem running this on SME 6 too.  I found a post on the LARTC mailing list archive, where someone downloaded a tarball for HTB3 and used the version of tc contained in it.  I suppose the version of tc included with SME 6 is broken, or is the wrong version for use with HTB.

The post that I read is here: http://mailman.ds9a.nl/pipermail/lartc/2004q4/013799.html

The HTB3 tarball is available here: http://luxik.cdi.cz/~devik/qos/htb/v3/htb3.6-020525.tgz

Extract this tarball to a directory of your choice, then copy the file "tc" into /sbin to replace the original one.  The HTBWonderShaper script should work after that.

Cheers!


OK, that fixed the errors. Thanks BeatJunkie.
It will be interesting to see over time how it works for VoIP.

Per

Offline hanscees

  • *
  • 267
  • +0/-0
    • nl.linkedin.com/in/hanscees/
testers wanted: htbwondershaper
« Reply #12 on: November 24, 2005, 10:38:10 PM »
Hi,
thanks for the bugreport. I have changed the online version to 1.2.1 which has $EXTDEV and $LANDEV everywhere.

By the way in my case I notice that you need to have your bandtwidh free for 70% to be able to really do effective traffic shaping. I have an 1mb upload line adsl. When I set the ceil to 700 trafficshaping really works well for upload traffic. When you ago above 70% theory says shaping will not work, since ip/tcp cannot do its flow-controle effectively since packets are dropped that need to do the flow-control. In practice that is true also.

hope to hear some testing from you all.

Hans-Cees
nl.linkedin.com/in/hanscees/

Offline psoren

  • *
  • 371
  • +0/-0
testers wanted: htbwondershaper
« Reply #13 on: November 28, 2005, 08:50:05 PM »
Hanscees,

I have done a few VoIP calls since doing the fix BeatJunkie suggested and i have done simultaneously downloads, and it seems that the script actually helps as there was no studdering in the sound. I can't do more testing since i am away from home until Xmas but i will when i get back home. I had the upload/download limits set to 75%, no other changes to the script.

Thanks!

Per

Offline hanscees

  • *
  • 267
  • +0/-0
    • nl.linkedin.com/in/hanscees/
testers wanted: htbwondershaper
« Reply #14 on: November 28, 2005, 09:30:45 PM »
Quote from: "psoren"
Hanscees,

I have done a few VoIP calls since doing the fix BeatJunkie suggested and i have done simultaneously downloads, and it seems that the script actually helps as there was no studdering in the sound. I can't do more testing since i am away from home until Xmas but i will when i get back home. I had the upload/download limits set to 75%, no other changes to the script.

Thanks!

Per


It makes sense since by default the script will prioritise outgoing traffic with TOS minimize-delay set.

But the question relevant here is if your voip did studder without the script?

Hans-Cees
nl.linkedin.com/in/hanscees/