October 14, 2009 at 5:28 pm #41975
OK…I am new to Zeroshell and it looks wonderful so far. My question is this:
I read about bonding multiple lan to lan vpns to aggregate the bandwidth…and in one area it talks about the balancer and round robin connections being used, and in another area…it talks about true bandwidth aggregation…see here:
“The BOND00 interface created is equivalent to an Ethernet interface: it may contain IP addresses, add VLAN 802.1q, or be assigned to a bridge. As mentioned at the beginning, since load balancing in bonding takes places in Ethernet frames, even a single TCP/IP connection will enjoy an increased band thanks to the presence of multiple links.”
So …here’s what I am asking:
If I have 2 locations…each with (2) 2mb up and down cable modem conections…can I put a zeroshell on each side, create 2 lan to lan vpns, bond them, and get 4 mb of throughput from site a to site b for a single tcpip connection? I am attempting to put san replication traffic across this and am checking if this will truly combine the bandwidth or if the first call would use the first vpn, and the second call would use the second vpn in a round robin format.
The docs seem to contradict themselves a bit.
All help is appreciated from this Zeroshell Noob
Tom POctober 15, 2009 at 1:47 pm #48897
A VPN BOND will have an aggregated bandwidth.
Any other interface traffic will be load balanced per connection with weighted round robin as documented.October 15, 2009 at 2:24 pm #48898
OK…so as I read this…
we bond the 2 vpn tunnels together…thus creating the Bond00 interface..done on each zeroshell device (Site A and Site B)
then I have to bridge eth00 and the bond interfaces (on both sides I assume) thus creating the bridge00 interface
the 2 bridge interfaces of the Zeroshell devices should be on the same subnet as my 2 san devices (example. San 1 10.1.1.1/24, San 2 10.1.1.2/24, Bridge00 of site A 10.1.1.3/24, Bridge00 of site B 10.1.1.4/24)…even though they are physically across these VPN connections.
Will there be anything I need to do to make sure the remote SAN and remote Bridge00 are learned on the Site A side?
Thanks for your insight here, trying to wrap my head around exactly how this will work.
TPOctober 15, 2009 at 3:20 pm #48899
It should work the way you describe it, but I would not bridge the BOND00 and ETH00, instead I would route between them.October 15, 2009 at 7:06 pm #48900
OK…so if we route between the sites…I can leave Branch A at the 10.1.1.0/24 network and Branch B at the 10.1.2.0/24 network…I don’t need to bridge and make them all the same subnet?
That would be even better if thats the case!!October 16, 2009 at 5:12 pm #48901
Yup that’s the case exactly. You’ll need one more subnet for the tunnel, let’s say 192.168.0.0/30October 21, 2009 at 8:18 pm #48902
Well…thanks to ppalias…I have sucessfully configured the ZS devices, bonded and got it all running.
Here’s the interesting part:
We have setup 4 routers, the first pair with back to back T1 (1.5mps up and down)…and addressed accordingly.
The second pair…same thing…different addressing schemes.
The zero shell boxes (each with 2 wan connections) are on either side of these routers….so effectively, the routers are simulating 2 internet connections.
If I take the ZS out, and copy 100 MB of small files from 1 side to the other, over 1 set of routers…I get around 1.4 mb constant throughput till done. When I put the ZS in, setup a vpn between the 2 ZS boxes, it transfers, but I see only 800K of usage on the routers.
I setup another VPN over the 2nd set of routers,
When I bond the two together…I get about 400-500K of usage over each connection. Overall Better…but nowhere near what the routers themselves can do.
What am I missing here?
TPOctober 22, 2009 at 7:07 am #48903
Could you check if there is a CPU utilization bottleneck issue?October 22, 2009 at 7:14 pm #48904
The processors are asleep.
5% on the “server” side (ZS box A) of the vpns, 1% on the client side (ZS Box B).
Memory usage at 140 MB of 2 GB.
TPOctober 23, 2009 at 4:19 pm #48905
Connect two ZS routers over one connection only over vpn.
Measure the speed.
This should show us if there is a problem with vpn overall or with bonding.October 24, 2009 at 1:02 am #48906
After some massive testing…here’s what we found.
The big issue is QOS on the bond. If we turn on the default and tell it it has 3mb guaranteed…then it used all of the lines (except in bonded TCP vpns…see below)
Baseline : No Zeroshell…Transferring 100 MB over the routers back to back , single T1, 11 minutes
Single VPN – 100 mb, TCP, no encryption, no compression – 14min 30 sec
using the 1.2 to 1.3 connection speed.
Dual VPN TCP bonded – 12 min ( again…this is the wierd not full bandwidth issue…even with QOS)
Once we switched to UDP, on a single vpn we got 12 minutes (expected for a little overhead)
Dual UDP bonded VPN we got 7 minutes (and the t1s were both utilizing 1.2 – 1.3 MB each )
We even tested 2 vpns bonded, and pulled one connection mid transfer, then put it back in…and we watched ZS dynamically adjust to the speed differences nicely.
with UDP, we could even turn on encrytion (different ciphers tried up to 256 AES) and we saw little or no difference – maybe extra 30 seconds to transfer on 256 AES
So in controlled testing…the TCP VPN connection really pales compared to the UDP….obviously the connection based TCP protocol adds some overhead on the tunnel…like 30%.
The other thing we did see is this…
We decided to see what would happen if one of the uploads was not capable of the same speed as the other. So we took one of the simulated T1 connections and reduced it to 756K. What we saw was the 2 udp vpn tunnels both used 700K over each line…even though the other line could have done more. Is this by design or a flaw?
SO….if we have tested correctly…we have found that the UDP vpn bond is superior to the TCP bond by over 30%…and we found the a bonded VPN will throttle all connections down to the lowest speed…so if you have 2mb connection, a 1 mb cnnection, and a 256K connection bonded…you will get 3 connections running at the lowest speed of 256K EACH.
AND QOS on the bond is essential to get the bond to utilize the available bandwidth.
So moral of the story…use UDP, QOS, no compression, and make sure the speeds match.
BTW…the graphs all said that /mrtg/statisitcs.html was not found on this server….ideas?
Overall…this is a fantastic product and very resiliant. We are extremely hapy with it once we leaarned the quirks.
I may have been too verbose in these forums…but hopefully all this will help the next guy who reads this.
So..ppalias….what do you think about the 2 issues above?
1. 2 different speed links will link and use only the bandwidth of the lowest link
2. Missing mrtg graphics
TPOctober 24, 2009 at 2:37 pm #48907
1) It sounds reasonable as the round robin algorithm sends one packet on one connection and the next over the other. So when the low speed connection is fed up the second will not be utilized too. If there was an option to assign weights on the connections it would solve this issue.
2) Don’t know , try to reinstall mrtg.October 26, 2009 at 8:51 pm #48908
So we are ok with #1….but #2….reinstall MRTG…..we’re running this off the ISO burned to a cd….the mrtg isn’t part of it by default?October 27, 2009 at 3:29 pm #48909
IF you have beta12 version, it must be preinstalled. In beta11 it is applied as a patch.
Try to run this command in post boot.
service mrtg startNovember 10, 2009 at 6:29 pm #48910
I am trying to make a link aggregation (layer 2) with zeroshell without success.
The problem is that only ONE gateway is used at time (seem like connection based balancing) NO BALANCING occurs. I have followed the guide exactly.
Seem you have found the way. May you show me your configuration and a map of your hardware setup?
Thanks a lot in advance!
You must be logged in to reply to this topic.