QOS IP addresses

This topic contains 12 replies, has 0 voices, and was last updated by  almoffit 8 years, 10 months ago.

Viewing 14 posts - 1 through 14 (of 14 total)
  • Author
    Posts
  • #42782

    almoffit
    Member

    I have created a bridged network and am attempting to qos a few networks. These are all associated with google/youtube

    74.125.0.0/16
    74.14.0.0/16
    173.194.0.0/16
    209.85.0.0/16
    74.119.0.0/16

    For testing purposes I have created a class with a max bandwidth of 1k (So I’ll know when it’s working for sure)

    I’m not sure how to use the classifier. Do I apply the qos rules to the bridge?

    Also, do I place the networks that I am limiting in the source or destination IP field?

    The qos stastics keep showing that all the traffic is going through the default class. From time to time it will randomly show a blip going through my test class, but definitely not the video.

    Thanks in advance!

    #51438

    AtroposX
    Member

    In the Qos classifier, use the subnets as the src, and point it to the test class, this will limit the upload from lan clients to the sites.

    Reverse it for the subnets to be the destination, and point it to the same class, to limit the download speed.

    And don’t forget to save the changes on the classifier at the top.

    And, don’t forget to add the class to the correct interface as well on the main qos/interface manager section. The download interface will be your lan side, and the upload interface will be the internet/wan side, and save the changes at the top as well.

    #51439

    almoffit
    Member

    Below are 2 screenshots, the first one is of the classifier and how I have it set up currently. The second is of the qos interfaces. I also have applied the classes to the bridge they are asssosiated with.

    The current setup doesn’t limit youtube at all. 🙁

    #51440

    AtroposX
    Member

    You can try a server that you know is consistent such as a webserver to test on. Since Youtube goes through Akamai and Limelight CDNs, it’s be hard to match all subnets that they would be on, since they could have more than your stated five.

    Or, if you can, get a ip of a machine that is on your lan side, make sure it is not doing anything, check the firewall section, then connection Tracking section, put in their ip. Make sure it’s not doing much. Then load a youtube video, and see what connections are going through, get the ips of the source on the left, and go to http://www.arin.net, and put in the ip address, see if it goes to Akamai/Limelight, get the subnet, and make a classifier for that subnet.

    Also, restricting speed to such a CDN subnet could result in slow downloads/uploads for other things as well, windows updates for example, or anything else that would reside on those subnets, anything legitimate to use. Limiting youtube, vimeo, google-video, etc., is hard to do by ip, it’s a pain. Try a known static webserver first to download from, to make sure the classes/classifiers are working correctly. And, if so, then more than likely the youtube stream is being fed by a different CDn that is not on your subnet list.

    #51441

    AtroposX
    Member

    You could alternatively try the “flash” L7 pattern with your lan subnet as the src, and dst, to limit speed to flash-based patterns. Usually it doesn’t work, but i have had a few occurrences were it did. A while back it did, but don’t think it will match youtube/vimeo or any others, if at all, anymore. Worth a shot though.

    #51442

    AtroposX
    Member

    Also, your stated subnets in the first post don’t contain any Akamai/LLNW servers, they are just Google and some EV1Servers. You’d for sure need to use the connection tracking in the firewall section, load some videos, and track the ips, and convert to a subnet for sure. That is more than likely why your rules aren’t working.

    #51443

    almoffit
    Member

    @atroposx wrote:

    You can try a server that you know is consistent such as a webserver to test on. Since Youtube goes through Akamai and Limelight CDNs, it’s be hard to match all subnets that they would be on, since they could have more than your stated five.

    Or, if you can, get a ip of a machine that is on your lan side, make sure it is not doing anything, check the firewall section, then connection Tracking section, put in their ip. Make sure it’s not doing much. Then load a youtube video, and see what connections are going through, get the ips of the source on the left, and go to http://www.arin.net, and put in the ip address, see if it goes to Akamai/Limelight, get the subnet, and make a classifier for that subnet.

    Also, restricting speed to such a CDN subnet could result in slow downloads/uploads for other things as well, windows updates for example, or anything else that would reside on those subnets, anything legitimate to use. Limiting youtube, vimeo, google-video, etc., is hard to do by ip, it’s a pain. Try a known static webserver first to download from, to make sure the classes/classifiers are working correctly. And, if so, then more than likely the youtube stream is being fed by a different CDn that is not on your subnet list.

    I thought about the implications of blocking such a massive amount of addresses. I am putting this in for a school. I will have all of our servers bypassed from the qos all together, along with the administration.

    I also figured that *most* of the sites students go to for educational purposes probably wouldn’t be on those subnets.

    I figured out why the classifier wasn’t working.
    The proxy was proxying the bridge and didn’t play well with the classifier for some reason. I have yet to figure out why it was doing that.

    Is there a way to create a L7 rule for FLV formats? I had tried to do an iptables variable that had -s youtube.com and another for -d youtube.com
    but that seemed to resolve it one time only.

    if I could write an L7 rule for FLV and MP4 I probably wouldn’t have to worry so much about addresses but as of now I have absolutely no idea how to do that.

    #51444

    AtroposX
    Member

    Yah L7 would be prefered, especially OpenDPI from http://www.opendpi.org, by iPoque. They have DPI appliances that use DPI to traffic shape, and have released by open source their DPI engine.

    It has flash, youtube, and bunch of other L7 patterns based on pcap files rather than headers that L7 uses for their open source engine. A netfilter wrapper has been created for it. I’m trying to figure out how to install it into Zeroshell for better shaping.

    I am working on L7 filter patterns for flv and custom ones as well too.

    #51445

    AtroposX
    Member

    Perhaps this is of help…

    http://stackoverflow.com/questions/638288/regular-expression-to-extract-flvexample-flv-465-301-from-a-string

    http://l7-filter.sourceforge.net/Pattern-HOWTO

    That is a start for custom regex patterns.

    Please share if you have any success! 😀

    #51446

    AtroposX
    Member

    I just did various websites, google, youtube, vimeo, various sites with embedded videos, with just a drop of the built-in flash pattern, and get a 50/50 chance of the video loading, so the flash pattern does “work”, just not 100%, but worth a shot in using, to at least get some shaping, better than by subnet for sure.

    #51447

    almoffit
    Member

    I did a packet capture and just watched an entire video.

    The specific information about the FLV isn’t stated until the last packet prior to that it’s all gibberish(to me).

    I can see how it would be having problems finding it.

    I’ve been told that there are specific headers that deal with video types. Unfortunately I’m completely ignorant to hex. But if we can find out what that is I bet we could have better results limiting videos.

    Edit: http://osflash.org/flv
    The flv header offset field
    “Total size of header (always 9 for known FLV files) “

    I’m not sure. I’m just updating as I find stuff.

    edit edit:
    HTTP/1.1 200 OK

    Last-Modified: Sat, 19 Jun 2010 13:52:00 GMT

    Content-Type: video/x-flv

    Date: Tue, 14 Dec 2010 20:01:17 GMT

    Expires: Tue, 14 Dec 2010 20:01:17 GMT

    Cache-Control: private, max-age=24823

    Accept-Ranges: bytes

    Content-Length: 1541317

    Connection: close

    X-Content-Type-Options: nosniff

    Server: gvs 1.0


    FLV………….K………
    onMetaData…….duration.@S…+….starttime……….
    totalduration.@S…+….width.@t……..height.@n…….
    videodatarate.@Y4…..2..z..
    bytelength.A7…….
    canseekontime…
    sourcedata.. BADC208C6HH1292356877961411…….purl…………………………………………………………………………………………………………………….pmsg…………………………………………………………………………………………………………………….httphostheader…v6.lscache7.c.youtube.com

    Edit edit edit edit: I have written some l7 filters, but they vanish when I reboot. I even ran make install. How do I make them permanent?

    #51448

    AtroposX
    Member

    I believe the patterns should go in /Database/var/register/system/net/L7/l7-protocols/(your directory here). There’s extra, malware, testing, etc… in here, pick the appropriate one. The /Database is important in the beginning, as the /Database directory keeps everything on a reboot. If put in just /var/… it will be gone on reboot.

    run “service l7 restart” to restart the service and to be accessible in the gui

    #51449

    AtroposX
    Member

    Perhaps this might work, still testing though…

    “^.*(get|GET).*(flv).*$”

    Without quotes though…

    From a wireshark capture, a GET shows up with a request always for a url that contains “noflv”. I keep seeing this “noflv” in every capture i save. Perhaps just matching against a GET request for that string name, it will match and shape correctly.

    The original flash pattern file states, without quotes…
    “[FC]WS[x01-x09]|FLVx01x05x09”

    #51450

    AtroposX
    Member

    This forum states how to use squid to cache video content to save on bandwidth. In the acl config section are some regular expressions for youtube and its /videoplayback and /get_video, which may aid in generating a new pattern file.

    http://ubuntuforums.org/showpost.php?p=6153907&postcount=2

Viewing 14 posts - 1 through 14 (of 14 total)

You must be logged in to reply to this topic.