IMPOSSIBLE PROCESSOR LOAD 2503%

Home Page Forums Network Management Signal a BUG IMPOSSIBLE PROCESSOR LOAD 2503%

This topic contains 23 replies, has 0 voices, and was last updated by  pgbuz 2 years ago.

Viewing 15 posts - 1 through 15 (of 25 total)
  • Author
    Posts
  • #44645

    pgbuz
    Participant

    It is possible to have a load of 2.506 % 1056 connections with a perfect run of zeroshell? Seems to be a wrong information.
    I don’t have strong activity on the router except the upload continuously 24/24h by ftp of 500.000 1.000.000 of files a day with ftp session continuously open. there are also 25 users voip connected. I don’t know if this can generete the error. I remember that I didn’t had this error in the past. I made the last update and I have the kernel 4.4.13 – 64. I finished this message and the load grow up to 2.516 % and after go down to 2503% 1056 connections! ???

    #54255

    iulyb
    Member

    Hi,
    You should identify a few things in order to figure out if that number if real or bogus and where is coming from. Login to command line and do a top command. Pay attention to load.
    First, on a server with a real load like the one described by you it will be a delay from a few seconds to tens of minutes. If your router is responsive it is possible that the top result to be bogus.
    Then take the number from top and dived it to the number of cpus.
    To know how many CPUs your system are seeing do a

    $ cat /proc/cpuinfo |grep ^processor 

    Usually the number of CPUs is the number of cores multiplied with the number of threads so what you see here should match with what you know about your hardware.
    Then divide the load from top to the number of CPUs. This is the number you should see on ZS interface.
    Let me know what you find.

    #54256

    pgbuz
    Participant

    At top command the page upload in 1 second, not more.

    cat /proc/cpuinfo |grep ^processor –> processor 0/1/2/3 (it is OK: quadcore)
    top command: I read only 3 load number 100.92 100.51 100.3
    Are not the load of zeroshell webpage that now is 2509% (1225 connections)

    I’m using zeroshel as virtual box machine, 4 processor, 1 Gbyte RAM, release 3.6.0 kernel 4.4.13 64.

    #54257

    iulyb
    Member

    100.92 100.51 100.3 are the average system load (short time, medium time, long time)

    Basically your load is 100 / 4 approx 25. So ZS should display something around 25
    This is high but should have nothing to do with what ZS display.

    I just did a check and this is the command used by ZS.


    uptime |awk -F"average:" '{print $2}' |awk -F"," '{print $3}' |awk '{print $1}'

    this script is in kerbynet/api/LOAD15
    Similar script is in scripts/bitrate (perl) the one that produce the result on ZS main page but is slightly different:

    uptime|awk -F'average:' '{print $2}'|awk -F, '{print $1}'

    Please check both commands from command line to see if they return right or not. On my laptop and my ZS system returns right

    $ uptime
    16:19:21 up 1 day, 21:05, 2 users, load average: 0.33, 0.40, 0.34
    $ uptime |awk -F"average:" '{print $2}' |awk -F"," '{print $3}' |awk '{print $1}'
    0.40
    #54258

    pgbuz
    Participant

    first command result 100.26
    second command result 100.42
    zeroshell gui webpage 2504%, 1045 connections

    #54259

    iulyb
    Member

    Sorry my bad.. forgot to ask you that after those commands to run

    root@zs ~> /root/kerbynet.cgi/scripts/bitrate
    22.71 Kbit/s (Connections: 178 Load: 0%)

    If this show 2500 then please run:

    root@zs ~>/root/kerbynet.cgi/scripts/cpus
    4

    Please post the results of those 2 scripts.

    #54260

    pgbuz
    Participant

    first command –> 29.89 Kbit/s (Connections: 972 Load: 2506%)
    second command –> 4

    #54261

    iulyb
    Member

    Ok, so now we know the culprit. (bitrate script)

    I don’t like how the commands are executed on that perl script. I am going to rewrite the part related to cpus and load. This info is available in /proc/cpuinfo and /proc/loadavg. I think this is because of reusing file handler in a busy env, maight be also a perl bug.
    I have some experience in perl 😉 but I will have to rely on you for testing.

    I am going to change how the data is acquired, never like it to use open and pipe instead of qx//

    #54262

    pgbuz
    Participant

    I don’t have so big experience and I’m happy to help this nice project. Can I ask you if zeroshell can be comparable with cisco systems…. To me seems comparable

    #54263

    iulyb
    Member

    Hi,
    Can you replace /scripts/bitrate script with

    #!/usr/bin/perl
    use warnings;
    use Scalar::Util qw(looks_like_number);
    sub bits2Xbits {
    my $bits=shift;
    if($bits > 1000000000) {
    $bits = ( sprintf( "%0.2f", $bits/1000000000 )). " Gbit";
    } elsif ($bits > 1000000) {
    $bits = ( sprintf( "%0.2f", $bits/1000000 )). " Mbit";
    } elsif ($bits > 1000) {
    $bits = ( sprintf( "%0.2f", $bits/1000 )). " Kbit";
    } else {
    $bits = sprintf( "%0.2f", $bits ). " bit";
    }
    return $bits;
    }
    sub bytes2Xbytes {
    my $bytes=shift;
    if($bytes > 1000000000) {
    $bytes = ( sprintf( "%0.2f", $bytes/1000000000 )). " GB";
    } elsif ($bytes > 1000000) {
    $bytes = ( sprintf( "%0.2f", $bytes/1000000 )). " MB";
    } elsif ($bytes > 1000) {
    $bytes = ( sprintf( "%0.2f", $bytes/1000 )). " KB";
    } else {
    $bytes = sprintf( "%0.2f", $bytes ). " B";
    }
    return $bytes;
    }



    open PREV,"/tmp/bitrate-prev";
    while ($entry=) {
    ($Proto,$sIP,$dIP,$sPort,$dPort,$RX,$TX) = split(':',$entry);
    $PTraffic{"$Proto:$sIP:$dIP:$sPort:$dPort"}=$RX+$TX;
    }
    close PREV;
    open PREV,">","/tmp/bitrate-prev";


    open CONNTRACK,"conntrack -L 2>/dev/null|awk -F= ' /^(udp|tcp)/ {printf "%s:%s:%s:%s:%s:%s:%s\n",substr($1,0,index($1," ")-1),substr($2,0,index($2," ")-1),substr($3,0,index($3," ")-1),substr($4,0,index($4," ")-1),substr($5,0,index($5," ")-1),substr($7,0,index($7," ")-1),substr($13,0,index($13," ")-1)} /^icmp/ {printf "%s:%s:%s:type=%s:code=%s:%s:%s\n",substr($1,0,index($1," ")-1),substr($2,0,index($2," ")-1),substr($3,0,index($3," ")-1),substr($4,0,index($4," ")-1),substr($5,0,index($5," ")-1),substr($8,0,index($8," ")-1),substr($15,0,index($15," ")-1)} /^unknown/ {printf "%s:%s:%s:::%s:%s\n",substr($1,0,index($1," ")-1),substr($2,0,index($2," ")-1),substr($3,0,index($3," ")-1),substr($5,0,index($5," ")-1),substr($9,0,index($9," ")-1)}'|";



    $Throughput=0;
    while ($entry=) {
    ($Proto,$sIP,$dIP,$sPort,$dPort,$RX,$TX) = split(':',$entry);
    if ($dIP ne "127.0.0.1") {
    chomp($TX);
    if (looks_like_number($TX) && looks_like_number($RX)) {
    print PREV "$Proto:$sIP:$dIP:$sPort:$dPort:$RX:$TXn";
    if (defined ($PBytes=$PTraffic{"$Proto:$sIP:$dIP:$sPort:$dPort"})) {
    if ($RX+$TX-$PBytes > 0) {
    $Throughput+=$RX+$TX-$PBytes;
    }
    } else {
    $Throughput+=$RX+$TX;
    }
    }
    }
    }
    close PREV;
    close CONNTRACK;

    open TS,"<","/tmp/bitrate-ts";
    $PTS=;
    close TS;
    if (!looks_like_number($PTS)) {
    $PTS=0;
    }
    $NOW=time();

    $DELTA=$NOW-$PTS;
    open TS,">","/tmp/bitrate-ts";
    print TS "$NOW";
    close TS;

    my $CONN = `cat /proc/sys/net/netfilter/nf_conntrack_count`;
    chop($CONN);

    my @Load=split / /, `cat /proc/loadavg`;
    my $CPUs=`/root/kerbynet.cgi/scripts/cpus`;
    $CPUs=1 unless $CPUs;
    my $Bitrate=$Throughput*8/$DELTA;
    my $LOAD=int(100*$Load[1]/$CPUs);
    printf "%s/s (Connections: %s Load: %s%%)n",bits2Xbits($Bitrate),$CONN,$LOAD;

    I rewrote it from line 78, Let me know if it works for you. If copy paste doesn’t work then I can put it somewhere.

    About your question .. well you can compare everything with anything as long as you have a common base. Now against cisco, they sell a product that has hardware + software. If we throw in some decent HW that have lunx drivers then ZS can compete especially on the small biz side. However some stuff is more easy on Cisco, but you pay for that.
    On hi end Cisco is a different beast, mainly on th HW optimization side like drivers and so. Now keep in mind that underlying OS is a linux flavour but most of stuff is build tested and optimized in house so is hard to compete on this scale. Note, as a full disclosure I still work for one of Cisco’s former competitors.

    #54264

    pgbuz
    Participant

    Iulyb don-t be afraid, but can you help me more? I don-t know how I can download it in zeroshell machine (cannot do copy paste). Will be perfect download from somewhere. And you have to tell me preciscly what is the script to change and how. Is a file /script/bitrate to change?

    #54265

    iulyb
    Member

    Hi,
    Yes there is a script /root/kerbynet.cgi/scripts/bitrate
    I uploaded the new version here: http://jb68.com/pkg/bitrate

    You will need to go on a dir on DB (the only one that survive reboot ). On mine I created a structure /DB/opt/patch .You can do the same on your ZS

    mkdir -p /DB/opt/patch

    # download bitrate
    wget http://jb68.com/pkg/bitrate

    #make it exec
    chmod +x /DB/opt/patch/bitrate

    # cross your fingers and run it
    /DB/opt/patch/bitrate

    # If works th overwrite the originaenl bitrate
    cp /DB/opt/patch/bitrate /root/kerbynet.cgi/scripts/bitrate

    If everything OK then add last cp into postboot to preserve functionality after reboot

    #54266

    pgbuz
    Participant

    I’m sorry JulyB, doesn’t resolve the problem. I downloaded the patch in the patch folder, I run it and I read 4.09 Mbit/s (Connections: 987 Load 2504%). If you like, I permit to you to see and analize directly the problem using teamviewer. Contact me by personal mail.

    #54267

    iulyb
    Member

    We are talking of 3 line of codes.. Any way I added some debug print lines on a new script
    http://jb68.com/pkg/bitrate_dbg

    Just download, make it exec then run it on zs and post the results. Here are mine to see what I need:

    Load Line=0.11 0.16 0.26 2/527 25021
    CPUs=1
    Load=16
    0.00 bit/s (Connections:35 Load: 16%)

    It might be an issue with printf, but for now just post the output.

    #54268

    pgbuz
    Participant

    This time is good.
    CPUs=4

    Load=8
    200.76 Kbit/s (Connections: 1007 Load: 8%)

    good work!

    CAN I overwrite bitrate with this file _dbg???

Viewing 15 posts - 1 through 15 (of 25 total)

You must be logged in to reply to this topic.