Tag: Debuggers

Network Interface Utilization in Solaris

A friend asked me how he could see the network utilization in Solaris. It seems like a fairly simple request but for some reason this is not a simple command line away.

In Linux I would instinctively go straight to iptraf. I don’t know if iptraf is the tool of choice these days but I’m pretty sure it is an apt-get away if not already installed.

If you are a DTrace wizard, you could whip something up. Maybe you could get the information from one of the of the DTraceToolkit scripts if their installed. The DTraceToolkit scripts I’ve seen seem to give too much information as most of them are concentrated on not only telling you if the network is loaded but what is loading it as well.

For the sake of practice I wrote the following script:

#!/usr/bin/perl -w
print "Interface: ";
$if=<>;
chomp($if);
$max=`dladm show-dev -p $if | awk -F= '{print \$3}' | awk '{print \$1*1024*1024/8}'`;
print "Max speed: ",$max,"\n";
$if=~m/([a-z0-9]+?)(\d+)/;
($module,$instance)=($1,$2);
$last_rbytes=0;
$last_obytes=0;
while(1){
@kstat=`kstat ${module}:${instance}:mac:/[or]bytes\$/ |awk '{print \$2}'`;
chomp(@kstat);
if($last_rbytes!=0){
printf("%02d%%\n",
(($kstat[$#kstat-1]-$last_rbytes)+
($kstat[$#kstat-2]-$last_obytes))/$max*100);
}
$last_rbytes=$kstat[$#kstat-1];
$last_obytes=$kstat[$#kstat-2];
sleep 1;
};

This script will ask you which interface you want to watch and then print out the utilization percentage on a new row every ~second.

On a side note, it seems strange to me the the received bytes are stored in kstat as rbytes while the transmitted bytes are stored in obytes. The only answer I can come up with is that if they would have chosen ibytes (in bytes) instead of rbytes, then the ‘i’ and ‘o’ might become interchanged in typos since they are next to each other on the keyboard. If they would have chosen tbytes (transmitted bytes), the same situation occurs- ‘r’ next to ‘t’. Still, as a friend pointed out, they could have used sbytes (sent bytes) which makes more sense than obytes.

Top on Solaris

Recently, I was asked to give some advice on an integration project involving some Solaris web servers . One of the sides requested to install the top command.
Now I know and love top for Linux but using top on Solaris is a waste in my opinion. Solaris comes with the prstat command built in- why use something else?
Of course he answered that top was standard for him and he was used to it but I felt obliged to convince him otherwise so I dug around and found some proof 🙂

Brendan Gregg wrote up a great piece comparing top vs prstat using dtrace on his website:
http://www.brendangregg.com/DTrace/prstatvstop.html.

In summary, he finds the following:

  • Top uses more system calls than prstat
  • Top opens and closes the psinfo file over and over while prstat only open it once and saves the file handle
  • Top takes more cpu time to do its job than prstat due to the overhead in the extra system calls and code differences
  • When top uses the cpu it uses it for longer than prstat
  • Most of the issues top has compared to prstat are connected to the number of processes running on the server so the more processes running, the worse top will perform compared to prstat

Aside from the performance issues, prstat also has the ability to give you project and zone related information which I doubt top knows about.

In short top is great for Linux but if you are going to use Solaris, use prstat!

Sun Apache2 breaks PHP?

This post isn’t going to solve anyone’s problems but maybe someone will solve mine.
I recently compiled and packaged a new release of php4 (4.4.7) for use with Sun’s Apache2 (Solaris 10 11/06 patched).
Unfortunately for some very strange reason, Apache segfaults whenever it tries to server a page. It segfaults even if the page has no php involved and only if the php module is loaded.

Here is some sample truss output:

2146:   stat64("/var/apache2/vservers/htdocs/favicon.ico", 0xFFBFF720) Err#2 ENOENT
2146: lstat64("/var", 0xFFBFF720) = 0
2146: lstat64("/var/apache2", 0xFFBFF720) = 0
2146: lstat64("/var/apache2/vservers", 0xFFBFF720) = 0
2146: lstat64("/var/apache2/vservers/htdocs", 0xFFBFF720) = 0
2146: lstat64("/var/apache2/vservers/htdocs/favicon.ico", 0xFFBFF720) Err#2 ENOENT
2146: Incurred fault #6, FLTBOUNDS %pc = 0xFE887CF8
2146: siginfo: SIGSEGV SEGV_MAPERR addr=0x00000054
2146: Received signal #11, SIGSEGV [caught]
2146: siginfo: SIGSEGV SEGV_MAPERR addr=0x00000054
2146: schedctl() = 0xFEE06000
2146: lwp_sigmask(SIG_SETMASK, 0x00000400, 0x00000000) = 0xFFBFFEFF [0x0000FFFF]
2146: chdir("/usr/apache2") = 0
2146: sigaction(SIGSEGV, 0xFFBFF2B8, 0xFFBFF358) = 0
2146: getpid() = 2146 [912]
2146: getpid() = 2146 [912]
2146: kill(2146, SIGSEGV) = 0
2146: setcontext(0xFFBFF298)
2146: Received signal #11, SIGSEGV [default]
2146: siginfo: SIGSEGV pid=2146 uid=80

I configured Solaris to dump core and gdb’d the core file:

#0  php_handler (r=0x1964e8) at /root/dev/php-4.4.7/sapi/apache2handler/sapi_apache2.c:470
470 /root/dev/php-4.4.7/sapi/apache2handler/sapi_apache2.c: No such file or directory.
in /root/dev/php-4.4.7/sapi/apache2handler/sapi_apache2.c
(gdb) bt full
#0 php_handler (r=0x1964e8) at /root/dev/php-4.4.7/sapi/apache2handler/sapi_apache2.c:470
ctx = (php_struct *) 0x0
conf = (void *) 0x0
brigade = (apr_bucket_brigade *) 0x197f38
bucket = (apr_bucket *) 0x0
parent_req = (request_rec *) 0x0
#1 0x0002e730 in ap_run_handler ()
No symbol table info available.
#2 0x0002ed20 in ap_invoke_handler ()
No symbol table info available.
#3 0x0002baa8 in ap_process_request ()
No symbol table info available.
#4 0x0002670c in .st_double_foreff ()
No symbol table info available.
#5 0x0002670c in .st_double_foreff ()
No symbol table info available.
Previous frame identical to this frame (corrupt stack?)
(gdb) quit

I found some people with a similar problem but no answers here: http://forum.java.sun.com/thread.jspa?threadID=5137425&tstart=270

I even tried my old package for php 4.4.3 but no luck. The only difference between the old machine and the new that I know of is that the old machine is 06/06 and the new one is 11/06????

In the end, I ditch Sun’s Apache2 and install Blastwave packages and everything works.

All I can say is that Sun is wasting it’s time on desktops. Sun needs to bring their software delivery and upgrade management up to Redhat/Debian standards. If they would just do that, they could start charging for their OS again.