Open Webalizer DNS cache with PHP

Problem:

Webalizer produces a cache file for DNS to speed up the process for resolving addresses to names but no utilities exist to browse this kind of file.

Background:

Webalizer is a log file analyser which can provide basic site usage stats by processing your apache log files. Its a standard utility in a RHEL envirnment. One feature of it is caching DNS information to speed up resolving addresses for reporting purposes. It uses a Berkeley DB to store these key/value pairs for quick lookup. Unfortunatly there really arn’t an gui tools for getting a look at this data. On my machine, its a 70MB file and I’d really like to get a look at that data.

So there are a couple of ways to do this.
MySQL comes with a bdb storage engine which can read this kind of file.

Or, you could use PHP and its driver to load this information.
I chose the latter.

Environment:
I’m starting with a base install of ubuntu 11 and zend server.
Using zend server makes the process more complicated but makes the process more standardized. ZS uses the same paths for files on every OS it installs on.

Install dependent software.

Zend doesn’t come with this module by default, so we’ll need to compile it.
Make sure you’ve got the basics needed to compile under ubuntu installed.

aptitude install install build-essential checkinstall autoconf

Next we need to make sure that libdb4 is installed so we can reference it in php.
aptitude install libdb4.8-dev db4.8-util db4.8-doc libdb4.8

And finaly, we need the header files for zend server’s php.
sudo aptitude install php-5.3-source-zend-server

Sanity check – Lets make sure that the module isn’t installed.

php --ri dba Extension 'dba' not present.

Great. Lets build it.

cd /usr/local/zend/share/php-source/php-5.3.9/ext/dba/ /usr/local/zend/bin/phpize ./configure --with-php-config=/usr/local/zend/bin/php-config --with-db4 make make install echo "extension=dba.so" > /usr/local/zend/etc/conf.d/dba.ini

lets check that again

php --ri dba dba


DBA support => enabled

Supported handlers => gdbm cdb cdb_make db4 inifile flatfile
Directive => Local Value => Master Value

dba.default_handler => flatfile => flatfile

Done. We can now access the dba functions from the command line.
They’ll also get picked up by apache the next time you restart.

Build a quick script to read the dns cache and output it as a comma separated list.

<?php /** * Requires the dba module be enabled with db4 support. */ $path = dirname(__FILE__) . '/dns_cache.db'; $logPath = $path . '.txt'; echo 'Loading ' . $path . chr(10);


$row = 0;

$h = dba_open($path,'r','db4');

$fh = fopen($logPath,'w+');
if(!$h){

echo 'Failed to open db';

die();

}
echo "Exporting Key/Values pairs...\n";

$key = dba_firstkey($h);

$value = dba_fetch($key,$h);

$value = filter_var($value,FILTER_SANITIZE_URL);

$key = filter_var($key,FILTER_SANITIZE_URL);

fwrite($fh, "$key,$value\n");

$row++;
while($key = dba_nextkey($h)){

$value = dba_fetch($key,$h);

$value = filter_var($value,FILTER_SANITIZE_URL);

$key = filter_var($key,FILTER_SANITIZE_URL);

fwrite($fh, "$key,$value\n");

$row++;

}

echo "Printed $row key/value pairs." . chr(10);

echo 'Done' . chr(10);

dba_close($h);

I called my script test.php.
Then run it from the command line.

php -r ./test.php

Loading /home/{MASKED}/dns_cache.db Exporting Key/Values pairs... Printed 703268 key/value pairs. Done

Not sure if this is the best way to get at this data but its the route I chose and it worked.
Your mileage may vary.

I sketch in code

Open Webalizer DNS cache with PHP

One response to “Open Webalizer DNS cache with PHP”