Open Webalizer DNS cache with PHP

Problem:

Webalizer produces a cache file for DNS to speed up the process for resolving addresses to names but no utilities exist to browse this kind of file.

Background:

Webalizer is a log file analyser which can provide basic site usage stats by processing your apache log files. Its a standard utility in a RHEL envirnment. One feature of it is caching DNS information to speed up resolving addresses for reporting purposes. It uses a Berkeley DB to store these key/value pairs for quick lookup. Unfortunatly there really arn’t an gui tools for getting a look at this data. On my machine, its a 70MB file and I’d really like to get a look at that data.

So there are a couple of ways to do this.
MySQL comes with a bdb storage engine which can read this kind of file.

Or, you could use PHP and its driver to load this information.
I chose the latter.

Environment:
I’m starting with a base install of ubuntu 11 and zend server.
Using zend server makes the process more complicated but makes the process more standardized.  ZS uses the same paths for files on every OS it installs on.

Install dependent software.

Zend doesn’t come with this module by default, so we’ll need to compile it.
Make sure you’ve got the basics needed to compile under ubuntu installed.


aptitude install install build-essential checkinstall autoconf

Next we need to make sure that libdb4 is installed so we can reference it in php.

aptitude install libdb4.8-dev db4.8-util db4.8-doc libdb4.8

And finaly, we need the header files for zend server’s php.

sudo aptitude install php-5.3-source-zend-server

Sanity check – Lets make sure that the module isn’t installed.


php --ri dba
Extension 'dba' not present.

Great. Lets build it.


cd /usr/local/zend/share/php-source/php-5.3.9/ext/dba/
/usr/local/zend/bin/phpize
./configure --with-php-config=/usr/local/zend/bin/php-config --with-db4
make
make install
echo "extension=dba.so" > /usr/local/zend/etc/conf.d/dba.ini

lets check that again


php --ri dba
dba

DBA support => enabled
Supported handlers => gdbm cdb cdb_make db4 inifile flatfile

Directive => Local Value => Master Value
dba.default_handler => flatfile => flatfile

Done. We can now access the dba functions from the command line.
They’ll also get picked up by apache the next time you restart.

Build a quick script to read the dns cache and output it as a comma separated list.


<?php
/**
* Requires the dba module be enabled with db4 support.
*/
$path = dirname(__FILE__) . '/dns_cache.db';
$logPath = $path . '.txt';
echo 'Loading ' . $path . chr(10);

$row = 0;
$h = dba_open($path,'r','db4');
$fh = fopen($logPath,'w+');

if(!$h){
echo 'Failed to open db';
die();
}

echo "Exporting Key/Values pairs...\n";
$key = dba_firstkey($h);
$value = dba_fetch($key,$h);
$value = filter_var($value,FILTER_SANITIZE_URL);
$key = filter_var($key,FILTER_SANITIZE_URL);
fwrite($fh, "$key,$value\n");
$row++;

while($key = dba_nextkey($h)){
$value = dba_fetch($key,$h);
$value = filter_var($value,FILTER_SANITIZE_URL);
$key = filter_var($key,FILTER_SANITIZE_URL);
fwrite($fh, "$key,$value\n");
$row++;
}
echo "Printed $row key/value pairs." . chr(10);
echo 'Done' . chr(10);
dba_close($h);

I called my script test.php.
Then run it from the command line.


php -r ./test.php

Loading /home/{MASKED}/dns_cache.db
Exporting Key/Values pairs...
Printed 703268 key/value pairs.
Done

Not sure if this is the best way to get at this data but its the route I chose and it worked.
Your mileage may vary.

Configure Crashplan on Centos 5.3

Crashplan is a backup tool similar to carbonite or mozy with one great exception: It allows pc to pc backup without a fee and they have clients for windows, mac and linux.

The configuration of this on a headless linux server is a little tricky but can be done.

The crashplan software is divided into two parts: The engine and the desktop. The desktop is the configuration gui and the engine does the work. To configure a headless server, you’ll need to use the desktop on another computer to connect to the headless one via port forwarding.

Check out the instructions

Their instructions are mac centric. Here is how to achieve same thing with windows and putty.

  1. Open putty
  2. Fill in the host and port info to connect to your server
  3. Under ‘Category’ on the left, click on ‘SSH’
  4. Next click on ‘Tunnels’
  5. Check both ‘Local ports accept connections…’ and ‘Remote ports do the same…’
  6. Enter ‘4243’ into the source port box
  7. Enter 127.0.0.1:4200 into the ‘Destination’ box.
  8. Select ‘Local’ and ‘Auto’
  9. Click ‘Add’
  10. Now click open and login to your server as normal.
  11. You’ll now be able to open the crashplan desktop and configure your server.

Before you can begin backing up to your linux machine, you’ll also need to configure your linux firewall to allow connections to ports 4200 and 4243. You’ll need to allow both tcp and udp connections on port 4200.

On centos, add these lines to your /etc/sysconfig/iptables

-A RH-Firewall-1-INPUT -p tcp -m tcp –dport 4200 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp –dport 4243 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp –dport 4200 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp –dport 4243 -j ACCEPT

Right before this line:

-A RH-Firewall-1-INPUT -m state –state ESTABLISHED,RELATED -j ACCEPT

CentOS – Installing GNOME with yum

For me, Linux is a server os and a gui doesn’t belong on a server.
(unless its a vmware server, hense the post on installing gnome)

Here is a one-liner that will add GNOME to your box.

Log in as root and run:

yum groupinstall “X Window System” “GNOME Desktop Environment”

Note: Yum will need to download about 250MB of stuff to install via this method.