forsetti

UCARP and Squeeze

~ Matt Smith ~ 3 Comments

IP fault tolerance on Debian Squeeze couldn’t be easier than with UCARP!:

  • Install UCARP on all nodes: 
    apt-get install ucarp
  • Edit /etc/network/interfaces on all nodes, replacing <CLUSTER IP> and <PASSWORD> as appropriate: 
    iface eth0 inet static
...
ucarp-vid 3
ucarp-vip <CLUSTER IP>
ucarp-password <PASSWORD>
ucarp-xparam -n
iface eth0:ucarp inet static
address <CLUSTER IP>
netmask 255.255.255.255
  • Reboot, and you’re done!  <CLUSTER IP> will now be available from one of the nodes in your cluster at all times!

A few notes:

  • All nodes in your UCARP cluster must be in the same subnet, and the shared cluster IP must also be in the same subnet.
  • UCARP does not do cluster resource management for you, but it does provide hooks.  You will need to write some scripts to handle unmounting/mounting of shared storage, etc.
  • UCARP does not detect service failures, only whole-host (or IP stack) failures.  But if you can detect the failure, simply send SIGUSR2 to the ucarp process to force into passive mode, forcing another node to become master.
All-in-all, about as simple and sufficient as one can get!

GitHub

~ Matt Smith ~ Leave a comment

Whelp, I’ve gone and done it — my first Git project, hosted at GitHub.  I’ve used Subversion quite a bit, but this is my first foray into the world of DVCS.

Aside from the solid technology, the benefits of DVCS, the functionality of the GitHub platform … the GitHub Bootcamp and Help pages really sold me.  The folks at GitHub really know how to ease the transition for new members.  Great job, Hubbernauts!

Convert Dropbox’s Ubuntu package for Squeeze

~ Matt Smith

The Dropbox Ubuntu package almost works on Debian Squeeze … except for one dependency with a numbering scheme that differs between Ubuntu and Debian.  Here is a quick script that extracts the package, corrects the one dependency in DEBIAN/control, and repackages, leaving a new .deb suitable for Squeeze.

convertDropbox4Squeeze:

#!/bin/sh
if [ $# -ne 1 ]; then
	echo "Usage: $0" 'nautilus-dropbox_$version_$arch.deb'
	exit 1
fi
DIR=`mktemp --directory`
mkdir -p $DIR/DEBIAN
dpkg-deb -x $1 $DIR/
dpkg-deb -e $1 $DIR/DEBIAN/
sed -i s/'libnautilus-extension1 (>= 1:2.22.2)'/'libnautilus-extension1 (>= 2.22.2)'/ ${DIR}/DEBIAN/control
dpkg-deb -b $DIR  ${1%.*}_squeeze.deb
rm -rf $DIR

Just download the .deb from the Dropbox site, then run

./convertDropbox4Squeeze nautilus_dropbox.......deb

And then just install the newly created nautilus_dropbx….squeeze.deb .

Linux Management and Monitoring: Zabbix + Puppet ?!?

~ Matt Smith

I’ve spent a good part of this week evaluating various solutions for monitoring and managing Linux servers.  There are many solutions that meet some of the needs, but I have not found a single product that does everything I need.  Without going into significant detail, here are my needs:

  • Simple service up/down monitoring and alerting
  • Detailed metrics for disk, CPU, memory, network, etc
  • Storage and graphing of historical data
  • Flexible creation of custom monitors, triggers, etc.
  • Significant pre-configured monitors, triggers, etc.
  • Deployment/upgrade/removal of packages
  • Configuration management
  • Inventory of hardware/software
  • Rich access control to satisfy various distributed administration and audit requirements
  • *ALL* functionality exposed by web interface
  • Inexpensive, preferably open source
  • Able to scale to several hundred, maybe thousands of Linux servers – and possibly even Windows servers.
  • Should be developed in a language I know so I can modify myself – (C, Perl, Python, Java)
  • Should run on Debian, though it does not need to be in the repo.

Candidate solutions:

  • Nagios
  • OCS-Inventory
  • Hyperic
  • Puppet
  • Zabbix

Conclusion:

Nothing definitive yet, but I’m narrowing in on a combination of Zabbix and Puppet.  I am still looking for something to automate the collection of inventory data, but I think this could be done using the Zabbix API to populate the Zabbix inventory (which is otherwise a manual process).

LazyWeb – any opinions?

Enterprise, Large Scale File Services

~ Matt Smith ~ Leave a comment

Admittedly, the world of file services has changed since Novell ruled the roost with Netware. All sorts of new buzzwords exist: Web Content Management, Enterprise Document Management, Document Archiving, Knowledge Management … but you still can’t beat simple file storage service like Windows offers natively. Except, it is really hard to provide that kind of service at a very large scale.
So here is the question — how do I provide a file service with the following requirements:

  • Must scale beyond 25,000 users (potentially 100,000), each with private “home” directories, plus whatever permutations of group space can be imagined.
  • Must support large amounts of storage, including individual files of several hundred gigabytes, user/group quotas of several terabytes.
  • Must support access from OSX, Windows, and Linux such that applications on these systems can natively open, read, write files — in other words, similar to simple CIFS access, though a non-native client to support this functionality is acceptable.
  • Must support some level of access from mobile devices, including Android, iPhone/iPad, Windows Mobile, and ideally Blackberry too.
  • Must provide a rich “sexy-looking” web interface.
  • Must provide consistent abstract interface — in other words, scaling across hundreds of servers is acceptable, as long as users never need to be told “connect to server #17 for X, and server #53 for Y”.  There should be some sort of abstracted virtual filesystem.
  • Must support user-controllable ACLs to facilitate sharing and security.
  • Must be accessible by non-technical end users with very little handholding – should be “intuitive”.
  • Must allow integration with backup solution that can provide file-level restoration.
  • Should allow for storage of data to be accessed by Linux and Windows servers, such as user generated web content, HPC-generated research data, etc.
  • Should allow for attachment of metadata for searching.
  • Should allow integration with backup solution that allows end-user to perform file-level restoration.

Some have tried to convince me that Windows DFS can do all this, but I have yet to see a deployment that actually encompasses all of the above.  Anyone have any references?

I am quite intrigued by OpenAFS, using the filedrawers web interface, and possibly using the Samba gateway to avoid deploying the OpenAFS client to every machine — anyone with any experience doing this?  Anyone serve OpenAFS data out over DAV via Apache, mod_dav, and mod_waklog?  Is filedrawers or DAV an acceptable mobile device access mechanism?  Pitfalls?

What else should I be considering?