Monitoring VMFS disk space

I always thought that monitoring free space on VMFS volumes was not useful, but I was wrong. If one creates a snapshot of a VM, and forgets to remove this snapshot after a while, the VMFS might become full and some VMs will crash.

So... how can it be monitored? Well under ESX the quickest solution I found is writing a script, called from a key-authenticated SSH that is executed by Nagios.

I'll spare the nagios side, but here's my generic script that runs on ESX. It uses "vdf" which produces df-like output for VMFS volumes. Of yes, it's not executed as root but as some other unpriviledged user that calls vdf via sudo: monitor_vmfs.sh

N.B. Installing NRPE in ESX is a venture I will not consider. Many people in the forums tend to have problems with this, and it would take too much effort. SSH works right out of the box, is more secure, and does not require me to install third-party software in the ESX console (except my script).

Making (fax)modems work under VMware ESX

Yesterday I was asked if it was possible to make an external faxmodem work under a Windows Server 2003 VM. The customer has an application that is linked against the Microsoft Fax Service DLL, and they have to use a honest-to-goodness modem. The answer was: probably. I checked and indeed, serial ports can virtualized in a VM.

That's nice, but...

• The modem being physically connected to a host, you can't put the VM in a cluster (unless maybe you have identical modems hooked up on each of your hosts, but I didn't try it)
  
  
• We have blade servers, and to use their serial ports we need to hook up a three-way cable in front of the blade, which has a USB, serial and video connectors. This is for diagnostic purposes only. Hooking a modem on this will work, but that's not really pretty to look at.
  

The solution I found was this: we already have Digi PortServers in production, which we use to access consoles on Alpha systems, and service a few modems as well. By installing RealPort inside a Windows VM, one can redirect a COM port, through the lan, to a port on the PortServer. Windows thinks it's speaking to a physically attached serial device, while in fact it's hooked up on a PortServer.

Tada! The modem now works inside a VM. And it will also work if I VMotion it to another host. Case closed.

RealPort exists on Linux and Solaris as well (no mention if it's x86 or SPARC, though...) so these VMs can also use a similar solution as well.

Sidenote: You can even assign a virtualized serial port on a named pipe on the ESX host. I initally thought of writing a perl script that would have opened the pipe, and use Net:Telnet to telnet to the PortServer. That was before I found out that RealPort existed.

Petit cours sur vxfsd

On me pose souvent des questions sur vxfsd et le rôle de ce daemon est peu documenté et il n'a pas de man page. Alors voici des explications sur vxfsd.

Sous Unix, une inode est une structure qui contient le metadata sur les fichiers (taille, emplacement sur disque, propriétaire, etc). Ici vous trouverez un .pdf avec un exemple d'inode: http://www.tux4u.nl/freedocs/unix/draw/inode.pdf

Sous HP-UX, ces structures sont mis en cache pour accélérer les performances des I/O. La cache est dynamique et grossit au besoin, consommant de la RAM si nécessaire.

vxfsd est un daemon dont la job est de scanner la cache des inodes afin de libérer celles qui n'ont pas été référencées depuis un certain temps, et de libérer la mémoire. Plus il y a d'inodes à libérer, plus il va travailler longtemps.

Quand un backup roule, il lit tous les fichiers, donc tous les inodes sont ultimement mis en cache. Il est donc normal de voir vxfsd spinner peu de temps après un backup.

C'est un trade-off, on peut tuner le kernel et diminuer l'utilisation de vxfsd en configurant une cache statique. On peut le faire sur un serveur qui a peu de puissance et s'il faut réduire la consommation de vxfsd au minimum.

Mais par souci de simplicité je préfère laisser la configuration par défaut ailleurs même si c'est pas la config idéale.

Références:
Common Misconfigured HP-UX resources: http://docs.hp.com/en/5992-0732/5992-0732.pdf

First post & welcome

Welcome to my blog.

I'll post mostly work-related technical stuff here. I tend to cut my teeth on technical issues quite a lot, and I'll post my findings here. Both in english in french. Don't expect personal information here, unless I'm really in the mood.