Tuesday, June 2, 2009

Migrating LIVE from one datastore to another in an Integrity VM

Did you know that it is possible to easily migrate data using LVM's pvmove in Integrity Virtual Machines running HP-UX ? I tried it today and not only does pvmove works well, it's almost as fast as if I was on physical hardware. For those who are familiar with ESX's Storage VMotion, this is as close as you can get to achieving similar results. Using pvmove is not as slick as an svmotion since you need to do it directly on the guest. But it works, and that's good enough for me.

Why would you want to do this? Here are some examples:
  • Migrating devices from scsi to avio_stor, without rebooting the VM (assuming it has the avio drivers, of course)
  • Switching from a datastore to another. For example, moving from a flat file to a raw disk
  • Moving data from a RAID-5 volume to RAID-1
  • Migrating from one disk array to another
As an example, here is a sample procedure to move data from an LV datastore to a raw disk datastore, without having to copy data manually or turn off your VM.

Add your new raw device to your VM:

vmhost# hpvmmodify -P myvm -a disk:avio_stor::disk:/dev/rdisk/disk39
vmhost# hpvmstatus -P myvm

[Storage Interface Details]
Guest Physical
Device Adaptor Bus Dev Ftn Tgt Lun Storage Device
======= ========== === === === === === ========= =========================
disk avio_stor 0 1 0 0 0 file /ivm/myvm/disk1.vm
disk avio_stor 0 1 0 3 0 lv /dev/vg_myvm/rlv_vgdata
disk avio_stor 0 1 0 4 0 disk /dev/rdisk/disk39

In the VM itself, make an ioscan to discover the new device

myvm# ioscan
myvm# insf -eC disk # insf required only on 11iv2
myvm# ioscan -kfnC disk

Class I H/W Path Driver S/W State H/W Type Description
disk 0 0/0/1/0.0.0 sdisk CLAIMED DEVICE HP Virtual FileDisk
/dev/dsk/c0t0d0 /dev/dsk/c0t0d0s2 /dev/rdsk/c0t0d0 /dev/rdsk/c0t0d0s2
/dev/dsk/c0t0d0s1 /dev/dsk/c0t0d0s3 /dev/rdsk/c0t0d0s1 /dev/rdsk/c0t0d0s3
disk 3 0/0/1/0.3.0 sdisk CLAIMED DEVICE HP Virtual LvDisk
/dev/dsk/c0t3d0 /dev/rdsk/c0t3d0
disk 4 0/0/1/0.4.0 sdisk CLAIMED DEVICE HP Virtual Disk
/dev/dsk/c0t4d0 /dev/rdsk/c0t4d0

See that new "Virtual Disk" device? Just pvcreate it and add it to your the VG on which you need to migrate data. Here I'm using legacy devices since my VM runs 11iv2, but you can do this with agile devices if you wish.

myvm# pvcreate /dev/rdsk/c0t4d0
myvm# vgextend /dev/vgdata /dev/dsk/c0t4d0
myvm# vgdisplay vgdata

--- Physical volumes ---
PV Name /dev/dsk/c0t3d0
PV Status available
Total PE 1249
Free PE 0
Autoswitch On
Proactive Polling On

PV Name /dev/dsk/c0t4d0
PV Status available
Total PE 1279
Free PE 1279
Autoswitch On
Proactive Polling On

Then, use pvmove to move your physical extents from one datastore to another using pvmove, as you would on a physical server.

myvm# pvmove /dev/dsk/c0t3d0 /dev/dsk/c0t4d0
Transferring logical extents of logical volume "/dev/vgdata/lv_mydata"...

Here, LVM will be moving LIVE all your data from c0t3d0 (an LVM datastore) to c0t4do (a raw disk datastore). The process can take a while, since it's going slow voluntarily to prevent the process from interfering with normal operations.

When it's over, remove the empty PV from the VG

myvm# vgreduce vgdata /dev/dsk/c0t3d0
myvm# pvremove /dev/rdsk/c0t3d0
myvm# rmsf -H 0/0/1/0.3.0

Then, remove it from the VM host:

vmhost# hpvmmodify -P my_vm -d disk:avio_stor::lv:/dev/vg99/rlv_vgdata

And you're done. You've just migrated storage live in your VM.

Can the pvmove be done at the VM Host level?

Here we did the pvmove at the VM Guest level, i.e. inside the VM Guest. Can this be done on the VM Host itself? The answer is yes, but only if you're using LVM datastores. Just pvmove the PVs on the VM Host, and you can migrate data transparently.

I did useLVM datastores for a while, but they're a pain in the butt to manage, as you have two levels of volume groups to consider: one on the host, then one on the guest. There is also a lot of flexibility that is lost: you can't easily export and reimport the guest volume groups from virtual to physical servers, back and forth, when using anything except raw disks.

Yet there is one place where LVM datastores could be useful: to enable a Storage VMotion look-alike for VM guests that do not have any volume manager that can easily do this. I'm thinking here of VMs running Windows, for example.


1 comment:

Ram said...

Oliver, have you tried VGMove and LVMove commands in LVM? They allow online data mobility in physical and virtual environments completing the value prop that PVMove started with.