Monday, January 19, 2009

Shrinking and Compressing Linux Virtual Image with Linux Host

I was always short of disk, because I have several virtual machines with my laptop. The virtual disk sizes grows fast, even the actual disk sizes I saw in the guest machine are much smaller. So, it's really necessary to shrink the disks from time to time. And, I also need a good compressor which can provide better compressing rate.

Since I run Linux guests in Linux host, it's a little tricky to shrink the disks. After googling and trying several approaches, I got following solutions:

Shrinking with X Based vmware-toolbox


vmware-toolbox is easy to use, but it requires X. Of cause, for most of my vms, I don't want a X server in them. So, remote X server helps now. With xauth (Debian package xbase-clients, or Yum package xorg-x11-auth), X forwarding works with SSH:

ssh -X user@vm

Then vmware-toolbox can be called, and the window will be forwarded to your desktop.

Notice: vmware-toolbox doesn't support logical volume. Please see the following section for shrinking virtual disk with logical volume.

Shrinking with Command Line


If there's no X forwarding for the guest machine, the virtual image can be shrinked with command line also. The brief steps are:

* Download VMWare Server 2 package and unpack vmware-vdiskmanager from it. I'm still using VMWare Server 1 now, since the browser based client doesn't work well with my Linux. But the vmware-vdiskmanager doesn't provide the feature of disk shrinking.
* Create symbol links for libcrypto.so.0.9.8 and libssl.so.0.9.8 . Assuming the vmware-vdiskmanager is placed in folder /usr/local/bin, then the symbol links are:

sudo mkdir -p /usr/local/bin/libdir/lib/libcrypto.so.0.9.8/
sudo ln -s /usr/lib/libcrypto.so.0.9.8 /usr/local/bin/libdir/lib/libcrypto.so.0.9.8/libcrypto.so.0.9.8
sudo mkdir -p /usr/local/bin/libdir/lib/libssl.so.0.9.8/
sudo ln -s /usr/lib/libssl.so.0.9.8 /usr/local/bin/libdir/lib/libssl.so.0.9.8/libssl.so.0.9.8

Now, the new vmware-vdiskmanager with disk shrinking (-p and -k) works.

Then, shrink the disk:
(Following content are based on http://communities.vmware.com/message/257602 .)
* Power on the guest machine, fill the file system with zero:

dd if=/dev/zero of=zerofile bs=1024k count=2048
rm zerofile

And swap partition also (assuming /dev/sda5 is the swap partition):

sudo swapoff
sudo if=/dev/zero of=/dev/sda5 bs=1024k count=2048

* Shutdown the virtual machine.
* Mount the virtual drive using the vmware-mount command; e.g.

sudo vmware-mount.pl debian40server.vmdk 1 -t ext3 /tmp/debian40/

* Using the vmware-vdiskmanager prepare disk for shrinking; e.g. using v: as mounted drive:

sudo /usr/local/bin/vmware-vdiskmanager -p /tmp/debian40/

* Unmount the drive.
* Shrink the disk (it's recommended to execute the following command with common user, instead of root):

/usr/local/bin/vmware-vdiskmanager -k debian40server.vmdk


Now, the disk is shrinked.

If there file system is in a logical volume of the virtual disk, manual operations of LVM are required for the shrinking (based on information available at http://seattlecentral.edu/cgi-bin/cgiwrap/dmartin/moin.cgi/VMware#head-b6e75c6b7ea67c9dd57b2adb91c9367ccaa24bcd and http://www.debuntu.org/how-to-install-ubuntu-over-lvm-filesystem):

# Make sure the VG in your VM is not the same as any VGs you have anywhere on your host box.
# Stop the VM
# load the nbd (Network Block Device) module into the running kernel
# Make sure you have /dev/nb* (use "MAKEDEV nb" if you don't)
# run "vmware-loop -p filename.vmdk" and note which partition you have your LVM on
# run "vmware-loop filename.vmdk partition-number /dev/nb*" to attach the file to the network block device
# run "lvmdiskscan" to make it re-scan the disks
# run lvscan to see the nice list
# run 'lvchange -ay volgroup' where volgroup is the volume group on your VM (Please try sudo modprobe dm-mod on "Incompatible libdevmapper" error.)
# run lvscan to see the nice list, this time with your VM's LVs listed as ACTIVE
# Mount and play with LVs
# Then jump to the step of vmware-vdiskmanager -p

Shrinking with Command Line


Then, comes to the second goal: a compressor with better compressing rate. I archived a virtual machine folder with GZIP, BZIP2 and 7Z. Following are the archive files I got:

-rwxrwx--- 1 root plugdev 195722873 2009-01-19 20:28 debian40server.7z
-rwxrwx--- 1 root plugdev 262460006 2009-01-19 20:29 debian40server.tar.gz
-rwxrwx--- 1 root plugdev 229599667 2009-01-19 20:34 debian40server.tar.bz2


Of cause, 7Z provides better compressing rate (and requires much longer compressing time). But, for me, it's worthy when comparing to the disk usage of versions of VM backups and transferring time cost over the network.

Labels: , ,

0 Comments:

Post a Comment

<< Home