Category Archives: XenServer

XenServer GrubConf.py fix script

In a previous article (Fixing XenServer error “Unable to find partition containing kernel”) I described how to fix a recurring problem after patching XenServer 6.2 installations. While the fix is known from years it’s never been adopted, and different distros (such as Ubuntu LTS 14.04) fail to boot properly when the GrubConf.py (on dom0) gets reset to its default state.

Being the lazy person that I am I decided to set up a script to do the work for me, after all we’re admins, not monkeys.

#!/bin/bash
GRUBCONF="/usr/lib/python2.4/site-packages/grub/GrubConf.py"
PATCHED=$(grep "_entry" $GRUBCONF | wc -l)
if [ $PATCHED -eq 2 ]; then
  echo "GrubConf.py is already patched"
else
  echo "Patching GrubConf.py to fix boot..."
  sed -i 's/_entry}":/_entry}":\n                        arg = "0"\n                    elif arg.strip() == "${next_entry}":/' $GRUBCONF
  PATCHED=$(grep "_entry" $GRUBCONF | wc -l)
  if [ $PATCHED -eq 2 ]; then
    echo "- Patch was applied successfully."
  else
    echo "- There was a problem while applying the patch."
  fi;
fi;
echo

This does just what I/we used to do manually: detects if GrubConf.py has been reverted and, if not, patches it up. Supplementary tests added for paranoia 🙂

Solution to XenServer VM landing on initramfs

In my journey through XenServer lands, I once experienced a change in the UUID of the root partition, which resulted in a failed boot and being dropped into initramfs. Although this solution should have worked just fine, I either didn’t know of it at the time or it wouldn’t work for some reason.

While inside the VM initramfs I also had the pleasure of not having any text editor of sorts: no vi, no vim, no nano. Nothing at all. Even though I found the new UUID through the use of ls -al /dev/disk/by-uuid/ (and some guesswork), I had no way to edit the grub configuration. So, after some trial and error, I came up with the following:

(initramfs) mount /dev/xvda1 /mnt; cd /mnt
(initramfs) cp grub.cfg grub2.cfg
(initramfs) cat grub2.cfg | sed s///g > grub.cfg

After the proper root partition UUID was set in place, a reboot was all it took to set the machine back up and running.

Fixing XenServer error “Unable to find partition containing kernel”

Edit: I provided a scripted solution in this article. To know why the error happens and its fixes just keep reading.

Error: Starting VM ” – The bootloader for this VM returned an error — did the VM installation succeed? Unable to find partition containing kernel

This has been the major nightmare I had so far with XenServer machines. When upgrading distros it might just so happen they will refused to boot forever after, in my case it affects Ubuntu 14.04.x, not officially supported. Let’s look at the solutions.

Modifying GrubConf.py

Although the fix is in Citrix’s repository since 2012, give or take, it has not been streamed to the executables yet for some uncertain reasons. If you open /usr/lib/python2.4/site-packages/grub/GrubConf.py at line 428 you see:

if arg.strip() == "${saved_entry}":
    arg = "0"

This causes a problem during the parsing and two lines should be added:

if arg.strip() == "${saved_entry}":
    arg = "0"
elif arg.strip() == "${next_entry}":
    arg = "0"

After the file has been modified and saved you will be able to properly start the virtual machines. This holds currently true for Ubuntu 14.04 & 14.04.1 LTS server installations, but might also work for other distributions. Also take in consideration that applying some patches to the host might revert this change, so you might need to do it again at some point in the future.

Modifying grub.cfg

This might not be enough if the problem does not relate to PyGrub but rather to the configuration file itself. While on the host machine, you can run the following command:

# EDITOR=vi xe-edit-bootloader -n  -p 1

This command will prepare and mount the drives assigned to the virtual machine, edit the boot loader configuration in vi and after quitting from vi will unmount and cleanup. If you installed Grub2 or you made mistakes in its configuration, this will allow you to edit it from inside the host machine, after which you will be able to properly boot it up.

XenServer VMs and easy autostart

One of the most tedious tasks I find myself doing on XS installation is switching VMs’ autostart off and on. While no big deal it got boring real fast. I thus crafted a couple bash scripts to be run on the XS host that speed the task up.

#!/bin/bash
pool=$(xe pool-list | sed -n "s/.*\uuid.*: \(.*\)/\1/p" | awk '{ system("xe pool-param-get param-name=other-config param-key=auto_poweron uuid="$0" 2>&1") }' | sed 's/Error: Key auto_poweron not found in map/false/g' | sed 's/false/INACTIVE/g' | sed 's/true/ACTIVE/g')
list=$(xe vm-list | sed -n "s/.*\(uuid\|name-label\).*)[ ]*: \(.*\)/\2/p" | awk '{ print $0; } NR%2==1 { system("xe vm-param-get param-name=other-config param-key=auto_poweron uuid="$0" 2>&1" ) }' | sed 's/Error: Key auto_poweron not found in map/false/g' | awk '{id=$0; getline; atrun=$0; getline; print atrun " " $0 " [" id "]"; }' | sort -k 2 | grep -v "Transfer VM for VDI" | grep -v "Control domain on host" )
active=$(echo "$list" | sed -n "s/true \(.*\)/\1/p")
inactive=$(echo "$list" | sed -n "s/false \(.*\)/\1/p")
echo -e "\nAuto restart in the pool is $pool\n\n"
if [ "$active" != "" ]; then
  echo -e "*** VIRTUAL MACHINES AUTO RESTARTING ***************\n\n$active\n\n"
fi
if [ "$inactive" != "" ]; then
  echo -e "*** VIRTUAL MACHINES _NOT_ AUTO RESTARTING *********\n\n$inactive\n\n"
fi

This script will parse all the VMs while skipping a few system instances, and will clearly show which ones are starting automatically, together with the corresponding UUID, so no guesswork or matching is needed.

# ./autostart_status.sh

Auto restart in the pool is ACTIVE


*** VIRTUAL MACHINES AUTO RESTARTING ***************

VM1 [UUID1]


*** VIRTUAL MACHINES _NOT_ AUTO RESTARTING *********

VM2 [UUID2]
VM3 [UUID3]

With our UUID in hand, we can now enable/disable it through this tiny script:

#!/bin/bash
if [ "$2" == "true" -o "$2" == "false" ]; then
        xe vm-param-set other-config:auto_poweron=$2 uuid=$1
else
        echo Correct syntax: $0 \ \
fi

With these two scripts the task becomes checking the VM name and switching it on/off in a matter of seconds. Mission complete.

Mass XenServer updates with batch script

I was looking for a way to optimize (read: not doing repetitive tasks by hand) the patches upload on my XenServer machine. I based it off a script, shown in this article, that I modified slightly for my lesser needs and built it around a Win7 XenCenter installation.

@echo off
set XE="C:\Program Files (x86)\Citrix\XenCenter\xe.exe"
set Patch=<\\FILESERVER\PathToUpdates\|DRIVE:\LocalPathToUpdates\>
set Server=-s  -u  -pw 
set HostID=

if not "%1"=="" goto :Patch

del "%Patch%*.bz2" >nul 2>&1
for %%f in ("%Patch%*.zip") do %XE% %Server% patch-upload file-name="%Patch%%%~nf.xsupdate"
echo -- Run the command again passing the IDs as parameters
goto :End

:Patch
echo -- Now installing: %1
%XE% %Server% patch-apply host-uuid=%HostID% uuid=%1
shift
if not "%1"=="" goto :Patch

:End

The first four parameters are shortcuts to the xe.exe, the path to the patches location, the remote server data and finally the UUID of the XS host. I download the packages in the directory and unzipped its content. When the script is ran without parameters it will remove any sources package I might have mistakenly extracted, it scans the path for any and all ZIP packages and, based on their names, uploads the extracted XSUPDATE to the XenServer host, returning a list of patch UUIDs. After that I can launch the script again passing a parameter list with the returned patch UUIDs, and it will cycle through them all and apply them. After they have been applied correctly I can reboot the XenServer host and delete the ZIP packages.

This is a bit rough around the edges, but it works when you only have a handful machines to upgrade. There is a lot of room for improvement though, and I might get back to it at a later date.