I endeavour to spend a bit of time over the Christmas/New Year break trying out and learning about new things as these opportunities don’t present themselves much through the year. On Friday night over dinner, a good friend Russell Robinson mentioned a virtualization product XEN, which was being put to very good use where he does some contracting work. I’d heard of XEN but didn’t know anything about it. I still have Cherryl’s new PC that I can play around with, so this seemed like a good time to check XEN out.
Computer virtualization enables you to run a guest operating system from within another host operating system. Lets say you are running Windows XP and would like to run Linux at the same time, well that’s where virtualization comes into play.
VMWare Workstation is one of the best known and most widely used virtualization products. I purchased a copy about 3 years back when I finally owned a PC powerful enough to run it and wish I’d started using it years before. I use it for software testing as you can start with a cleanly installed version of Windows or one with lots of programs installed and test against known environments and base lines. I run versions of Windows 98, 2000 and XP under VMWare.
Besides testing the programs I develop, I use VMWare for testing software I download from the Internet. This lets me run these programs in a safe secure sandboxed environment with no way any malicious program can touch my real host operating system, its disk drives, registry etc. As far as I’m concerned a sandboxed environment or one where you can roll back the clock, are the only ways to try software you download from the Internet, unless that is, you know it is from a reputable company. The prime reason PC’s become such a mess is because of all the programs people try and then uninstall, but bits and pieces are frequently left behind or worse.
VMWare and similar programs like Microsoft’s free Virtual PC are what’s called virtual machine emulators which add a layer that hides the physical PC’s hardware and present a new isolated set of virtual hardware. Emulation and virtualization is done in software and requires a fair chunk of processing power. This means you need a reasonably powerful PC with plenty of memory. The downside of all this is that the guest operating systems run a quite a bit slower than if they were running directly on the same PC and they also have restricted resources available to them. Nevertheless the advantages of these products clearly outweigh their disadvantages.
New hardware virtualization technologies introduced by Intel and AMD dramatically change this picture and enable guest operating systems to run at something a lot closer to full speed. This is where XEN fits in. I have to say I found the XEN Web site disappointing and confusing. It seems like they expect everyone to know what XEN does before they visit the site. I couldn’t find any basic overview material that gave me the on-high picture of XEN, how it works and how you use it. I just kept reading how it will give me near bare metal performance!
Fortunately they have the XEN Express edition free starter pack. I downloaded this, burnt the ISO image to a CD and went from there. For some reason I had trouble though as XEN couldn’t locate or initialize my Ethernet Adapter. I tried several times to no avail. XEN Express is built around a modified Linux distribution. The ASUS motherboard I’m using on this particular PC has 1G LAN on-board which uses an nVidia nForce 430 chip. Under WinXP you need to install drives before the LAN port is recognized, so I could understand why Linux might have trouble. I temporarily gave up and played with installing some other OS’s on this new PC, including versions of Linux and MS Vista and they were all able to initialize the Ethernet Adapter (eth0). So I gave XEN another shot and this time it worked!
XEN Server is very a different product to VMWare Workstation and its ilk and is designed for a different set of users. You install XEN Server on a dedicated computer, preferably one with lots of grunt, memory and hard disk space, typically a server. You then use a different PC to access and administer the XEN Server. This latter PC can be most any PC with a network connection. You install the XenSource Administrator Console from the same CD as the Server, start the Admin Console and enter the IP address of the XEN Server (displayed on the server screen) and your login password (set when you installed the server). Once the connection to the server is established a Management window is displayed. From here you can Install a new XEN VM (Guest OS), Power On (start) a VM, Reboot and Shutdown a VM and more.
Lets see a few screen shots from the Administrator Console program.
This shows the Overview information for the XenServer. It includes full details of the host computer, its network adapters, amount of free disk space and how much memory is allocated to each of the installed XenVM’s. From here you can Install a new XenVM (Operating System), Reboot or Shutdown the Server, access its Console, view Performance graphs, and manage installed XenVM’s.
Â
This shot shows the Overview of a XenVM guest operating system, specifically Windows XP. From here you can change the amount of memory allocated to the VM, alter the virtual disk size, add or remove disks and Ethernet adapters and change the CD-ROM Drive. You can also Reboot or Shutdown the OS if it is running or Power On, Clone, Export or Uninstall the VM if it is Off. Note that to change memory or drive size etc. a VM must be Off.
I was impressed with the ability to add more virtual disks and resize them. There was a bit of mucking around in WinXP to recognize the new drive, but it wasn’t too bad. Disk drives in a VM scenario aren’t physical hard disk drives, but a chunks of space on a physical drive. As I mentioned earlier XenServer uses Linux which includes LVM or Logical Volume Management which provides the capabilities to create and resize these virtual disks.
Â
The next screen shot shows the Graphical Console for the Windows XP XenVM shown above. The Graphical Console uses VNC to view and interact with the operating systems running on the XenServer. As shown here you are only seeing a part of the XenVM operating systems screen and you need to vertically and possibly horizontally scroll to work with all parts of the screen. To make this easier you can Undock the VNC Graphical Console (see below) so it runs in a Window of its own, which should be the same size as the actual OS Screen.
If you’ve ever used VNC you’ll know it is ok but not great, in that screen redraws are a bit sluggish making the user experience less than ideal. You have to keep in mind though that this is a management console, and isn’t really something you would sit at and use as the real screen for the OS. VMWare Workstation is better in this regard as everything runs on the one computer and the guest OS screen appears in a Window with nowhere near the lag of VNC.
The real user of an XenVM would instead use Windows Remote Desktop if the guest OS was Windows and for Linux you would use WinSCP, PuTTY and the like to get Shell access or VNC for GUI access. Or more likely just access it via. a Web Browser when the OS’s prime purpose is a Web Server.
This image shows a WinXP XenVM in an Undocked Graphical Console window. It would be very useful to be able to run the Administration Console on the XenServer itself, however it doesn’t seem that this is possible.
Â
The next screen shot shows Debian Linux in the Graphical Console with a Web Browser showing our Surfulater.com Web site. You can also see the CPU % for each running XenVM and for Linux VM’s other information, like how much memory is being used, and disk and network throughput. It is a pity this information isn’t shown for Windows OS’s. Update: Installing the Xen WinXP Drivers resolved this.
Â
This last screen shows the XenServer Performance screen, which should speak for itself. You can also see similar pages for each running XenVM, but for Windows OS’s you only see CPU Usage as mentioned above.
Â
So who would use XenServer and why. First up for my use in software quality assurance and testing, VMWare Workstation is far better suited. However if I wanted to set up a box in the corner running Web Servers, Mail Servers and the like and I wanted multiple instances of Linux running, it would be ideal.
Another possible use would be a SOHO where you could effectively share a single server computer amongst all staff/family members, each with their own OS, Disk Space etc. I have to say this use is questionable though.
Web Hosting companies would certainly be able to make good use of XenServer as would Enterprise IT and these seem to be the markets Xen is aimed at.
Note that XenVM’s are currently limited to Windows XP and Windows 2003 Server, and RedHat and Debian Linux, so if you want to use anything else you are out of luck.
XenSource keep pushing near bare metal performance and while it is indeed very good and very impressive, it isn’t “near bare metal”. These are some WinXP benchmark results, with it installed directly on the PC, versus XenServer on the same PC with WinXP installed as an XenVM.
Drystone ALUÂ 14,538 MIPS Native XP, 7,048 MIPS XP on XenVM
MM Integer 36,895 it/s Native XP, 17,800 it/s XP on XenVM
Disk Burst Speed 226.1MB/s Native XP, 142.2MB/s on XenVM
Disk Average Read Speed 63.2MB/s Native XP, 9.7MB/s on XenVM. This was fixed once the Xen XP Driver Installed.
* See my Comment 8 for an update on these performance issues.
CPU benchmarks were down around 50% of native performance and Disk speed down around 15%. Three XenVM’s were running with two sitting idle. The drop in disk performance is significant and would impact heavily on disk bound processes. The PC has an AMD Athlon 64 X2 3800+ CPU, 1G DDR2 RAM and 2 x 320G SATA II Disk Drives.
Overall XenServer worked very well indeed. I did run into a problem where the Administration Console kept disconnecting from the server and restarting the Console program didn’t resolve this. In the end I had to hit the reset button on the server as I couldn’t restart it from the console. This resolved the console disconnection problem, but was a drastic measure in the extreme.
I also had problems with the XEN Servers CD-ROM drive where I wasn’t able to eject the drive so I could put in the next CD for a Windows 2003 Server installation. The only way I found to do this was to shut-down the XenVM. Same thing with Windows XP. The CD-ROM drive also couldn’t be written to, even though it is a burner. The Administration Console is a bit annoying in places as it continually interupts your work flow with confirmation prompts. It would be far better if these “Are you sure?” prompts included a “Do not ask again” option like I use in Surfulater.
XenServer has only recently been released, so you might want to sit back for a little while before using it in a production environment.
The addition of hardware virtualization to CPU’s is a leap forward for running virtualized operating systems at far better performance than software virtualization. I get the feeling we are just at the beginning with this new technology and we will see lots of new and interesting developments over the next few years.
Thanks for the rundown. Just curious about licenses. How do you go with Microsoft and the activation of Windows and Office? I presume each Windows VM needs its own licence.
Great writeup. I’ve been playing with XenExpress a bit lately and its pretty cool. The reason you don’t see the performance information is you need to install the add-on Windows drivers. That is also why you are seeing slow performance on Windows. The drivers speed things up a lot! They include a CD image with the driver install. Look under the CD-ROM dropdown.
Neville —
Thanks for your comments.
The obligatory blah-blah on benchmarks, of course, is that all virtualization solutions do some things great and others less so — I’m sure I can identify benchmarks on which other virtualization tools are well below native performance as well.
That said, our feelings about XenServer products are that:
– user-mode-CPU-bound benchmarks absolutely should be within a small margin of native performance, and if we are not, we will investigate that — our testing with SPECjbb, for instance, puts us there
– memory-bound benchmarks must be tested on like memory configurations — for example, the point of comparison for a 1GB server running one copy of a benchmark is not the same server running it in a 512MB or 768MB virtual machine, but a >1GB system running it in a 1GB virtual machine
– most network and disk workload-style benchmarks should perform at close to native performance on like configurations — again, buffer space requires that you compare a 1GB system to a 1GB virtual machine, not to a 1GB server running a few virtual machines, most of which is idle — most disk and network benchmarks are heavily dependent on memory configuration — are you comparing results of a 256MB virtual machine with the same box, or a 256MB physical configuration?
– disk and network micro-benchmarks won’t look very good on XenServer, any Xen packaging, or any other virtual machine technology — the underpinnings of the I/O simply take longer paths, even when paravirtualized, but in workload benchmarks the impact is covered up in the mix — as it would be on real workloads
– performance for XenServer guests is measured only with accelerated (paravirtualized) I/O drivers — we *expect* performance on the default emulation drivers not to be comparable or even acceptable, and that’s an acceptable design decision — we should probably be stronger in our documentation that the supported and measured guest configuration requires installation of the accelerated drivers and guest agent
The last point stands out in your results: the lack of a Xen circle-X logo next to your guests indicates that you have not installed them. (Yes, I know that the lack of something is not a useful way to deliver information if you don’t know what you’re looking for — we will address that in a forthcoming release.)
You have my email address — we would be happy to review your configurations and benchmark results, try to duplicate them, and address or explain them.
We would, of course, appreciate your clarifying your configuration information and being sure you’re comparing like configurations and properly installed guests.
Thanks again.
One more thing: how many threads do the compute-bound benchmarks run? And is the system single or dual-CPU? Since Windows virtual machines can only use one virtual CPU — which will be addressed in near-future releases — I’d expect to see bout half the CPU performance of a multi-threaded benchmark when comparing a dual-processor box with a windows XenVM.
David, yes for XP and earlier I think you are right. I understand that one of the Vista versions (probably Ultimate) enables you to install multiple copies of a single license on a VM.
For Developers etc. the best way to acquire multiple MS product licenses is to sign up for their partner program and then purchase a Microsoft Action Pack Subscription. This includes XP, Server 2003, Vista, Office etc.etc.
Jim, glad you liked the article and thanks for the info on the add-on Windows Drivers. I didn’t and can’t now see any reference to these.
I’m just reinstalling XenServer from scratch as I’d overwritten it with other OS’s etc. I’ll report back with my findings.
See http://www.xensource.com/support/3.1/xenserver-userguide.html#creatingVMs-installingwindowsvm
– step 12.
This is the section called “Installing a Windows XenVM” — it is also in the user guide on the CD as well as in the administrator console help system.
That step says:
#
Before using this XenVM, you should install the paravirtualized device drivers. While not strictly required, performance is far better with these drivers installed, and it is the only supported configuration. These are installed from an ISO available to the virtual CD-ROM drive.
Select the Overview tab and click on the Disk row of the CD-ROM drive. Select xswindrivers.iso.
Click back on the Graphical Console tab. The driver installation program should start by itself. If not, browse the CD-ROM disk in the Windows Explorer and double-click on the file XENSETUP.EXE.
Roger, thanks for the prompt and detailed response. I’ve reinstalled XenServer with Win XP Pro and Debian Sarge VM’s and have nothing but good news.
Installing the WinXP Driver as Jim and yourself suggested brought the Disk Benchmark up to the same numbers as the native WinXp install benchmarks, which is a pretty impressive feat.
The CPU is an AMD Athlon 64 X2 3800+ as I mentioned in the article. This is a Dual Core CPU, but as you said the Windows XenVM only uses one virtual CPU, so this explains why the CPU Benchmarks are all at around half the numbers for the native WinXP install.
I’m just playing with Remote Desktop and running various apps to get a better feel for the actual user experience.
Can I install FreeBSD on my XenServer?
In the early releases we’ve focused on specific installer options for specific guest releases (hardware-assisted for Windows, P2V for some RHEL and SLES flavors, vendor installer for some RHEL flavors, template for Debian Sarge), and haven’t really provided a generalized installer for either paravirtualized or hardware-assisted guests. We’re planning on doing some work on that in the next couple of releases.
Until then, some folks have reported success installing other things in hardware-assisted guests by pretending they’re Windows — but you won’t get the paravirtualized drivers or the agent, so poor I/O and no stats will result.
Others have loaded paravirtualized guests by installing a supported paravirtualized guest, then block-copying the unsupported one into its disk image and fixing the configuration file manually.
As you might guess, neither of these is a supported road 🙂
Roger, thanks for the reference to “Installing a Windows XenVM†Step 12. I had actually read that in the PDF User Guide a day or so before I did the install and unfortunately didn’t read the Help during the install.
Might I suggest the text be reworded to more strongly indicate it is installed and to also make this more visible and obvious in the Admin Console. Something like the VMWare Tools setup where it is on the “VM” Menu would be good.
I was just discussing this with the team working on the UI this morning, in fact.
I was wondering if I could make a request of you — a little pushy, I know. Since your post has been search-engined and RSS’d in a way that leaves the no-PV-drivers benchmark stuff visible but doesn’t always show the comments, would you consider adding an “edit:” box in the post itself with the PV-driver results (for I/O) and the explanation of CPU?
Roger I’ve already edited the article to mention PV Drivers and the revised performance.
Not sure what you mean by an ‘edit box’ but if there are other clarifications you’d like made to the article let me know.
Your edits are cool — thanks.
very interesting read..
Hi,
Thanks for the very detailed write up. I am curious as to how you managed to ge XEN to recognise your Lan card which you initially had trouble getting XEN to recognise.
Do you get the XEN P2V tools on XEN express ? Have you tried them to import any windows PCs / Servers ?
I have treid the Free Vmware product and it works like charm importing and starting a full server in my VMware WS5.5 environment.
I would be grateful if you could let me know your experience on the above questions please.
Thanks & Best Regards
Nalin.
Hi Nalin,
I didn’t do anything to get XenServer to recognize the on-board LAN adapter, it simply started working on its own, as I mentioned in the article.
XenServer is built on Linux, so that’s what needs to recognize all the hardware.
I can’t answer the XEN P2V tools question as I didn’t try them and no longer have Xen installed. I suggest you post on their support forums.
Roger,
thank you for your well written and detailed report on Xen Server. I am still confused about Xensources’ statement on their website (“Which Xen is right for me?”) about the lack of support for Linux guests in Xen Server. Yet you mention having installed Debian and Windows XP at the same time.
We just bought a new company server Dell PE 2950 with 8 GB Ram and would like to install Debian Etch AMD64 as a development platform for web applications and Windows 2003 TS for our office stuff (the goal is to shutdown the whole bunch of old servers in our basement to dramatically reduce total power consumption).
Ideal it would be to have the Debian Linux paravirtualized in DomU and the Windows in HVM mode with benefit of the closed source PV drivers.
I did all this for testing purposes with the open source version but now I am stuck and surprised about the bad windows I/O performance because of lack of PV Xen drivers (I feel this little but important detail can not be stressed enough, just out of fairness to the vmware guys who deliver their free vmware server with the necessary windows drivers in vmware tools)
Can Xen Server run Linux and Windows guests simultanously or do you think I need the Xen Enterprise Version for that purpose?
Thanks and best regards
Matthias
Hi Mathias,
Once I installed the XP PV Drivers the performance issues I had went away, as I replied in my comment 8 above.
I did have Debian and WinXP running ok but I didn’t do any serious stress or application testing, so I can’t coment on how well this works in practice.
You might be better better running Windows Server on one PC and Linux on another though.
Best to ask the Xen folks which version is best for your needs.
Thanks for your reply.
Great article, I’ve just started investigating Xen for inhouse use.
From the Xensource site it says that for Windows platforms you need Intel-VT or AMD-V enabled processors, your CPU specs were a AMD Athlon 64 X2 3800+, I could not find any reference on AMDs website to see if it is an AMD-V enabled processor, the only AMD-VT processors I could find are the Optitron series
How did you manage to get windows to install/run on your CPU?
Thanks again…
Stephen,
I can’t recall having to do anything special here. There may have been a BIOS setting though.
I’m pretty sure the Athlon 64 X2’s have the VT capability.
Glad you liked the article.
I installed XenExpress on VT Enabled AM2 Athlon1218 dualcore, 4GB RAM (DDR2-800×4), PCI-E Escalade 1.2TB RAID6 (6x320GB SATA2 Seagate HDD), 3 1000T(eth), DVD/CD burner: The xe install went well. The Xen Center provides the .vxa for Windows VM templates, but I didn’t have the xen express linux.iso downloaded yet, so I wasn’t sure about the lack of Linux templates.
Theres an Etch 4.0.xgt available by someone, but the Xen Center VM Import asks for a virtual appliance image. vxa – All the information I found for creating Linux VM in the Xen Express, shows the Xen Center with different Distros as templates. I only have Windows templates available so far…
VM1= Windows 2003 SBS, Exchange,… and more. (2GB RAM).
The Win2k3 Server runs great, at half the memory resources.
I would like to install the latest Linux which supports PV, on another VM with at least 1GB ram, possibly 1.6GB. Since I’ve used Debian the most (on a remote server with Sarge Backports Xen3.03 using lvm2, debootstrap, xm create, apt-get, (Debian Etch or Ubutnu Server)… this is great for our Internet colocation server, but I’m not sure which approach to take for installing linux on this new PV hardware optimized local server. The remote server is Debain perfect setup, no local console though… it’s running the latest Etch 100% stable in VM on Sarge Backports with Hyperthreading enabled, no issues for over a year and constant usage averaging 20% and peaking often without hickup. It seems that there’s a lot of people having trouble installing linux VM’s, templates, images, and I’m totally confused for how to use XenServer4 compared to the Xen3. The xm commands are like the xe commands.
The new local server is Xen Express… how can you install an image without debootstrap, and no ability to use the “install VM from CD” template for ‘New VM’ in Xen Center.
I guess I’ll have to keep searching for a XenServer Debian Linux Server virtual appliance.
Although it would be nice to take lvm snapshot from the remote server linux and create an lvm on Xen Express… I’m clueless how exactly I can load a quality hardware supported Linux VM on this xen express server. I guess try xe create-image on the new linux partitions? I have no idea exactly…
I’m confused about what the XenServer4 wants me to do for adding virtual appliance templates. That’s not what the .xgt templates are,… new to me, but reminds me of the .cfg file used with the xm create in Xen3.
What is XenExpress doing for managing Linux VM creation and backup?