How Virtualization Will Improve Your Code

Good testing will result in better code. If you have to wait endlessly for on SVN commits, uploads or compile steps, you will simply produce less inventive code. This has to do with: patience, creativity flow, will, and of course time. Constantly being interrupted breaks concentration. If there's one thing I've really learned, it's invest in a good testing environment. Rapid review of code results will pay off (I promise).

So it's OK to spend some time on learning a good IDE, and another trick to improve the speed & quality of development, is to virtualize your production platform to your local workstation. The fake-production server (virtual machine) will mount your code directory directly as it's webroot, and so it can serve your IDE-work instantly.

Basic Idea

Check out this picture

You can:

save your code in your IDE
switch to your 'fake production server', running as a virtual machine
instantly view results, as if it was served by your real server

Some advantages

no delays
no touching of production environment
clone machine and expirment with different PHP / module versions
shut down virtual machine when done programming

About the Article

This setup took me a day. Writing the article added some time, but should enable you to do it in a couple of hours.

I'm setting this up on an Ubuntu workstation. But since the actual work is done inside a virtual machine, this should work on every OS (Windows, Mac, FreeBSD, everything). For support on setting up VirtualBox on those operating systems, just checkout the VirtualBox site & forum.

System administration skills are required. You could mess up your system. I warned you.

Ubuntu on Ubuntu?

You may have noticed I already run Ubuntu. So why not directly run apache on top of that? Why install another Virtual Ubuntu?. Simple. Because I want to:

Experiment with multiple virtual development machines, maybe running different versions
Not pollute my workstation with server software
To be able to turn all server software off, by shutting down the virtual machine, freeing up resources for, let's say, streaming HD content to my PS3.
Share my virtual development machine image with colleagues (once it's perfect). So they all can have the joy of directly seeing the results of their code, without being able to cause any damage to existing systems.

Ok, let's get on with it!

Install VirtualBox

In the process of writing this article, Sun has acquired VirtualBox and improved the installation procedure. Hence, checkout: dlc.sun.com/virtualbox/vboxdownload.html

Download Operating System for Virtual Machine

My production server uses Ubuntu (Server edition) as an OS. So my virtual development server should be Ubuntu as well. I'm going for the Desktop edition however. Further on, you'll learn that installing the desktop version has some advantages in our case, and Apache & others won't care about the fact that we have a different kernel and some extra GUI packages installed.

So download Ubuntu Desktop like this:

$ mkdir -p ~/ISO
$ curl https://ftp.snt.utwente.nl/pub/linux/ubuntu-releases/hardy/ubuntu-8.04-desktop-i386.iso -ko ~/ISO/ubuntu-8.04-desktop-i386.iso

Or however you see fit.

Creating a New Virtual Machine

First of all, start VirtualBox OSE. It's under Applications->System Tools

Click New, Next

->VM Name and OS Type

Come up with a name for your machine

Choose Linux 2.6

->Base Memory Size

Choose 512MB (depending on your host's configuration)

->Virtual Hard Disk

Click New..., Next

->Virtual Disk Image Type

Choose Dynamically expanding image, Next

->Virtual Disk Location and Size

Come up with a name for your disk

Choose 12GB (depending on your host's configuration)

Click Finish

->Virtual Hard Disk

Click Next, Finish

Configure the New Virtual Machine

Click on your machine, Settings

->In General, Advanced

Enable VT-x/AMD-V if your host supports it.

->In CD/DVD-ROM

Mount CD/DVD Drive

ISO Image File, Select

->Virtual Disk Manager

Click Add

Select ~/ISO/ubuntu-8.04-desktop-i386.iso

->In Shared Folders

Click Add, Select

Select ~/workspace/project/

Click OK

Start the new Virtual Machine

The Ubuntu installation will show up.

Install Operating System

Install Ubuntu like you would normally, this is a next, next, next alike procedure.

Reboot eventually, and Login

Drivers on the New Virtual Machine

Click Devices, Install Guest Additions, Yes, Download, Mount

Open a terminal, type

$ sudo /media/cdrom/VBOXLinuxAdditions.run
$ sudo reboot

Upgrade Operating System

First let's unlock all of Ubuntu's wonderful extra repositories.

Open a terminal, copy & paste:

$ sudo sed -i "/^# deb.*multiverse/ s/^# //" /etc/apt/sources.list
$ sudo sed -i "/^# deb.*universe/ s/^# //" /etc/apt/sources.list

And now let's upgrade, type:

$ sudo aptitude -y update \
 && sudo aptitude -y dist-upgrade \
 && sudo aptitude -y install subversion msttcorefonts \
 && sudo reboot

Wait a bit for Ubuntu to get up to speed. It's possible that Ubuntu finds a new kernel. In such a case, you may have to reinstall the VirtualBox Drivers (Devices->Install Guest Additions).

Make It a Server

We should probably stick as close as possible to your production server's configuration. I'm going to assume it's a LAMP server, so let's set that up first.

Open a terminal, type

$ sudo aptitude -y phpmyadmin mysql-server
$ sudo a2enmod rewrite

phpmyadmin has a lot of dependencies so that it automatically installs apache2, php5, etc.

You probably want mod_rewrite enabled as well.

Does it work?

Well let's open up a browser and find out. Goto https://localhost.

Cool, apache is serving up a page nicely. You can also check out https://localhost/phpmyadmin/

To confirm that MySQL & PHP are working as well.

But this is all very default. Not quite similar to your production server environment yet.

Make It Your Server

These links will help you copy data from your production server to your virtualized server:

Restore packages using dselect-upgrade (for your APT & PEAR packages)
Synchronize files with rsync (maybe copy config files or content)
Tranfer all MySQL databases to another server (maybe copy your database)

But know that there is no generic way to make it your server. Only you know what config files are important for your server. So let me show you how I made it my server, then.

Make It My Server

Going Root

Because it's me, I'm done sudoing. It's a virtual server and the risks are minimal... I'm going root.

$ sudo passwd # (enter password 3 times)
$ su -

Get Code

I need the project's code. But - and this is very important - I don't want a separate checkout from SVN. I want the fresh code that's in my workspace folder on my workstation. This way, I can just save my code, and my virtual server can instantly show me the changes. Without having to commit, and without harming anything else then my local working copy.

We have already shared our project's working copy in step "Configure the new Virtual Machine". In my case it's stored on my workstation in ~/workspace/project/, and so the share is called 'project'.

To make it available in the 'production' directory of the virtual machine, I type the following:

$ mkdir -p /var/www/project
$ mount -t vboxsf -o uid=33,gid=33 project /var/www/project
$ ls -al /var/www/project

Because /var/www/project is the same directory that the production server uses, you shouldn't get any 'require_once failed to open stream: No such file or directory' errors.

Get Environment

Of course I need to tell Apache to serve the code in /var/www/project when it sees a request for your.projects.domain.com. That's done with VHosts, and I'm just taking the production server's VHost as an example (the closer the resemblance, the better, right?).

$ scp my.production.server.com:/etc/apache2/sites-available/project /etc/apache2/sites-available/

$ a2ensite project
$ /etc/init.d/apache2 restart

Fixing Apache Errors

What does apache say? Now is the time to resolve any missing paths, config files, or modules you may encounter. This is where you have to do some actual work!

In my case, I needed to fix the following:

Add 443 to ports.conf
Enable SSL with a2enmod ssl
Generate SSL certificates
Change some references from /etc/apache2/ssl/apache.pem to /etc/apache2/ssl (had to do with switching from Debian to Ubuntu)
Change all external IP references to 127.0.0.1 in my VHost
Install memcache module, aptitude install php5-memcache
Include memcache module add extension=memcache.so to php.ini
Allow requests from Virtual Machine, by adding an Allow from 127.0.0.1 directive to my .htaccess

OK that fixed all of my initial errors.

Get Database?

You could go two ways:

Allow your outgoing IP access to your real production server and work with live data (easy, but insecure)
Copy your production database to your virtual development server

In my case, I chose option 1: to let the Virtual instance connect to the real production database, so I could test with live data, and didn't have to copy the entire database to my instance.

Whether this is wise or not, depends on your situation & skill.

But if you want a real & separate test environment, you might also want a copy of your database.

Transfer your live data to your virtual machine's MySQL server
Point the database hostname to 127.0.0.1, by including a line in your /etc/hosts file, like: 127.0.0.1 database.your.projects.domain.com

Now, all connections from PHP made to database.your.projects.domain.com, will end up pointed to your localhost, where a local MySQL instance is running.

The same goes for memcached servers, and all other types of servers for that matter.

Testing Your Project

Now. We could set up bridged network interfaces (tap) that allows the virtual machine to claim it's own IP. This way you can use your workstation's browser to surf to that IP and test your code. We'd have to change your workstation's /etc/hosts file to have your.projects.domain.com point to the IP of the virtual machine (let's say 10.0.0.2)

But setting up tap interfaces is a drag, and changing your hosts file every time is just as well.

So why not test within the virtual machine? It's a desktop anyway :) And you only have to change it's hosts file once. You can test your workspace's code within your virtual instance, and you keep your workstation clean for showing the real thing. This means you don't have to modify your own workstation except for installing VirtualBox. This could also open up opportunities to share your virtual machines image with colleagues. An entire development & testing environment in 1 file. That's quite awesome.

Change Hosts File

In order for everything to work properly, we need to point your.projects.domain.com to 127.0.0.1, this way our local apache is contacted, even though we request your.projects.domain.com. And requesting the domain name is a must, to ensure you internal code is treated in the same way, and that apache knows which VHost to serve.

$ $EDITOR /etc/hosts

And add your.projects.domain.com after 127.0.0.1 localhost, make the head look like this:

127.0.0.1       localhost your.projects.domain.com
127.0.1.1       hulk

Browse Away!

So?! What are you waiting for? Fire up firefox and enter https://your.projects.domain.com. What do you see?

Fixing Project Errors

Well I saw some errors. Some includes could not be found.

File Errors

When I commit to our production server, some files are copied to the document root that are not in my working copy. This is done by a deploy script tied to the post-commit hook.

To fix this I added a line to my virtual host:

php_admin_value include_path .:/usr/share/php:/usr/share/pear:/var/www/project/_common

So it would also look in the _common folder. Other solutions may be:

Change some code in your project, making it aware that it's running in development mode when it's hosted on 127.0.0.1, thus pulling files from some other location.
Make the dependencies available on your virtual machine
Let rsync run in daemon mode and pull files on changes
Creating a local deploy script (could be a lot faster than te remote one, but try to stay away from 'compile' steps, we're doing this for agility & speed!)

Performance

More cores help ; )

Improve This Article

This was quite a long post, so if you spot errors or know better ways, help me improve this article by leaving a comment.