[Update: 19 June 2009] Reports are coming out that Oracle is terminating Virtual Iron as a product, which changes things somewhat. How this affects the potential changes to the virtualisation market (especially for large enterprises) time will tell.

So here we are half way through 2009, virtualisation (virtualization) and cloud computing are hot, there are new services, new blogs, new companies popping up everyday and the hype is intolerable! So we thought we’d take a look at the state of play and summarise where we see automation, Virtualisation and Cloud.

Virtualisation:

There are two big names, and some close seconds. VMware continues to be the big name in virtualisation, despite the competition breathing down it’s back and big changes internally within management etc. Xen is the other big name to consider. Now part of Citrix Xen has been putting pressure on VMware for sometime and the battle is working out pretty good for the users really. You can now get a free hypervisor from either party and get started right away. Hardware assisted virtualisation and general advances are lessening the performance issues, so virtualisation becomes a more and more sensible business decision for any size business.

The number twos are a quite large group. Virtual Iron, VirtualBox, KVM and the fabled Hyper-V from Microsoft are all broadening the offerings to consider. Virtual Iron is now part of Oracle, VirtualBox is Sun… now Oracle. So expect more pressure from Oracle in the future. Especially if they can bring the virtualisation strands together (which they will/are). As a BIG name in enterprise computing anything Oracle offer should be considered pretty carefully.

For a smaller organisation Xen perhaps is still the first port of call, followed by VMware as you grow. That said, ESXi makes the case for VMware from the start quite compelling. What Oracle will bring to the table will be interesting and of course Microsoft’s Hyper-V as always threatens to change the landscape completely through pure ubiquity of Microsoft products in IT. VirtualBox (now in Oracle’s portfolio) is a bit of a dark horse, especially in terms of Competing with VMware’s Workstation product.

Automation:

There is one king here, Puppet. It is probably the most common automation tool in use in the Linux world. On the Windows side, life is more complicated and diverse.

In terms of automating server infrastructure, the landscape is quite barren. Cassatt are… well they have been acquired by CA, will this be them saved or euthanized time will tell. OpenQRM is progressing very well as an open source project after the commercial business that spawned it passed away. And of course Puppet is here also.

VMware are making inroads into this area and we watch with interest to see when/if they move beyond managing VMware to take on physical machines and other hypervisors in earnest. It could be a compelling move on their part. There partnership with Cassatt seemed to spur developments in this area, but now Cassatt is… well, the question is does VMware see a business in managing physical machines and other hypervisors?

Cloud

Cloud computing has grown on the hype level exponentially, and actual use and services grow also, though not at the same rate as the excitement surrounding the subject. Amazon are the clear leaders here, offering storage, computing, database, queues, map reduce and a content delivery network. It is arguably the most mature platform and probably the largest.

But the competition is growing and finding niches and unique features that allow them to compete with Amazon. Google’s AppEngine continues to attract Python developers, and the rumours continue about support for other languages and new features. Also more people are making their services applicable in a cloud environment or building their entire business on other peoples hardware/services.

We for example provide our cloudbackup product using more than one storage provider. Which also raises the subject of localisation. You are now able to base your cloud infrastructure in a region of your own choosing rather than just the USA. People like Rackspace and Amazon have expanded their services to allow you to keep your infrastructure in Europe for example. But there are also local providers who fill a niche by being (in our case) in the UK, so any ambiguity about moving data outside the UK and the DPA is lessened, though the issue of hosting your data on another companies hardware still remains.

Summary:

All in all, the hype exceeds the practical substantially, which is a shame.

Virtualisation, Automation and the cloud are useful tools that any size business can and should in our opinion be considering. Everytime your business encounters a physical restriction it is time to consider one of these methods. If your servers disk drive is getting full, can you use a cloud based storage service (like our cloud backup software) to create a new drive with unlimited capacity?  Need a new server, then consider creating a virtual server. Have an application that is straining a server? Consider cloud based processing power perhaps?

What we are seeing is that the hype in many ways is scaring off the “normal” businesses.

The “sexy” tech startups are leaping onboard and we have the poster children like Animoto basing everything on cloud services, but the more traditional companies are moving more slowly. This is in part their cautious nature perhaps, but also the hype is confusing the issue. Virtualisation, Automation and Cloud make good business sense but if over hyped start to look like “snake oil”.

To sum it all up, everything is basically a little further along than it was a year ago, the landscape is suprisingly stable with Amazon, Google, VMware and Xen and the other big names still being dominant and competing more with one another. Bringing new offerings to a public which is “almost” ready to come onboard.

As we speak to more and more people about our Online Backup to the cloud software solution and service, we are finding answers to the more common questions about what we provide, you will have noticed that by the content of the site recently. We are posting it online to make it easier to understand the benefits of what we offer and where be differ from a tape based central backup solution like Symantec BackupExec for example.

There are basically two ways to backup data files, centrally or at the edge on users machines. There are mixed solutions, but they are less common.

Centralised Backup

The standard deployment of a centralised backup is to have a tape drive (or more commonly now days a tape library/tape robot system) connected to one or more servers (or direct to the SAN). Backups are then run at regular intervals, traditionally overnight. The tape(s) are then taken away to be stored offsite. This is so that should the building be demolished, the tapes you backed all your data onto are not destroyed also.

Centralised backup is an excellent solution for many environments. There is a good reason it is the standard configuration for backup. However, it has limitations and risks that are worth considering. To start with, the data needs to be accessible to the backup software/server/tape drive. So typically a centralized backup solution will backup data stored on servers and not on users machines. Which works well in the traditional office environment and when staff are well trained to avoid putting data on their local machine.

The other big limitation of centralised backup is that it is time consuming to and can create a heavy load on the network and servers. You are often copying every single file on your network to one location and trying to write it to a magnetic tape as fast as the tape drive can handle it. Common issues surrounding this are backups slowing down the servers. Also the backup can take a very long time, many hours even, and start occuring during office hours and affecting staff performance.

Also the traditional model leaves huge (24 hour) windows of risk where data is on disk but not backed up. So if John down in accounting corrupts the excel spreadsheet he worked on all day, you have to restore from last night. If he corrupts the file in the afternoon all his work from the morning is lost. Again, there are solutions out there that can do multiple backups through out the day, but they are both expensive and complicated.

User Based Online Backup to the Cloud.

Our solution is different to the traditional model, we recommend for most users they backup every 5-15 minutes. Yes, as much as every 5 minutes. Here in the office we backup every 15 minutes. We can do this because each one of our backups only needs to send the changes to the data we have made in that 15 minutes period, which is normally pretty small. It means that should a file get corrupted or we just decide we hate what we have written in a document, we can roll back changes by restoring anyone of the backups we take every 15 minutes. I can restore the one from 15 minutes ago, from 30 minutes ago from 3:15 ago, which ever one suits me best.

Also, this restore process is something I do through an easy to use interface on my own machine. It happens immediately and IT do not need to be involved at all. So no need to call the IT guys/girls and ask for a file to be restored from tape, no more waiting for the tape to come in from the offiste storage site (the next day?) and then wait for someone to restore the file for me and eventually let me know its there, just to find it’s the wrong file or the wrong version.

The backups are automatically stored offsite, and so if my laptop or PC dies, the data has already been backed up offsite. So if a herd of elephants stampeds trhough the office destroying all the machines, I know my data is safe.

If a machine is left that can connect to the internet, I can access all my files right away. I can access the web interface from any machine and copy down the files I need and keep working. This might be a office pc, my personal laptop or a computer in an internet cafe or client site. This is all within my control as a user.

There are limitations to consider of course; our service is not good for making system state backups that include the operating system. So it is not good for people looking to be able to restore an entire machine in the event of a serious issue. For that you want to look at the more expensive and complicated solutions, and probably look at disk to disk replication. It is a approach fraught with problems, we find most people who actually have to do this regularly (large organisations) prefer to start from a standard build of all new machines and then install/restore applications and data unique to the user afterwards, it is easier and quicker on the whole.

RPath have posted a fun little video that summarizes what cloud computing is. It is focussed on the external cloud area, but it does give a simple and enjoyable outline of what Cloud Computing is and what some of the benefits are. So please watch the video below and if you are looking at an internal cloud just make the conversion in your head.

They get it right, virtualisation is the engine that drives both internal and external cloud infrastructures. This is where you should look first if you want to benefit from cloud computing (internal or external). Virtual servers are the first step, automating the virtualisation is the second step, simplifying and managing the expenses based on use is the thrid step. All of which can be done internally and externally.

Much of the focus in the “Cloud Computing” universe is to do with integrating external services like Amazon AWS into your company’s way of working. This is fine if you are a start-  on the Internet primarily, but for those businesses in more traditional markets this can be a difficult transition, and often one that makes little sense. Creating an “Internal Cloud” infrastructure however makes sense for many organisations with large numbers of servers, especially in terms of server deployment, management and internal book keeping/expense tracking.

An internal cloud is (at least for the purposes of this article) an infrastructure that is managed via remote access only, meaning via web interfaces and software connections only. When a new server is required by the Sales team to host a new tool, say a CRM system, they request it via internal channels. The provisioning of the server is done via a web interface on the internal IT infrastructure. The size and type of server is specified along with the department (cost centre) for internal billing.

The cloud software then creates a physical or virtual server based on the request and provides details as to how to connect to the server via email. Depending on the situation, this may be all that is needed, if further customization of the server is required IT connect remotely and make these changes. Should a server cease to be needed, it can be removed from service via a web interface and the hardware made available for other purposes.

At the end of each month/quarter IT produce a report (from the cloud software), that breaks down the server costs by department for internal billing.

A cloud based infrastructure automates much of the manual intervention that adds cost to a server farm. It also provides efficiency gains in terms of speed of provisioning and re-allocation of hardware assets. This can result in savings in staff, electricity and cooling for example. It can also be beneficial in the speed at which changes can be dealt with, be they planned or unplanned.

For example, a cloud based infrastructure is able to adjust to a hardware failure automatically. Should a physical server fail, the application based on the server can be restarted on another physical or virtual server by the cloud software itself, without human intervention. Planned deployments can be vastly quicker also as the physical elements of creating a new server are removed (such as cabling power and data leads); also Operating System installations and even application installations can be automated.

But thats not all…

A cloud based infrastructure can be interacted with via software, allowing not only people but other software to create servers as required. You could quite easily tell your website load balancer to start an extra web server if load was too hide, and also shut that extra server down if not needed later. You could integrate into your internal applications hooks to manage the server infrastructure based on a variety of factors. Perhaps you’d like your accounting software to create an extra server to process reports on at month end? Maybe you need to start an extra database server when you start the de-duplication process on your sales database?

The benefits of cloud computing for entirely internal server infrastructures are yet to reach mainstream acceptance, in part because the cloud computing hype is mainly around external services, which although incredibly powerful are not a panacea that will solve many business requirements.

Internal cloud computing is also cheaper and easier in many ways to test and evaluate, a spare server can easily become the hub of a test cloud computing infrastructure. Your IT team can easily start creatinng example implementations and running pilot projects within the company, without incurring costs from external services like Amazon and also being confident that there are no security issues to contend with. Pilot projects can, once proven, easily become production solutions and migration of existing infrastructure to the new internal cloud can be a phased ongoing project.

enVirtua is of course available to help with both pilot projects and large scale migrations. Please contact us on email: sales@envirtua.com or
Phone: 020 7193 8987 to discuss your ideas on how an internal cloud might work in your business situation.

PHP I have heard described as “the working man’s programming language”, or shall we say the working persons language. On the web in particular PHP has become the dominant language, Rasmus Lerdorf’s little collection of hacks is now powering websites all over the place, including this one. However, in the enterprise other languages such as Perl have maintained a stonger presence. In the virtualisation space, VMware has for a long time had a Perl toolkit. This, perhaps, has meant that internal projects have been harder to get started. But this is changing, in this post we’ll look at using PHP with both VMware and Zeus ZXTM.

PHP, is one of the easiest ways of creating a dynamic web page. You can create something quite clever and useful with relatively little effort.

For example, we can talk to a Zeus ZXTM load balancer and identify all the running virtual servers, using only a few lines of PHP:

<?
$conn = new SoapClient( "VirtualServer.wsdl",
array( 'login' => "username",
'password' => "password" ));
$names = $conn->getVirtualServerNames();
$enabled = $conn->getEnabled( $names );
for ($i=0; $i < count( $names ); $i++) {
if ( $enabled[$i] )
print "$names[$i]\n";
}
?>

In PHP we can also connect to a VMware VirtualCenter server and gather information:

< ?php

 require_once("lib/nusoap.php">;
$myconnection = new soapclient(”https://myVCserverIP/sdk”);
$namespace="urn:vim2";
$soapmsg[data]=new soapval('_this','ServiceInstance','ServiceInstance');
$result=$myconnection->call("RetreiveServiceContent",$soapmsg,$namespace);
print_r($result);

?>

Expanding this code further you should be able to learn more about the virtual infrastructure that provides your web servers and monitor the ZXTM server that balances the load between the web servers. This could allow you to build a very simple “control panel” of your infrastructure. This control panel might be useful for providing information to management for example.

A good example like this might be the catalyst to allow you to implement a larger project, perhaps using something like Perl or the ZXTM’s very similar TrafficScript language. Using TrafficScript for example you can already control VMware with the VMPoolControl Java Extension where a simple bit of code like this:

$slm = slm.conforming();
if ( $slm < 80 ) {

java.run("com.zeus.vmware.VMPoolControl", "poweron", "webservice");
}

could easily provision VMware virtual machines to match load.

References:

http://www.run-virtual.com/?page_id=125
http://knowledgehub.zeus.com/code/2005/10/20/list_running_virtual_servers_using_php5
http://knowledgehub.zeus.com/articles/2008/08/29/vmware_vi_and_dynamic_provisioning_with

One of the large changes in IT departments over the past 3-5 years is the use of virtualisation for the building of test servers. In this post we shall describe how this can be enhanced through the use of internal cloud computing technology.

Most businesses test new software, they trial solutions promoted by vendors and test internally developed applications. They also test patches and updates to the operating systems. Increasingly virtual machines running on hyper-visors like VMware or Xen are being used to do this. An Internal Cloud Computing solution can manage this process and minimise the waste in terms of  time and virtual machine sprawl.

A real world example:
A staff member needs a server to play with to test a new web application they have discovered.

Normally this will require that a existing virtual server is cloned or a new VM created from a template. This is a manual task generally still, it takes time and energy and the person who knows about the web application may not know anything about VMware.

An Internal Cloud solution can simplify and speed up this process. The user fills in a simple web based form which says what they need (Operating System, memory, cpus,  etc.), and when they need it for, tomorrow lunchtime through to next Thursday for example.

The Internal Cloud software can then await approval from a manager, or simply automatically provision the requested physical or virtual server at the requested time and automatically email the user the details they need to login to the box.

The virtual server is decommissioned without human intervention, ready for the next person, it does not sit there eating up resources on the ESX server.

The benefits for an Internal Cloud Computing solution for a smaller organisation can be substantial as the automatic provisioning and decommissioning of servers minimises the amount of servers required and also the time and expertise it takes to manage the servers. It also increases the speed at which servers can be built, tested and business decisions made.

The benefits of using virtual servers for this sort of testing are becoming well known, by managing your virtual infrastructure with an Internal Cloud Computing software solution you can further increase the value of the investment you make in virtualisation.

The ability deploy a physical server is also available, so that provisioning a server to test something that needs a hardware peripheral connected can be managed via your internal cloud computing portal also.

As virtualisation continues to spread, managing and automating server provisioning will continue to increase in importance. With technology available today, you can make deploying a new server for a user a completely automated process managed via a simple form on a web page. The benefits of this are various and include increasing the use of virtualisation, speeding testing and development cycles, removing human labour and error, managing VM sprawl and saving costs of course.

Please contact us on 020 7193 8987 if this sounds interesting to you and your business.

Our original involvement in “The Cloud” came from becoming involved with the Cassatt Collage product, now called “Active Response”. Collage was all about shutting down and powering up servers based on demand. The product integrates with VMware and Xen as well as physical servers and is very flexible.

If however you are a pure VMware shop, or have less complex needs, then the new Dynamic Power Management (DPM) functionality that VMware has in the pipeline is probably a good place to start looking.

The basic idea is that a machine that is turned off uses less electricity and needs less air conditioning than a powered on server. So if you can turn servers on and off as required, then you can potentially save your business a considerable amount in electricity and air conditioning costs. Your mileage may vary, but savings in the region of 55% are being quoted.

Below is a video from Cody Bunch in the USA, which shows an example of this all happening, including showing the monitoring of the power usage by servers. It shows a ‘typical‘ work day in a server room, low use before and after hours. In this example we start with a single VMware ESX host, and all the virtual servers running on it. As load increases, new ESX hosts are powered up and virtual machines “VMotion-ed” to the new servers to balance the load. This continues until all four ESX host servers are running. Then at the end of the day, the extra servers are shut down as CPU load drops.

For those of you reading this in a RSS feader the video is available at http://uk.youtube.com/watch?v=7CbRS0GGuNc .
*** Warning, Cody put a rather loud musical soundtrack on this video, so I suggest turning down your speakers before you hit play.

The video is a great visualisation of what an internal cloud infrastructure might need to respond like. Without any human intervention, this example configuration responds to increased load. This is really powerful as it means that as well as saving money in electricity, it is protecting your users from performance issues that might occur if a human had to be part of the process of balancing the load.

Of course, the limitation to DPM (other than being a test feature at this time), is that it supports only part of your infrastructure. Specifically, it only is able to manage VMware virtual machines and ESX host servers. If you have a more complex (some might say normal) infrastructure with a mix of virtualisation platforms (KVM, Xen, etc.) along with physical servers then DPM may not be right for you. Cassatt or OpenQRM might be the answer you are looking for.

The promise of Cloud Computing is often this: when your application servers become overloaded, new servers will start automatically to share the load. This is the vision, the dream, the ideal. However, it is not often demonstrated.

Mark Boddington, one of the system engineers at Zeus has written and excellent article on how their product ZXTM can be used to automatically provision VMware virtual machines. Not only that, he has recorded a great little video to demonstrate the principles in action, which is embedded below:

Link to video on Youtube.

The ZXTM product monitors web applications and using the Service Level Monitoring class (SLM), Trafficscript and the VMPoolControl Java Extension to interact with a VMware ESX server to provision virtual machines when a web application becomes overly loaded.

This demonstration is important to consider in the context of cloud computing and specifically in regard to the “Internal Cloud”, and cloud computing generally. These sorts of methodology can be used as part of the infrastructure required to build your internal and/or external cloud. It is not and end in itself, but rather a tool to be used appropriately when designing your solution.

ZXTM is used primarily to optimise and load balance web applications, an area very close to most discussions about cloud computing. As ZXTM has this focus, it is a very good tool to use if you are dealing with any application that has a web interface of course. With this one tool you can do web optimisation, network load balancing and also manage VMware instances automatically.

This provides an excellent tool, that could be ideal in some “cloud” situations. This solution would probably be coupled with other technologies to handle things like b uilding the ESX servers from “bare metal” and provisioning operating systems and applications stacks onto virtual machines. This does not diminish from what thedemostration is showing us however. The ZXTM is very good at what it does, this example builds on Zeus’ area of expertise to extend the flexibility of their product. If you are a ZXTM user already, contemplating how to build an internal cloud, then this might be a starting point from where you can develop a more sophisticated solution.

Lance

p.s. Mark has also written a traffic script that sends information to Twitter, does your data centre send you messages?

UPDATE: December 17th 2008.

Mark has added some code to send ZXTM messages to a Nabaztag Bunny, specifically the one here at enVirtua.
Below is a small video of this happening on our Nabaztag bunny :

If you can’t see the video it is available at : http://www.viddler.com/explore/lancew/videos/2/

Data Protection Act 1998

One of the common arguments I hear against cloud based computing is around compliance and the Data Protection Act (DPA) 1998. In this post I want to look at the DPA and see how it actually affects the use of “The Cloud” for organisations based here in the United Kingdom.

Before i begin, this article is being written by a IT person, not a lawyer. It in no way constitutes legal advice!

The Data Protection Act 1998, describes the legal obligations all organisations in the U.K. have in terms of handling data. The act covers such areas as data collection, data storage and data sharing. The act itself is “policed” by the Information Commisioners Office (ICO). The act covers both electronic AND paper data storage. Basically of you collect data on people, you need to obey the act. You will need to register with the ICO and follow their guidance.

In the most simple of terms, the act is “supposed” to ensure that all data in the UK is stored securely, never shared inappropriately, and kept only as long as required. The act effectiveness in this area is questionable, with a large selection of public failing (without serious consequences) being catalogued ( ORG Website ).

The failings as I see them relate mainly to the secure storage of data and the sharing of that data. Why the software (and hardware) in these large public bodies has not been built to prevent the ability to transfer data insecurely is beyond me… but I digress.

In regard to Cloud Computing, the areas that relate directly to the DPA and that I want to examine in this post are:

• Secure storage of data
• Transferring Data outside of the U.K. “without adequate protection”

I shall be exploring this in the context of storage. Specifically in terms of using “the cloud” to store your organisations data.

Secure storage of data

So, this means is the data being adequately secured. Now in a paper world, this would mean that you kept the personal information in a fireproof safe, which only a small number of people knew the combination. The receptionist counld not walk in and read a file. In an electronic world the bare minimum would be storing it on a disk, that is backed up appropriately and secured so only a small number of people can access the data. The receptionist should not be able to type a name into the CRM database and read someones information.

In an electronic world, the data should also be securely encrypted. That way, if a server is lost/stolen, nobody will be able to put the disk in another machine and copy the data off. Nor should Joe the junior IT guy be able to copy data off the server and put it on a CD. Or perhaps more accurately, even if Joe the IT guy copies the database onto a CD it is useless as it has been encrypted properly, both on the disk and inside the database potentially.

In the cloud the same applies. You should be storing your files in a securely encrypted format. The data should be encrypted BEFORE it is sent across the WAN to the cloud. This encrypted data is then stored on a secure host in the cloud, that has adequate backups etc.

If I look at my laptop before me, I keep all the data on the disk encrypted. So not even taking the disk out of my laptop and looking at the disk on another machine will allow you to open my files. I backup my data onto “the cloud”, I do this via a 256-bit AES encryption mechanism. Each file is encrypted and then sent to my data storage on “the cloud”. It does not matter if someone is “sniffing” my traffic, the data is encrypted prior to transmission.

The data centre my data resides in on the cloud, is secure, more so than my house or office I suspect. It has redundant power, air con, fire protection, swipe card access, etc. Even if someone got in there and stole some servers, the chances are that my data is split across multiple servers. So the risk of someone gettingf my specific files is smaller than if they broke into my office. The data is also stored, you will remember, encrypted, so even if they get their hands on my data, they won’t be able to read it.

Compare and contrast this to the all too common backup tape, where data is stored unencrypted on tapes, often in a relatively low security offsite location, like a warehouse or staff members home! If some gets hold of that tape, they just need to put it in a tape drive and they are 90% of the way there to getting access to your data!

So in the storage area, I genuinely believe that “The Cloud” meets your DPA obligations.

[UPDATE: 28 November 2008] George Reese over at O’Reilly has just published a quick guide to securing Amazon’s AWS cloud services. If his “20 Rules”are used as a guide you should be able to build robust Cloud applications. The article is at http://broadcast.oreilly.com/2008/11/20-rules-for-amazon-cloud-security.html

Transferring Data outside of the U.K. “without adequate protection”

Okay… I am going to approach this from the angle of storing your data overseas as opposed to transferring/giving your data to a third party. I.e. selling the data to someone else overseas.

Much like a lot to the DPA, and all legal documents, what they are saying is difficult to understand and perhaps is intentionally vague. My interpretation here is that if you are transferring data outside the UK, you need to ensure that it is secure.

But what about storing you data on “the cloud”, where the disks the data resides on are in Europe or the USA?

This is a more difficult topic, but I again think that the encrypted file storage possible with cloud based storage meets the obligations set out in the act. Although the data has been “transferred” outside the UK, it has been done so with some serious (more than adequate) protection. Of course if you simply copied the raw files onto the cloud, then NO, you do not pass go and do not collect $500.

To return to my laptop example, the data i am transferring out of the U.K. is encrypted on my laptop before transmission. Even if a server techie in the USA gained access to a file, they can’t open my data. So I consider it protected. As only I have the pass phrase for my encrypted files, only I have access, so the data should be safe.

So I believe that the cloud meets the DPA obligations for data transferring outside the UK.

Those of you who are giving this some thought have no doubt spotted some holes in my arguments, what if the data is not encrypted first is the first question I’d raise. In that case, all bets are off and I do not believe that the cloud meets the DPA requirements.

It comes down to implementation (as with all data security, be it paper based or electronic).

This is why I think that people like me have jobs. It is my role to advise organisations as to how issues like this affect their organisations implementation of technology. A tape vendor can easily say the cloud for backup is all bad. It;s slow (it is), it’s insecure (it can be), it does not meet the DPA/SOX/HIPAA (it might not). But equally, as has been shown by the almost weekly reports of data loss in the media; neither is tape, or disk, or memory stick, or paper!

continue reading "Cloud Computing and the Data Protection Act."

Some time back my colleagues and I needed to build an example usage for server automation product Cassatt Collage (since renamed Cassatt Active Response ). It needed to be relatively portable, and yet show off as many of the bells and whistles as possible.

What we decided to do was build a “Medium Availability Cluster“.
(It appears that “Medium Availability Cluster” appears to be a term we coined.)

A Medium Availability Cluster is just a term we used to describe a configuration of servers that is able to automatically recover from a failure in hardware (or software) with minimal downtime automatically. This is not a system that never skips a beat, this is a compromise solution that will fail briefly whilst it recovers.

The idea of the build was to show the power and benefits of server automation. We were not trying to show the power management or load balancing functionality that the Cassatt products offer. It was a simple example of how Disaster Recovery could be automated by software.

So how to create a medium availablity cluster…

To start with, build up a server and install your server management/automation software (Cassatt, OpenQRM, Puppet, etc.)

Once this is up and running use the included tools to create your “images” for your MySQL, Apache tiers.

In the example build we created, it was 2 MySQL servers, 2 Apache servers and a load balancer.
MySQL was configured to be a MySQL Cluster, Each Apache server was a basic Apache build with a simple PHP application running. We added the load balancer also to give a single point of failure.

A more sophisticated configuration that is particularly relevant to smaller sites, is to utilise Virtualisation via VMware, Xen, KVM, etc. Then each server in the tiers is in fact a virtual server on a physical host. You can include the host server (such as VMware ESX), inside your automation software configuration also.

Depending on how you do all this, you will want some spare physical servers included in the mix so you can cope with the hardware failures.

Okay, once it is built, time to play!