Recently we have been working with a client to try and resolve some performance issues at one of their satellite offices.
The office is on the end of a 2MB line with around 40 staff using the standard sort of corporate applications (email, internet, MS Office, SharePoint, SAP, Intranet, MSSQL, Internal web apps, etc).
As you might have guessed, the staff have been suffering from performance issues and our client has been exploring solutions to improve the WAN performance. When we became involved they had just trialled the popular RiverBed WAFS solution… but it made the situation worse!
Why did it make the situation worse?
The problem with WAN traffic is often not about the simple amount of data going across the WAN, more often it is about the response times and “feel” of using the link within specific user applications. In our client’s case, the RiverBed did a good job, it compressed the traffic traversing the WAN quite well. Reporting a 50%+ “increase” in bandwidth, so in effect creating a 3MB virtual pipe where the actual link was only 2mb.
But the users complained MORE bitterly than before… why?
The reason is that that user performance is about more than raw data compression or TCP/IP optimisation. It is about how long it takes to open or close a file. How fast an application refreshes, how long it takes to load a SharePoint page. None of these examples were noticeably improved by the RiverBed’s configuration. And in fact made it worse.
It made it worse because it added latency as the RiverBed devices at either end processed the data traffic. It also provided a faster conduit for “chatty” protocols and for “greedy” applications. And based on the experiences of the staff it didn’t prioritise the traffic that mattered to them.
How to improve the situation?
What we would always suggest to a client is that they start by benchmarking the performance users perceive in both subjective and objective terms BEFORE any changes are made. You’ll want to benchmark across a variety of days and times. So first thing, last thing and say at lunchtime. On a Monday, a Friday and the last day of the Month, etc.
You will want to discover what traffic matters to your users and to your business (they are often different). And have your benchmarks ready to provide metrics on the level of performance of the network.
Then… and only then… do you want to make some changes. Once you have made one change, you will be well served to then re benchmark performance of the WAN, to see how the changes have changed the situation. Sometimes (as in our clients case), a supposed performance improvement change actually causes a decrease in performance. Often because solution vendors and their resellers, do not bother to benchmark with you; so have no idea what their product will do in your environment.
So what to change?
Here are some tips that we have used on many organisations networks and depending on their environment have worked well.
1. Try traffic shaping before traffic optimisation and compression.
Your WAN is host to a wide variety of applications, from recreational web surfing through to email and onto database traffic and sales systems. Some applications are “greedy” and will use all the WAN bandwidth they can and this is a really common cause of performance issues, a web download or upload can quickly kill the SAP traffic leaving your users tearing out their hair!Traffic shaping can limit how much bandwidth each application is allowed to use, prevently one traffic type from effecting another. You can also prioritise one traffic type over another, so that for example your internal Intranet can be given priority over an external website(s), or audio streaming etc.
2. Limit don’t block.
The temptation is to block bad traffic (like Audio and Video streaming, IM traffic, Skype, etc). But this can often be the cause of bigger problems. Simply severely restricting (say down to 25 kb/sec or lower) the users and applications will not detect that traffic is being blocked, just that it does not work well. This is more likely to prevent them using it than if you block the traffic and the application or user finds a way around the block and suddenly the traffic is unlimited again.
3. Protect the important before restricting the bad.
Think about what matters to your users and your business and protect that traffic first. Simply doing this is often enough, forcing less important and bad traffic/applications to fight for bandwidth below you important traffic will often give you all the improvements you need.
4. Think about what users experience.
Our most common suggestion is to severely limit the bandwidth that email is allowed to use. Which often comes up against resistance as email is often “mission critical”. The confusion here is about if the is important in terms of transport across your WAN. If an email takes 30 seconds or 90 seconds to reach a recipient is not noticeable to users as they are normally only told that it has arrived after the transport has occured. Also most email clients only poll for email on a 5 minute or more cycle. So limiting email traffic (so it takes longer) normally has no effect on user perception but can have a huge impact on the performance of the applications that need live updates (web apps, SharePoint etc).
Above are four tips that we hope you with your thinking about the problems you will encounter working over a WAN. Again the most important suggestion we have is to benchmark before, during and after implementing changes and solutions. You need subjective and objective reporting on performance to know if the time, energy and money you are investing is actually helping improve your situation. If your vendor(s) are not including some benchmarking, warning bells should be sounding in your head.
If you’d like some assistance in tuning your WAN, please don’t hesitate to contact us!
