Jim Frey, who I’m assuming is the Jim Frey who works at EMA rather than the Jim Frey who led the Kansas City Royals to their first American League Championship in 1980, has recently written an article in Network World about how network operators are, according to an EMA poll, most likely to be handed the responsibility for managing cloud services. 

One problem is that it’s a lot harder to visualized managed environments, as well as knowing where the boundaries are that clearly denotes “this is our stuff” from “this is our cloud service provider’s stuff,” and “this is our WAN service provider’s stuff.” 

Yet, concern over “poor service quality” – i.e., poor application performance – were not seen as a major issue in EMA’s poll, with only 18% of respondents indicating that they “had experienced or expected to experience poor service quality.”

Interesting.  I think this may be because while the problems associated with cloud computing and managed services are indeed new, in many ways, they’re not that different from the problems of having internal applications accessed through the WAN.

Which makes the networking group, which has probably spent the last half-decade or more getting the applications to behave on the WAN, old hat at many of the problems that cloud computing posts – and, of course, any cloud computing solution, by definition, has to go through the network.  If you’re talking about cloud services operated by a service provider such as Amazon or Microsoft, you’re talking about working on the WAN, with latency and bandwidth limitations. 

As always, there was support for network monitoring – more than half of the respondents indicated that performance and availability were the most important service monitoring disciplines – and application response time was considered the second most important metric.  (Fault reporting was considered the most important.) 

One last thing - poor performance in the “physical,” non-virtualized world could often be solved by “brute-forcing” the problem.  That is, you throw enough infrastructure at it until the problem “goes away.”  But that way of “solving” problems not only won’t work in a budget-tight recession – it won’t work at all in the cloud because you don’t own all of the infrastructure.  Ultimately, performance problems need to be handled by finding out where the bottleneck is and seeing what you can do to get resource usage under control, rather than just throwing more at the problem with diminishing returns. 

Brian Bakstran, VP of Product Marketing at our parent company, CA, recently blogged about a study from Network Instruments which talks about how 59% of IT organizations “lack the experience to manage virtualized environments effectively.”

Combined with the idea that by 2012, 80% of all new servers will be virtual ones, and you start to get this sinking feeling that the entire IT industry knows where it’s going, but hasn’t really thought about what it needs to do once it gets there… sort of like sitting in the first four rows at Sea World, all excited to see Shamu, but forgetting to pack a poncho.

And so vendors like us and our parent company offer that visibility. (In the case of CA, for right now, we’re offering it in spades, with the NetQoS stuff [PDF] and the e-Health stuff and CA Virtualization Management.) 

The main concern that the lack of visibility presents to enterprise IT shops is the idea that mission critical applications that performed fine before virtualization may perform poorly when virtualized, and the IT shop will have no way of being proactive in finding performance problems, nor will they have the tools they need to quickly find the root cause of the problem. 

And visibility is necessary even before virtualization to compare performance to the non-virtualized baseline.  There are some applications that simply will always perform poorly in virtualization, and the sooner those applications are discovered, the better.  Knowing what does and does not work in virtualized environments gives you options – you can replace the app, run the app on a dedicated server, or even recode the app to work better in virtualized environments.  But without visibility, you have no options.

Between the reduction in energy consumption and the better utilization of existing servers, the benefits of virtualization are worth the risk, but there’s nothing that says that you can’t bring in everything you can to get visibility into your virtualized servers and mitigate the risk. 

At the UBS 37^th annual Global Media and Communications Conference, Ralph de la Vega at AT&T mentioned that AT&T’s been having cellular data network problems because of heavy users – touting out the usual “80/20 rule” bandwidth hog rule (in this case, that 3% of users use 40% of the data) the company’s representative, Ralph de la Vega, suggested  to its investors that AT&T would likely introduce a pricing scheme that would penalize heavy data users, according to the LA Times.

We’ve written before about how the “X% of users use Y% of data” is kind of a boondoggle, but that doesn’t mean that congestion problems aren’t real for AT&T.

Which makes me wonder: If your data network couldn’t handle data streaming services for downloading web browsing, video, and applications, why, in fact, did AT&T team with Apple to sell a phone whose selling points were web browsing, video, and applications?

This is compounded with the idea that a lot of the problems have occurred in New York City, where AT&T rolled out the 850 MHz spectrum. The 850MHz spectrum travels further into office buildings and apartment complexes than the previous spectrum – and as a result data usage in the area went up 30%. 

Again – to me this seems like a case of infrastructure not being able to support the sales claims.  So the end result is that AT&T is trying to do the most difficult thing of all – trying to solve the problem by changing end-user behavior.  To their credit, AT&T is, at this moment at least, intending to use incentives to get people to use less bandwidth rather than punishments for heavy users. 

Still I feel a little frustrated.  Business Internet has been around for nearly, what, 15 years now?  How long will it take before people learn you don’t roll out new applications or services without baselining your performance and making sure that you have the infrastructure capable of supporting it. 

I have to admit I’m surprised – and amused – and a little frightened – (and maybe just a tiny little bit turned on) by news that Microsoft’s Bing search engine reached a 9.9% market share in October 2009 compared to Google’s 65.4%. That’s a huge slice of the search pie for a product that had only been around five months.

Additionally, ads on Bing’s searches are 75% more likely to be clicked than Google’s. I think I have an explanation for both. When was the last time you saw an ad on TV for Google? In fact, when have you ever seen an ad on TV for Google the search engine; rather than some side-project like Chrome, Chrome OS, Android or something like that?

Bing’s success is probably coming from their ad campaign – and by definition, if you draw people in with an ad campaign, you’re going to get more people who are influenced by advertising. (There are also those who need to use IE for Web applications, where Bing is the default search – and simply never change it.)

Anyway, whatever the cause, Search is a viable competitive playing field again. To be fair, Google has repeatedly kept improving their search algorithms, but nothing inspires improvement like real-world competition.

It may also be interesting from another perspective, in that, while searches don’t take up a whole bunch of bandwidth themselves, all the little add-ons and perks that search engines tack-on – from Yahoo’s Flickr to Google’s YouTube to Microsoft’s Virtual Earth 3D, can clog up the Inter-Tubes pretty quickly.

As you’ve no doubt heard, we’ve been acquired by CA, which, a few years ago, acquired Wily Technologies, which also has products in the field of “application performance management.”  Earlier today, there was a meeting among us to discuss similarities and differences between CA|NetQoS’s approach and CA|Wily’s.

And one of the things I noticed is that, ultimately, both our stuff and their stuff has the same goal, and in many ways, the same design philosophy.  We both knew that passive monitoring tools that can scale with the network and infrastructure were the way to go, and we were both trying to get down to the source of application slowdowns for quick resolution.  So from a high-level messaging perspective it can be hard for the uninitiated to tell Wily and NetQoS apart. 

But there was a difference in our philosophies, and this really did become apparent in the way the programs were designed and used.  NetQoS based their software primarily around using the information available from what the network gives you, namely packet headers and NetFlow.  And while we can drill down to the packet level if need be, our primary function is to baseline performance and then inform you when things start to deviate from the baseline. 

Wily, so far as I was able to ascertain from the discussion (which, I’ll admit, went so far over my head that I’m pretty sure it’s eligible for a DARPA prize for unmanned orbital flight,) drills down into applications much further than NetQoS.  For example, Wily’s stuff can drill down not only to the application process which is causing problems, but can steer you to the individual line of code which is causing those problems.  (This is important when you have, say, a Java application, which is just running JAR files on otherwise identical Java virtual machines – the process could be running fine, but there might be a hiccup in Java interpretation.

What’s interesting to me is that the two philosophies tend to have come from the different backgrounds of the two companies.  NetQoS’s stuff is really designed to be used by the network engineer, giving information which allows you to drill down to find and diagnose the problem as quickly as possible.

Wily’s stuff seems more geared towards the (networked) application developer, the person who needs to know where the problem is in the code to fix it. 

I’m sure that to many people, getting two application performance applications may seem like having both Matlock and Columbo on the case.  And in a way, that is.  Columbo goes around solving pretty much all the crimes in Los Angeles to make sure they have the right suspect.  Matlock comes in when they know there’s a problem – namely, the wrong person’s been accused – and then gets down to the nitty gritty in order to fix the problem. 

I like ‘em both.  (I still think Columbo’s cooler, though.)

There are only 17 shopping days till Christmas, only two shopping days till Hannukah, and eight shopping days till Samuel L. Jackson’s birthday.  Sure, it’s not a religious observance, but Jackson’s tired of monkey-fighting people forgetting his Monday-to-Friday birthday.

So – what to get a geek?  Here are some ideas – and some idea of the impact on network performance.

1) Mini HD Camcorder

Cisco’s Flip cams dominate the industry to the point where any small-sensor high definition portable cheap camera have been termed “flipcams” – sort of like “hoovering” the rug in the U.K., or blowing your nose with a Kleenex. 

Just be aware that at if you put those flip videos streaming on the network, you’ll end up with 9MBPS worth of bandwidth being sucked up. 

2) An internet TV streaming device

Whether NetFlix, Vudu, Amazon, AppleTV, or a Blu-Ray player with YouTube built in, it seems to be time of the Internet enabled set-top box.  Personally, I’ve just got a computer hooked up to my TV, and that seems to do the trick. 

Again, however, streaming video over the network could result in slowdowns, especially with high-definition content.

3) Online Games

Whether it’s World of Warcraft or Team Fortress 2, online games are often a nice buy for gamers.  They don’t take up a lot of bandwidth, per se, but they do require low-latency connections, which may be a problem if you’re also running some sort of Voice app or video communication program at the same time. 

4) A Cellphone

Droid, Blackberry, or iPhone, the smartphones of today all have some sort of Internet connectivity through WiFi – sometimes automatically.  So, as people come into the office after Christmas break, all of a sudden there’s a rush of new devices on the WiFi enabled Internet. 

And before you ask, yes, it was a bit of a slow news day. 

Anandtech has an interesting article out about how it might be time to move forward on standardizing the x86 instruction set – both Intel and AMD have had proprietary instructions included in their CPUs. On the desktop, these proprietary instructions aren’t annoying – in the worst case scenario, a program would take advantage of a “shortcut” on one processor that wasn’t available on another – but what has changed for IT users is that increasingly, x86 chips are being emulated in virtualization, which increases hardware requirements and slows systems down – cutting the benefits which have traditionally driven virtualization.

“Much worse is that this unstandarized x86 extention mess has made it a lot harder for datacenters to make the step towards a really dynamic environment where you can load balance VMs and thus move applications from one server to another on the fly. It is impossible to move (vmotion, live migrate) a VM from Intel to AMD servers, from newer to (some) older ones, and you need to fiddle with CPU masks in some situations just to make it work (and read complex tech documents).”

One of the more interesting riddles about the IT industry and computer development in general is the idea of standardization.

Innovation often occurs due to non-standard protocols and unique innovations. Some open source projects are forked and become a jumbled mess of programmers working at cross-purposes; but others are forked and create new innovations that merge their way back into the main stream of program development, and become a new standard.

And even though sometimes it seems like every vendor has a new way of doing the same old thing, the competition between technologies gives motivation and ability for improvement.

On the other hand, it can also get in the way of doing stuff you really want to do! Windows has become a “de facto” standard for operating systems – and even among Linux, Ubuntu is leading as a clear desktop standard. The entire internet was built on TCP/IP – and it is indeed ONE internet, not a collection of “data services,” like AOL, CompuServe, and Prodigy; nor is it a large collection of BBSes.

Standardization leads to efficiency – but competition can lead to innovation. Multiple solutions spring up to a problem, and the best (or surviving) one becomes a standard which is then made more efficient, which opens up new solutions to new problems, and the cycle repeats.

Kinda like the Hegelian dialectical method, if you don’t think about it too hard.

Now, if all the computer technology in the world was developed by one company – standards first, innovation second – computer technology would stagnate. Yet, still, before “standards,” interoperability and integration provides a middle ground that allows for better technology to come onto the field without overly burdening the end-user with tons of configuration problems – which is why it’s such a high priority when choosing among enterprise networking vendors.

We’ve mentioned a lot about data caps, and why they’re not effective methods for controlling congestion, though they’re often sold as such to unwitting consumers. And we’ve done the analysis that shows that the effective speed of a capped plan can be slower than uncapped dialup – at least, when you average it all out.

But Benoit Felton of the Yankee Group went further. Sure, he does the math too, but points out that the users that are labeled as “bandwidth hogs” are hogs because they’re the top 5% of users – not whether those users actually cause performance problems.

The fact is that what most telcos call hogs are simply people who overall and on average download more than others… TCP/IP is by definition an egalitarian protocol. Implemented well, it should result in an equal distribution of available bandwidth in the operator's network between end-users; so the concept of a bandwidth hog is by definition an impossibility.

Now I'm pretty sure that many telcos will disagree with our assessment of this. So here's a challenge for them: in the next few days, I will specify on this blog a standard dataset that would enable me to do an in-depth data analysis into network usage by individual users. Any telco willing to actually understand what's happening there and to answer the question on the existence of hogs once and for all can extract that data and send it over to me, I will analyse it for free, on my spare time. All I ask is that they let me publish the results of said research (even though their names need not be mentioned if they don't wish it to be). Of course, if I find myself to be wrong and if indeed I manage to identify users that systematically degrade the experience for other users, I will say so publicly. If, as I suspect, there are no such users, I will also say so publicly. The data will back either of these assertions.

A great challenge, of course, and the right approach to it: Get the evidence. Make the analysis.

As a follow-up to yesterday’s post, too often both in ISPs and in enterprise networks, social network and video usage gets blamed for problems with network congestion without enough evidence. It’s true that bandwidth consumption can cause problems with network congestion, but if your congestion problems are caused by a misconfigured router or a slow application, blocking YouTube and chastising FaceBookers are not going to solve your network performance problems.

Ann Bednarz’s latest article for Network World wasn’t that surprising in the broad sense – yes, social networks are popular at the workplace, yes, they take up a bunch of bandwidth – but I’m not sure it makes the case that it tries to.

The key to the article is an analysis from managed security services vendor Network Box, which tracked 19 billion URLs and ranked the top five Web sites visited from business addresses by volume of traffic. It recorded that 5.8% of all Web traffic went to Facebook, with Google at 4.1%. That’s by visit – by bandwidth usage, YouTube consumed 7.8% of that bandwidth, with Facebook at 4.4% and Windows Update at 3.8%.

But what gets me is that while these numbers are interesting, they’re not particularly useful. As a metric of what Web sites are popular with people at businesses, it’s okay, but it doesn’t give you any idea about the actual impact of Facebook or YouTube on network performance – in short, it doesn’t tell you whether there’s a problem with Facebook or YouTube in your company, or even in companies overall. Plus, it only looks at Web traffic. Without knowing what percentage of overall enterprise traffic is Web traffic, there’s no way to deduce the overall impact on the network. Even then, some networks might have a very little amount, percentage-wise, of Facebook traffic, and it might cause a problem, or conversely, another network might have a large percentage of their traffic going to Facebook, but their network is able to handle the demand.

As such, it’s hard to use that data in the article to make the case that “Social networks take a bite out of corporate bandwidth.” Does that mean that social networking isn’t taking a lot of bandwidth, or isn’t a concern? No – just that it’s not the right evidence to support the conclusion.

That said, gathering evidence that lets you know whether Web traffic (or any other traffic, really,) is affecting your business critical applications is still a vital part of a nutritious network breakfast.

There are two articles in NetworkWorld today that caught my attention.

The first article talks about Google Apps, and it’s inroads into enterprise environments as a replacement for Microsoft-based environments. And while it is growing – with an IDC survey saying that Google Docs is “widely used” in 20 percent of companies, there are issues with user training, legacy e-mail migration, interoperability, tech support availability, and concerns about security for companies that have to worry about data retention regulations such as SarBox.

There’s plenty of detail in that article if you’re interested in the pros-and-cons of Google Apps rollout, which I’m not going to get into here.

The other article talks about how one of the top data center challenges is now energy cost – with energy prices predicted to rise (though, let’s face it, when in our lifetimes have they ever really fallen?) companies are shifting the cost of running IT equipment from an infrastructure budget to the IT budget. We’ve talked before about how this is a compelling case for virtualization (and server consolidation.)

What is interesting to me is that these two trends combined reinforce the need and the possibility of application consolidation. While it’s not new that similar apps are bundled together, (Outlook, after all, is an e-mail app and a scheduling app,) one of the big things about Google Apps is that it’s one rollout, one setup, and I would imagine, one server. E-mail, scheduling, word-processing with network storage, and video sharing – all one app.

One of the major problems that large companies have is that between legacy systems, new business needs popping up, combined with the idea that virtualization decreases rollout time, and you have a situation where you might have thousands of applications on your network – if not hundreds of thousands. With the complexity comes waste, effort duplication, and as anyone who has played with Lego or C code will tell you, the more complex the system, the more likely something’s going to go wrong.

When one company went into gain a bid on a Navy/Marine Corps Project, which was the biggest government IT contract that had come up in years, part of the scope of the project was to reduce the number of applications on that Navy/Marine Corps network to only 5000 applications.

But when they asked the Navy/Marine Corps how many applications were on that network at the time, they had no idea. The company, along with rival bidders, had to make a guess – and 25,000 sounded like a good number. After all, there was no possible way that there could be more than 25,000 applications on this network, and so, they made that assumption and bid on that basis.

In reality, there were more than 105,000. Every single Navy base, every single ship, every single Marine installation had developed their own suite of applications independently of all the others.

If you’re able to cut down the number of applications by an order of magnitude, it also reasons that you’ll be able to cut down the number of hosts needed for those applications, and cut down the number of hard-servers needed for those hosts.

And this is where SaaS and projects like Google Apps come in. Essentially, anything that runs through the Web becomes one app – the Web app. In this way, it is almost the ultimate in application consolidation. (Well, maybe the pentultimate - there are still browser compatibility issues, extra stuff to download into the browser, etc, that increase complexity on that front. It’s simpler than having multiple apps perhaps, but another source of complexity nonetheless.) More than that, though, even among Web apps, the move is towards multiple applications becoming singular applications – one of the reasons that Google and Yahoo have been so acquisition-happy is because of the idea that these multiple Web applications – chat, voice, collaboration, photo sharing, video hosting, social networking, etc. – becoming more integrated. Today we think of Google Docs, Google Calendar, and Gmail as separate applications; tomorrow, I’m not sure we will.