Kevin Davis, a senior consultant at NetQoS, will be presenting a few training sessions at Symposium about SuperAgent, the end-to-end response time module of the NetQoS Performance Center. This will include a training session about how to use time-based network metrics in troubleshooting.  He talks about his upcoming training session below.

In the session, I’m going to be covering the importance of using a time-based metric in troubleshooting, because end-users complain foremost about time.  For example, they’ll say “the application is running slow,” or they believe “the network is slow.”  To users, everything is based on time, that’s what they’re complaining about.  And they’re correct.

It’s very new to many people to think of performance in “time” although that may seem counterintuitive - because most people are used to reading utilization graphs.  With utilization graphs, however, we don’t know if 70 or 80 or 90 percent utilization is necessarily impacting the user experience.  I mean, we buy networking equipment, routers, switches, firewalls, servers, and we want them to be highly – or efficiently - utilized.  Seeing high utilization could indicate a problem – or it could just indicate that you haven’t over-purchased.  So you can have a link at 90% utilization or a router at ninety percent CPU utilization but you won’t know if that’s impacting the end-user without a time based metric.

It’s time-based data that tells you how the users are being impacted.  Sure, the utilization data – the interface utilization, memory utilization, I/O utilization, can often tell what is doing the impact.  But the time base shows you the degree of the impact – the real-world effect on end-users.  With a time-based instrument, such as NetQoS SuperAgent, you can find out where the delay increase is occurring, and whether it’s based in the network, server, or application. 

In fact, you can take a look at time-based data and make a determination very quickly as to which entity is creating the performance issue – the beautiful thing about SuperAgent, in particular, is that it trends by time 24/7, so not only can you determine how your important business applications are being impacted today, but you can go back and look at recurring patterns in performance issues.  You can see if today is worse than yesterday or last week or last month.

In the session, I’ll also be going over how to architect the data center for performance.  Placement of servers that participate in inter-architectures is critical for the health and performance of the application and indeed the data center.  We also talk about how different protocols, for example, Microsoft’s TCP/IP stack, can impact application performance by enhancing or degrading it. 

It’s important for servers that are serving the same application.  For example, a front-end Web server and a back-end Oracle database really should be on the same switch on the same VLAN.  That way they receive optimum service from the network.  If they do leave the switch, they’ll have to contend with bandwidth going up and down the switch links, and they’ll be switched and routed multiple times. 

Based on measurements from customer environments and from our own laboratories, when two servers are on different switches they can have up to 18 milliseconds delay between them.  If we think of that in the terms of network engineers of one millisecond per 100 miles, what in effect we’re doing when we put two different servers on different switches, or two different VLANs on the same switch, we’re making it look like those servers are 1800 miles apart – like one server is in Los Angeles and the other is in Memphis. 

by Patrick Ancipink
Director of Product Marketing, NetQoS

There’s been a lot of growth (and attendant hype) in technology areas like WAN optimization and application acceleration over the past few years, and for good reason. Anything that helps companies speed up and reduce the risk of strategic IT initiatives like consolidating data centers, turning up new branches or serving an increasingly mobile and scattered user community will be popular.

To help with cope with the increasing reliance on the WAN and keep latency in check, there are a dizzying array of vendors and products out there – but if you’re trying to determine precisely which techniques and technologies to implement for your specific needs, the array of vendors quickly goes from “dizzying” to “disorienting” and finally “nauseating.” 

Cisco’s been in this Tilt-a-Whirl™ of a market for a while (and NetQoS has been right there with them) and they’ve taken some big steps recently to provide a more holistic approach that centers on building an “application aware” network, rather than trying to highlight one type of implementation against another for a narrow set of capabilities.

NetQoS started working exclusively with Cisco closely to help customers evaluate, measure, and prove the effectiveness of WAN optimization and application acceleration deployments. As customers are moving from pilot phases into full production, the before/after measurements and comprehensive monitoring are critical to ensure customers are getting the benefits they intended and doing what they need to deliver application performance. 

To help get the word out, Cisco just launched a new section of their web site today that contains a wealth of information about, as they call it, “WAN and Application Optimization.” The downloadable presentation, Cisco WAN and Application Optimization Technical Overview Presentation, puts Cisco technologies (and complimentary ones, NetQoS included) into a useful context with a methodical approach and framework built around four steps: Profile and Baseline, Optimize, Evolve, and Operate. A whole Campbell’s Factory of Cisco alphabet soup technologies are included—WAAS, ACE, NBAR, Netflow, CBQoS, IP SLA, PfR—to show how they work in concert and what role they play in the bigger picture.

There’s also the Cisco WAN and Application Optimization Solution Guide , a very in-depth publication—like 227 pages deep—that is targeted for “technical personnel involved in the specification, design, and implementation of specific WAN and application optimization solutions.” We, here at NetQoS, are proud to have contributed several sections to book regarding the methodology and implementation of network performance monitoring for WAN optimization and application acceleration. 

(If you are looking for some lighter fare, the video on the site tells a nice story in about 6 minutes including an airshow, snowmobiles, windsurfers, and skydiving—interesting choices for demonstrating the criticality of serving video over the WAN.  Then again, some company somewhere has to make the recreational products, I suppose.)

Verizon and NBC are working on serving up TV shows to home computers. The problem is, high definition video, (and I've done some HD video work for the Web - shameless plug), takes a whole mess of bandwidth.

Now, the obvious solution for NBC would be to move to some sort of peer-to-peer distribution system, right? I mean, it works for Linux distros.

The problem is that a normal peer-to-peer connection doesn't distinguish between the cheap local links - that is, links on the same ISP, in roughly the same geographic area - from the expensive remote links. So while P2P provides a more cost effective solution, it doesn't provide the most cost-effective solution for the ISP.

A third party, Pando, has developed a P2P system for pre-authorized, pre-approved content, and has come up with a way to force peer to peer connections to look for local nodes first. This increase the efficiency of the system, lowers the cost, and generally increases the performance of the streaming/downloading video.

This is exactly the type of thing we talk about when we say that how the application is coded can have a huge impact on the application performance over the WAN. Sometimes instead of needing more bandwidth, you need to find a way to make the apps work more efficiently.

In this case, decentralized P2P systems developed after the destruction of Napster. Though they were much less likely to get shut down by the RIAA, they were also much less efficient. This dominated development of P2P applications for years. But for offering only pre-authorized content, a centralized system - especially one that takes advantage of the structure of the physical network, makes a certain bit of sense.

NBC will be offering Verizon customers their shows via Pando's P2P service - which they're calling P4P, later this year. The name is a logical outgrowth, P2P, or "peer to peer," versus P4P, or "peer for peer." P3P was disregarded because it sounded too much like PCP. And if a kid with a lisp goes around school saying: "I downloaded the latesth Methallica album on P3P" and a teacher hears: "I downloaded the latest Metallica album on PCP," well, that's just not going to be a story that ends well, now, is it?

There's only one problem with Pando's plan: Each ISP will have to give up information about its subscribers in order to participate - that is, the Pando platform requires knowing which nodes are "local" and which nodes are "remote" in order to optimize for the local connections:

For other ISPs to reap the benefits Verizon did in the test, they too would have to share information about their networks with file-sharing companies, and that they normally keep that information close to their chests.

''That's one of the objectives we have to solve -- how are we going to consolidate this data and distribute it?'' Pasko said, adding that the result of the test gives ISPs plenty of incentive to collaborate.

(Okay, maybe there's two problems: No offense to NBC, but when your biggest hit is a veritable case study in game theory… you need some new shows.)

June 16, 2007, Network World:

"We're telling IT executives to not support it because Apple has no intentions of supporting (iPhone use in) the enterprise," Gartner analyst Ken Dulaney says. "This is basically a cellular iPod with some other capabilities and it's important that it be recognized as such."

March 6, 2008, Network World:

During a media conference at its San Francisco headquarters today, Apple unwrapped a host of new features that are designed to make the iPhone more attractive to corporate users.

Six months is a long time in the tech world…

We've warned that eventually the iPhone would be appearing on corporate networks and that the new (at that time) devices would introduce vulnerabilities into the corporate network and take additional resources. What we weren't counting on was Apple making overtures to enterprise networking - we had assumed that, much like the original iPhone was hacked to run on multiple carriers, that those who wanted to use the iPhone for enterprise applications would have to provide their own, messy, stop-gap solutions.

Back in January of 2007, when the iPhone was first announced, we wrote:

"That's another question - will this device have VPN support so that traveling employees can get the information they need while on the road? And if they do - how do you secure the data? The iPhone, like all small devices, is easy to lose, and easy to steal. That makes it vulnerable to illicit access. Does the iPhone have cryptographic abilities to make sure data stays safe?"

Well, apparently, Apple didn't take that as a rhetorical question because the fruit-based tech company is going to support Cisco IPsec VPN in the next iPhone update - the same one that will bring secure Exchange support as well as the possibility of an "iTunes Store for iPhone apps" - current Apple plans are to allow third party development but that Apple would have the final say on whether or not the applications could run on the iPhone. (Of course, clever hackers have already found a way around that.)

At any rate, the iPhone now seems to be competing directly with the Blackberry, which is good in the sense that competition in technical markets lead to innovation, and companies will have to expect new types of devices using different types of traffic, which - well, isn't bad, but which can be frustrating, absent a network device monitor.

Personally, I'm a bit confused by Apple's insistence to cripple the iPhone into only running "acceptable" applications on the iPhone, as A) it's clear that people are going to use it the way they like anyway, and B) if Apple took the same attitudes with their Macintosh/OSX general purpose computers, some of the best Mac apps (Quicksilver, Colloquy, Transmission, Burn,) simply wouldn't exist. Perhaps this increases the security of the device but at the obvious cost of utility.

It's just rhetorical, and I'd love to get some comments on this, but is the tradeoff between security and utility a false one? I'm not sure - havening not worked much in the security side of technology - but it seems to me that if the iPhone can be hacked to make it more useful, it can also be hacked to make it malicious, and so the choice is not between security and utility, but rather between a lack of security with utility, or a lack of security without utility. Hmm… maybe I should ponder this more.

by Brian Boyko
Editor, Network Performance Daily

The release of Adobe AIR today might just bring about major changes - both good and bad - for network performance. AIR is a way to produce Web apps that can be run as desktop apps. It is cross-platform and relies, like Java, on a just-in-time compiler and an interpreter of application bytecode. There are interpreters for Windows and OSX, and a Linux interpreter in development.

"It allows Web application developers - or just application developers - to use the Internet technologies they know, whether it's Flex and ActionScript to target the Flash part of AIR, or Javascript/HTML/CSS to target the AJAX part of AIR," said Phil Costa, director of product management at Adobe. "It allows them to take those applications and run them on the desktop."

Costa explained that through AIR, (depending on what the application does and how it is coded,) companies may theoretically experience a lowered amount of data throughput and an improved network performance.

"Today a huge number of corporate networks are moving towards browser based applications, and one of the extra bandwidth requirements that it puts upon the network is that every time you access a [Web based] application, you need to download it. Whether that's HTML or Javascript, or all kinds of Flex and Flash content, that needs to be pulled over the network. Having the application installed locally avoids that. All that will be going forth is the actual data that you're trying to access."

"We've done tests with some of our customers where they've seen our bandwidth [usage] go down for Internet applications in general, because unlike a Web site, which creates both the content and the formatting of the content, most AIR apps are just passing the information back and forth instead of refreshing the page each time."

"Now, depending on what the application does, it may actually add [to] bandwidth requirements for the network as well. One of the things that applications do, is run in the background and connect permanently to a data source's real time streams, or frequently check for data. That could increase the bandwidth requirements. But that's more about what the application specifically does than anything specific about AIR."

AIR's capabilities allow for offline usage as well, which will likely prompt more demand for online apps as the major drawback of SAAS - inaccessibility - is mitigated.

"In addition to giving the developers and then end-user of the application the convenience of launching the [Web] application like any other desktop application," said Costa, "it gives them additional capabilities that they didn't have when they were targeting the browser, such as local storage, either in flat-files or structured storage like a SQL database, which is embedded in there, or drag-and-drop integration with the file system, and cut-and-paste as well as the ability to take data or content offline, and run it when they're on an airplane or just not connected to the network."

"The runtime provides a whole set of APIs for notifying the application when it is on and offline, and so the developer can implement behavior that accounts for that; in many cases what we see is that the developers are caching some of the information offline, so that if the user takes it offline, it will still be available."

"To give you an example… one of our customers, Anthropologie, built an online catalog that lets people browse through things they have, and they built an AIR version which lets customers make little notes to themselves about the product, and rather than store them on the Anthropologie Web site, it stores them locally. The customer can put notes on things the same way they put stickie notes on an actual physical catalog, and they don't have to share that information with the Web site, so it's private to them. It also means, from Anthropologie's standpoint, that they don't have to create massive databases to store that information."

Costa said that Adobe hopes that there will be AIR apps on mobile phones, something that there's no specific date on, but which is on the Adobe roadmap.

by Brian Boyko
Editor, Network Performance Daily

By now, everyone will have heard of Microsoft's hostile takeover bid for Yahoo, and of Yahoo's board rebuffing the offer. What people may not be thinking about would be how a "Microhoo!" would affect IT application performance planning.

While it's clear that Microsoft, having been unsuccessful in promoting its own software-as-a-service offerings, is now trying to buy their way into this market simply by buying out the market leader, it shows how seriously Microsoft takes the SAAS space.

Yes, it's Yahoo, and not Google, that is the leader on online SAAS solutions - at least as far as consumers are concerned. Google gets more searches and does better with online advertising, but Yahoo Mail, Yahoo Groups, Yahoo Flickr, Yahoo Del.icio.us, Yahoo Voice, Yahoo Upcoming.org and all of the other services that are owned by Yahoo more than make up for Yahoo's second banana status in search - to the point that Yahoo has more users and page views. Typing the word "mail" into Google returns Yahoo! Mail as the top search result - over Gmail. Seriously. Try it.

One reason, perhaps, why Microsoft might want Yahoo!

It's not a sure thing of course, that Microsoft will build SAAS applications after a Yahoo acquisition, or that those applications will become commercially successful. It seems like a paradox that Microsoft has not been able to do well in SAAS development when SAAS applications discourage open-source solutions. Sure, there are open source SAAS applications but the overhead and cost of hosting and maintaining SAAS infrastructure favors larger, established proprietary software vendors, with more money to sink into the project.

This also provides enough reason for those who predict that Microsoft would somehow "ruin" Yahoo's online-app offerings to pause and consider what would happen Microsoft's business strategy combined with Yahoo's online development and marketing.

As we've mentioned on Network Performance Daily before, you don't stop thinking about application performance once applications move out from the data center to the Internet.

The conversion of Microsoft applications - which are still in a strong position in the enterprise - to SAAS applications would mean big changes to IT planning. When you move an app from the Data Center to the cloud, you're giving up control of the infrastructure, and submitting it to the vagaries of the Internet. As we've seen recently with the undersea cables, it's not always that great an idea to rely on consistent Internet performance for business applications.

Additionally, it's disconcerting when you realize data, in SAAS solutions, is typically stored online. This makes SAAS solutions convenient, but it also makes SAAS solutions particularly prone to vendor lock-in. Salesforce is a wonderful app, but I wouldn't want to switch to another CRM manager, online or offline, if all my data was already in Salesforce.

So with this sort of lock-in, IT managers have absolutely no back-up plan if things start to go wrong with their application performance - whether it's in the SAAS application's data center end or the Internet links in-between the SAAS data center and the enterprise data center. There's even less margin for error.

In addition to making sure that each of the offices on the WAN has the connectivity and performance it needs (after all, even in the hypothetical situation where all the applications a company uses are online, someone still has to make sure the Internet gets to every computer) network engineers in the future may be evaluating solutions by running hypothetical scenarios of what would happen if particular Internet links or nodes went down for a period of time, and recommending particular SAAS services based on "worst case scenario" disaster prevention and recovery capability.

Of course, I could be completely wrong about that - prognostication is fun, but invariably you look ridiculous with the passage of time. But if there's one thing is certain: Whether or not Microsoft is ultimately successful in its bid, the bid itself is a herald of new challenges for IT.

The Aberdeen Group, a provider of business research services surveyed 205 organizations last month to identify best practices for enterprise network visibility initiatives and controls. They called the report "The Real Value of Network Visibility."

In the interests of full disclosure, it should be said that NetQoS co-sponsored the study but we did so only after the survey was conducted and the analysis complete. That said, though, the study pretty much validates our entire "performance first" approach towards network and application performance management.

What the Aberdeen Group suggests is a PACE model (Pressures, Actions, Capabilities, and Enablers) to achieve corporate goals. The idea is that businesses are pressured to be responsible to customer needs, and the actions that are effective are to establish a proactive control of the network. In order to do this, you need to be capable of defining your escalation pathways for network performance issues, having normal networking performance baselines, understand interdependencies between applications on the network, be able to segment round-trip application response times into delays caused by the server, the network, and the application, and finally, have a centralized point for looking at the network performance data.

Frequent readers of this blog will no doubt notice that this is the point where I usually mention that NetQoS makes some of the products which enable those capabilities. The Aberdeen Group reports that these "enablers" are network performance monitoring through a Web interface, tools for remote analysis and troubleshooting of network performance, tools for creating custom profiles for monitoring groups of network hardware, a unified network performance and security platform, tools for Netflow data analysis, and a lab environment to simulate network performance.

There are some other gems in there to be found. The survey results showed that that top 20% of performance scorers:

• Were the most likely to have the capabilities and enablers mentioned under the PACE model.
• Were spending less time on troubleshooting network performance and application performance, managing changes to network design, or enforcing network usage policies.
• Were much more likely to have merged application and network management into a single job role, and more likely to merge the application, network, and systems management teams into a single organizational unit.
• Were able to fix problems faster and less likely to rely on calls to the help desk for determining network problems.

What do you think about the Aberdeen Group's report? Feel free to leave a comment below.

It's interesting that we're coming out with an announcement about our support for Cisco Application Control Engine modules and appliances today, considering it's the same day when Juniper announced that it's not going to continue their DX application acceleration offerings. Juniper made the decision, because, according to Network World, "[Juniper] regards it as insufficiently distinguishable from competitors' devices."

Application accelerators and application delivery controllers can indeed be hard to differentiate. As one poster on Fark (and I have no idea how it ended up on Fark) put it, "The load-balancer market is starting to commoditize. This is not unlike the HTTP cache market about ten years back."

We just put out a press release with details about our support for the Cisco ACE application delivery controller. The NetQoS Performance Center and its application response time, network traffic analysis, and device performance modules are available today, integrated with Cisco ACE - a module to Cisco's CAT6500 switch and 7600 series router which provides load balancing and content switching, focusing on acceleration, security, and availability.

And it's particularly important because one of the reasons that this partnership developed was because players in this market, including Cisco, are looking for ways to differentiate their offerings.

One of those ways is being able to quantify the performance gains of the solution - with NetQoS Performance Center integration, network engineers and sysadmins can tell exactly what benefit they got for their investment in the hardware. Being able to justify your budget easily and quickly is a major selling point, and while there hasn't been a lot of focus on using response times to measure the effectiveness of load balancers, it seems a logical next step, considering we've already worked with Cisco before to provide this functionality in Cisco WAAS WAN Optimization devices.

Often, network engineers are forced to rely on CPU utilization, memory utilization, and disk usage as measurement. However, in order to really get an idea of how the application is performing for the end user, network engineers need to baseline and track server response time and application performance. In order to continue to provide good performance for the end-users, it's important to get alerts when deviations from normal performance occur and automatically investigate the source of performance issues. Combining response time metrics, historical SNMP data, and NetFlow traffic analysis is a very powerful combination.

Now, I can't tell you that there aren't perhaps other ways to differentiate your offerings. If I had a solid but unremarkable application-delivery controller and I was trying to compete with the Cisco ACE/NetQoS Performance Center integration, I could probably… paint it pink or something. You know, so that it stands out in the data center, so that people looking around will say: "Hey, what's that pink box?" Would spread word of mouth, maybe.

Or I could give away beer from a microbrewery with every purchase. I know a guy named Orf who has his own brewery. He's a good guy.

Wait! I've got it! every fifth application delivery controller is filled with delicious candy! Mmm, Candy…

What would make you choose one application accelerator over another? Please leave a comment if you'd like to chat about this. Or want candy. Mmm, Candy.

EDITOR'S NOTE: I e-mailed Nicholas Carr about this post and he suggested that I pick up "The Big Switch" instead of relying on the Network World article, which he suggested might be a bit "sensationalistic." I'll swing by my local bookstore later tonight and see if they have it and will shortly go through it.

By Brian Boyko
Editor, Network Performance Daily

Nicholas Carr (who has kindly mentioned this blog in a post about Ad-block) has written a book called, The Big Switch: Rewiring the World from Edison to Google. And according to Network World, Carr, who wrote an article called "Does IT Matter?" for Harvard Business Review, said in this book that:

"In the long run, the IT department is unlikely to survive, at least not in its familiar form," Carr writes. "It will have little left to do once the bulk of business computing shifts out of private data centers and into the cloud. Business units and even individual employees will be able to control the processing of information directly, without the need for legions of technical people."

Now, we haven't yet read Carr's book and so we can't comment on whether or not he makes a compelling case for the obsolescence of the IT department, and for all I know that quote was taken out of context. But I do believe that it will be a long time before the IT department goes away.

SAAS is a wonderful development, and apps like SalesForce are, to the people that use them, godsends. However, unique company problems require unique solutions - SAAS services are looking to appeal to the largest common denominator. For that reason alone, IT will always have a place in the enterprise.

Additionally, if you want to connect to the network, which you most certainly will have to do to access your SAAS applications, you need network engineers to build and maintain the network - even if it's just for Internet connectivity. And what about application performance?

Google or other SAAS providers will not design your WAN to deliver large backups during off-peak hours, won't get your VoIP service to work with your data applications without clogging the lines, and won't help maintain your company's computer security. (Heck, if nothing else, when a key Ethernet cable gets unplugged, you need at least a sysadmin to find out which cable was unplugged and to physically run down there and plug it back in.)

Relying solely on SAAS is problematic at best. You're at the mercy of another company's quality control - and if the site goes down, so does your business. Your company's data - important and confidential data - resides on another company's servers. Finally, what about capacity planning?

That last one is crucial. You are usually not privy to the capacity of third parties. Larger SAAS services like SalesForce probably scale well and overprovision. But if Carr's thesis - that eventually most enterprise software will be SAAS - holds true, there will be some applications that are further down the long tail and service a much more limited number of customers.

With a typical client/server app, you have all the information there if you need it - the ability of the server, the number of clients, the average traffic per client, and if you have any network management software, you have a very good idea of how much total traffic you can handle. But put that application out in the "cloud" and you no longer can see that information, so you have no idea whether or not you're doing fine or teetering on the edge of a major slowdown in the service. It completely negates any possibility of meaningful capacity planning.

Sure, it shifts the blame from the IT department to the SAAS provider, but ultimately, it's the same thing: less productivity, less on the bottom line.

If Carr's thesis is that SAAS is going to play more of a role in enterprise computing in the future, we can't help but agree. But to say that there's no role for IT in a future with more SAAS applications is assuming far too much.

By Nathan Bragaw
Business Development Manager, NetQoS

NetQoS European Tech Rep. Peter Frame and I made a trip to Moscow last month to attend a network management event hosted by our new Russian partner, UNICORNS. I thought I'd share a few observations from that trip.

First, the obvious: Russia is cold. I was born in Maine and been in Minnesota in Winter, but Russia, complete with snowfall and brisk wind qualifies as the coldest place I've ever been, and that includes Minnesota in Winter. On the plus side, the food is fantastic - I didn't realize I liked borscht until I ate at Taras Bulba on Petrovka Street - wonderful Ukranian and Georgian food.

It was clear from the wonderful work our local partner, Boris Goldshteyn, did with getting some of the biggest companies in Russia to attend our first network management event in the country that the Russian economy is in full throttle.

I am always fascinated by how different cultures and environments change the needs of our customers. In Russia, it's not that their network performance problems are any different. Instead, they are facing the same problems, like VoIP performance and WAN Optimization difficulties, but they are moving at a faster rate. This is largely because they are not confined by the fiscal restraints that companies in the U.S. face due to the fallout of the dot-bomb era. Companies in Russia want to invest in technology but are challenged to ensure that the technology they seek will meet the needs of their business. The network engineers we talked to there are looking at massive growth, the push for more current technology, and demands for understanding application performance over their networks.

What differences in culture and environments have you found when working on multinational IT projects? Feel free to tell us about them in the comments section of this post.