Did you watch The Daily Show last night?  Or just the first 10 minutes of it anyway?  It was really good – even by the normally high standards of the Daily Show. John Stewart took CNN to task for failing in the most basic of its journalistic responsibilities – fact checking. 

If you’re in the U.S., you can watch it here, though, quite obviously, if you’re at work, your employer may consider it recreational network traffic. 

For those of you who can’t see the video just yet, here’s a quick summation: CNN bothered to fact-check a sketch about President Obama on Saturday Night Live (a comedy show whose political comedy has often been based on satire and hyperbole) but fails to fact check most of statements on their program, including statistics spouted by guests, claims made in press releases, and presenting two talking heads arguing not over policy but over a statement of fact – and not telling who was actually correct. 

In short, they don’t fact-check what they put out on the channel.  In many ways this is worse than Jayson Blair at the New York Times or Stephen Glass at the New Republic – as those journalistic travesties were the result of concentrated efforts to fool established fact-checking mechanisms, while CNN seems – well, not to give a hoot.  This is not due to any political bias on CNN’s part, it’s just due to intellectual laziness.

CNN could fire some of its reporting staff and hire some network engineers as journalists.  A good network engineer knows how to fact-check.

For example, WAN Optimization solution vendors are likely to make a claim that their solution will reduce traffic by X percent or whatever.  Those claims are usually true but based on a lab test, so it’s usually better to verify rather than trust, quantifying how much traffic changes before and after a WAN Optimization is installed, validating, (or invalidating), the vendor’s claims.  Mileage may, indeed, vary.

Similarly, network monitoring can be used to make sure that your network service provider is living up to their service level agreements. 

That’s just two of the obvious ways in which fact-checking and verification is important to network engineers, but troubleshooting is nothing but checking the possible causes of problems until you’re left, by a process of elimination, with the cause. 

All of these things are why it’s important for engineers to have network monitoring software and to know how to use it properly.  Which brings us to my last point, which is to engage in a bit of navel gazing.

Network Performance Daily is the company blog of NetQoS, and by definition, it has got a bias towards our company and towards our products.  I try to disclose this whenever there’s appearance of a conflict of interest.  I try to treat the blog like a journalistic outlet (my M.A. is in Journalism, and I used to be Associate Editor at the Daily Texan) when it comes to reporting – and the idea has always been to give you information that our customers would find interesting and relevant, and on those days when we can’t find anything interesting and relevant, we at least try to make you laugh a little bit.  But we do hold ourselves to a professional standard; and when we make mistakes or make a point that later needs clarification, we correct, clarify, and apologize.  (This happened very recently with the FCC Net Neutrality speech coverage, but we’ve made, and corrected, errors with Vint Cerf’s Interplanetary Internet, for example.) But the point is this: one can still be entertaining and interesting and adhering to a journalistic standard. 

Anyway, to sum all this up: In order to make informed decisions about how to manage a network, you need to have information about the network; not speculation about the network, and not wild guesses about the network.

In order to make informed decisions about how to manage a country (through the democratic process,) you need to have the facts – not speculation about the facts, and not wild guesses about statistics.  The problem is not that CNN has delved too far towards entertainment; but that it is possible to inform while entertain.  Which makes it all the more tragic that CNN chooses not to. 

Distributed computation has been around a while in different forms – Beowulf clusters, for example, - but Ian Clarke, the developer of Freenet and founder of Revver, has started working on a programming language, based on Scala, called “Swarm,” which he hopes will create a distributed programming language that can run on almost any operating system.

Because it runs on an application level, any computer can be a part of Swarm. You run Swarm on any computer you like, and you can access the computation of other computers running Swarm on the network; or, theoretically, on the public Internet. And Swarm allows a programmer to code an application for multiple CPUs and multiple computers with the same code that you could code for one CPU on one computer.

Now, there are projects such as SETI@Home or Folding@Home which do similar grid-computing tasks, but both are based on a model of breaking up the data to bite-sized chunks, moving that data to individual machines, where the information is processed, and then resending the output back to the central server.

Swarm is trying to flip that on its head. With Swarm, you can run the program wherever the data resides. So if you had a piece of data on Computer A, and a piece of data on Computer B, and you wanted to do a calculation that required both A and B’s data, you wouldn’t need to copy the data over the network – the program would execute on both A and B, returning the result of the calculations on B’s data to Computer A. Swarm is designed to manage which software runs with which data on which computer – without the programmer having to think about it beforehand.

Combine this with the latest advances in dynamic allocation of virtual servers according to need, and you start to really chip away at a whole bunch of scalability problems that have traditionally plagued massively-multi-user-applications… that is, Web apps.

Now, here’s the question: CPU latency is measured in picoseconds. Network latency is measured in milliseconds. The question is: How do you figure out what computations will actually benefit from being offloaded to another computer? – i.e., which computations are so far back in the stack that it would be better for them to go for a round trip across the Ether than to just wait patiently for the stack to clear? It seems to me that network latency monitoring would be very important for such an application.

For example, let’s use some of the NetQoS Network Estimation Tools (shameless plug) to determine how fast we can theoretically get a calculation going over the network. So, figuring a router latency of 0.5 milliseconds on both ends, a server latency of 2ms, a link speed of 64000, and a (very short) link distance of 10 miles – you’re looking at 132 ms of latency altogether – assuming point-to-point protocol.

In that 132ms, a 2.4 GHz quad-core computer can perform 1.26 billion calculations locally. That seems like a lot – and it is. But you actually start saving time once you hit 1.26 billion plus one calculations. For some applications, that might be worth it.

But other than pure speed, there’s another reason to consider running Swarm – and that is that applications coded with Swarm should have the ability to continue running on other servers – preserving the application in the case of fault or insufficient resources on the primary computer.

Right now, Swarm is more theory than fact, and there’s a lot of work to be done before it can be practical. But anything that requires less data to be sent over the network is something to keep an eye one when trying to preserve network performance.

YouTube, according to a blog post by its CEO and co-founder, Chad Hurley, serves up one billion video viewings daily.

And you thought your business had a lot of YouTube traffic!

To keep in mind the rapid growth of YouTube, it took only three years and two months for YouTube to grow a literal order of magnitude.  This article from TechCrunch in July 2006 showed an impressive 100m videos served daily. 

Now, to put that 1 billion number in perspective, the current ratings for the top 20 network primetime series – calculated weekly, not daily, total a mere 298,013,000 viewers. (Though, to be fair, those shows tend to last 44 minutes, not a maximum of 10 minutes.)

Now, you know about the effect of YouTube traffic on enterprise network performance; and the importance of making sure YouTube traffic does not interfere with the mission critical applications.  Network performance monitoring, of course, is essential.

But YouTube’s growth isn’t just a bandwidth issue – it is a great cultural change which heralds as big an impact – if not bigger – than the rise of mass media in the 1930s. 

First, there’s the creation aspect – anyone can create a video and put it online.  There are no guarantees for distribution like there are with mass media, but neither are there barriers to entry to the market.  I myself created a number of short documentaries, and placed them on YouTube.  Some of them reached 100,000 views.  Had I gone the film-festival route, far fewer people – on the order of hundreds, rather than thousands, would have seen the videos. 

But more than that is the idea that viewers are willing to accept that entertainment does not need to be centralized - or displayed on the TV.  The average length of a video on YouTube is 2 minutes, 46.1 seconds.  I don’t have the percentage of videos watched in the U.S. compared to the world, but the U.S. uploads 34.5% of the videos – which is probably a good estimate of consumption as well, barring a better metric. 

So let’s work out the math… 2768333333 minutes of videos daily, 34.5% of which is viewed in the U.S., meaning 955075000 minutes of video daily in the U.S., divided by a U.S. population of 304,059,724… that comes out to 3 minutes, 8 seconds of YouTube watching – or a little more than a video a day, for each American.

This is not a problem that will go away – this is a cultural shift that will only get bigger with time.  Ignoring it will lead to disaster, especially considering that the baby boomers who grew up on TV are leaving the workforce, and the Millennials who grew up on the Internet are entering it.  Managing YouTube traffic requires a proactive approach to traffic monitoring and policymaking. 

Lifesize, a video telepresence maker-of-thingies, just announced support for a new video telepresence thingy called the Passport.

Hook up the Passport to a 720p HDTV (or other HDMI enabled monitor) and a 1mbit/up Internet connection, and you have a teleconferencing system.  Since many places have both HDTVs and 1mbit/up connections, this vastly opens up the number of places that you can do teleconferencing from. 

Certain practical limitations apply, of course.  For example, while it might be theoretically possible to use a bar’s wi-fi connection and HDTV for teleconferencing, I do not think it’s a good idea to do so when a game is on.

The standards list includes a number of protocols, so it should interoperate pretty well with existing teleconferencing equipment – meaning you can have your salespeople check in from the road, telecommuters checking in from home, etc.

One interesting side-note is that the device supports Skype at 720p30 resolution.  I’m not sure what the resolution of Skype phone calls are now, but I think they’re maxed out at 640x480p30 for the desktop client – obviously, that’s going to cause a bit of a bump in Skype traffic for organizations that use Lifesize. (You are monitoring this stuff in your organization, right?)

But there’s another issue with the Skype calling.  That is, Skype has become a defacto standard for teleconferencing among consumers (though they’d just call it “video chat”).  At the $2500 price-tag for the Passport, while it’s pricy, it’s not too pricey for some early adopters who want to use it as a personal telepresence device.  I could see an upper-middle-class family dropping $5000 on it (one on each end) to keep tabs on a kid at college, for example. All of this requires sufficient network performance – about 1mbps, as mentioned earlier – and many “broadband” networks in the U.S. do not have that kind of speed.  At any rate, if these things get popular, we’re talking about increased demand for broadband speeds and increased usage of networks for latency sensitive communications. 

*delivered really slowly.

The Washington Post has an article on a phenomenon that we’re all familiar with – that advertised broadband speeds don’t always match up to the actual performance that the end-user actually receives. 

Actual broadband speeds lag advertised speeds by as much as 50% to 80%.

So more than half the time, and sometimes as much as eight out of ten times, consumers are paying for slower Internet access speed than they signed up for.

Now, with congestion, infrequent outages, problems on the other end of the connection, and other vagaries of Internet performance, the fact that a customer’s effective Internet speed varies widely isn’t a surprise. 

What is a surprise is that companies do not monitor the performance of their own networks – or that they do, but give consumers bad data – either promoting a peak speed as the “speed” of the network, or promoting an impossible speed. 

Really, though, do you think it would hurt sales that much to re-label a “15mbps” offering as “7-15mbps?”  (Hmm, maybe it would, if the ISP can’t consistently deliver 7mbps.) 

"This speaks to consumer empowerment. And if you are advertising one speed but delivering another, that takes power away," Kelsey said. "Consumers can't make accurate decisions based on quality of service from one provider off another."

Now, there’s the truth in advertising approach – add qualifications, like a speed range, or parenthetical like 15mbps (during off-peak times) – but I think the “up to” disclaimer is good if there’s someplace – say, the order form for the service, or the company Web site where you sign up for the service – that explains exactly what your real performance is after you sign up, as well as the performance of the average customer at each speed.  Heck, you could even have one of those LED billboards like they have for state lotteries that show you how much that day’s jackpot is worth. 

We’ve talked before about how we believe that broadband caps are not a solution to the problem and would greatly degrade the overall network performance of the Internet.  That’s still true.  We’re especially suspicious of any sort of “gas gauge” that would tell customers how much they’ve downloaded – and nothing else.  But a true network performance monitoring solution, giving ISP customers true information that is actually relevant to their performance would be very welcome. 

Imagine, if you will, if you could go to your ISP’s web page, log in, and get this information:

• Your average Internet Speed over the past two weeks is X/down, Y/up.


• Peak Congestion Times are X:00am to Y:00pm


• X% of your Internet usage occurs during peak times.


• Your average Internet Speed during peak times is X/down, Y/up.


• Your average Internet Speed during off-peak times is X/down, Y/up


• At that average speed, you can video at Xmbps.  This is (low/medium/high) quality for standard definition and (low/medium/high) quality for high definition video.


• Your latency is Xms round trip to our servers. You can expect (low/medium/high) quality for voice calls and video chat, and (low/medium/high) quality for computer gaming.


• Recommendations for improving your Internet Experience:




• Try to watch streaming video during off-peak times, or set your computer to download the video during off-peak times instead. 


• Set peer-to-peer programs to use less bandwidth during peak hours.


• Try to find gaming servers located closer to your geographic location to cut down on lag.


• We noticed a number of anomalous behaviors these past two weeks.  Please check your system for malware and viruses.

That’s not “techie” information – it’s all information the end-user can use, and it lets the user know exactly what they’re paying for. 

Peter Kretzman at CTO/CIO Persepectives points out a serious problem with tech journalism in his article on cloud computing.  Sometimes the message gets oversimplified. 

Mainstream media drifts into this oversimplification in part because they’re leery of delving into technical arcania (virtualization, scalable architectures, APIs) that many of their readers can’t relate to. Yet, there’s actually no need, when you try to explain its real impact, to make cloud computing sound geeky and complicated; it’s not, at least at core.

Kretzman specifically points out examples from Business Week and NPR, which equates consumer-facing Web 2.0 technologies and SaaS apps such as GMail, YouTube, and Flickr to “cloud computing” as a whole.

To be sure, these applications certainly are “cloud computing,” but it’s just one example of what cloud computing is – and not the best example, because it seems to limit cloud computing to Web-based applications with data storage on the Internet. 

Cloud computing is more accurately described as renting IT resources when they are needed instead of owning them.  Sometimes you are talking about applications – using Google Apps instead of Microsoft Exchange.  Sometimes you’re talking about servers – using the computing power of big iron to supplement your medium-sized iron for those stubborn tasks, like trying to find the Higgs Boson, mapping known space, or making the best possible Fantasy Football picks.  And sometimes you’re talking about the network – using co-location to lower latency and provide more throughput to highly trafficked sites like CNN.com, YouTube.com, or ICanHasCheezburger.com.

Hmm… application, server, network… where have I seen that collection before?  Oh – that’s right.  That’s the three sources of network performance problems – sometimes the problem is in the application, sometimes it’s in the server, and sometimes it’s in the network.  The problem with cloud computing from a network performance perspective is that when application, server, and network were in-house, you could find out which one of them was the problem relatively quickly and start fixing it. 

But I digress.  The real reason I’m worried?  Cloud computing doubles the applications - and sometimes the networks – needed to do business.  Poor application performance could be caused by a poorly coded application – or is Firefox having a memory leak?  And the application is no longer from client to server and back.  Now you have client, to server, to cloud server, back to server, to client.  Fixing performance problems in this environment isn’t impossible, but it requires good monitoring solutions and an even better brain in the engineer doing the manual monitoring.

According to a post by Ann Bednarz’ on Network World, employers are beginning to understand that poor application performance can have an interesting impact on the bottom line. As more employees are working outside a central office, poor application performance impacts the productivity of branch offices and telecommuters.

A study by Harris Interactive (commissioned by WAN Optimization vendor Riverbed) suggested that employees that can’t be guaranteed application performance on the road are less likely to work offsite. As each employee who stays in the office costs money (water, electricity, office/cube space, and a parking space), telecommuting can be a powerful cost savings… if productivity can be kept the same.

But people who are frustrated by poor application performance are likely to stay in the office instead of working remotely. This might include those who probably should stay home and work, like, say, sufferers of H1N1. And for those who have no choice but to telecommute, they’re not nearly as productive as they could be.

Among employees surveyed, 40% say they would work offsite more often if their business files or software would load more quickly, and 33% report that accessing business files or software remotely negatively affects their productivity.

Now, there are some things out of company’s control – a bad public Internet connection on the employee’s end can’t be helped. But even so, applications can be optimized not just for in-house use but for WAN use as well, by lowering the amount of round trips the application requires, and by lowering the latency of the Internet connection. The latter is probably more difficult, the former requires recoding the app.

The only problem I have with the article is the title: “Mobile employees want speedy apps.” I thought everyone wanted speedy apps. I know I do.

I wanted to make a clarification regarding a blog post we recently published, called “FCC Weighs In On Network Neutrality.”  In that post, we talked about the FCC plan for network neutrality policies and legislation, and quoted FCC chairman Julius Genachowski extensively when he explained his reasoning for those policies.

What we did not make clear enough was that NetQoS does not specifically endorse those policies or agree with the FCC’s reasoning.  But the same token, NetQoS does not specifically not endorse those policies or not agree with the FCC’s reasoning. 

I know what you’re thinking: that we’re just another corporate company that can’t take a firm stand on a controversial issue.  But that’s not it at all.  The reason we don’t have a stance is that NetQoS isn’t just one person – it’s a collection of people, all with different viewpoints and different ideas of what makes good governmental policy regarding the public Internet.  We have every political viewpoint amongst our ranks: libertarians, neo-conservatives, paleo-conservatives, moderates, liberals, socialists, and one guy who votes for the candidate that looks most like Sam the Eagle. 

I don’t think we did anything wrong, per se, in our coverage, but I do think we could have done a better job.  When the article was written, the primary concern was answering three questions for the readership: What was going to be the FCC’s new policy? Why did the FCC come to its decision? How would the enforcement of these policies impact network performance?

What was in retrospect, misfortunate, was not asking a fourth question: Are there other alternatives, and what are they?

Where we are not ambiguous is on the ideal of a neutral network.  That is, that we believe that the primary goal of any governmental policies towards the public Internet should be those that encourage innovation from all comers and in all possible ways.  Tools such as traffic shaping, if used at all, should be used to preserve performance for everyone.  They should not be used to raise the barriers to entry for new technologies, new competitors, and new ideas. 

But as a collection of individuals, we're just divided on the best way to get there. 

The liberal viewpoint will tend to agree with the FCC’s reasoning for new policies.  If you trust the government’s ability to keep a neutral network more than the market’s ability to keep a neutral network, then in your view, it likely follows that you believe network neutrality policy needs to be drafted and enforced.  

But a libertarian viewpoint would suggest that a free market would be better able to keep network neutrality than the government, if only for the idea that the government can be bribed or intimidated into passing legislation that would make networks non-neutral as easily, or more easily, than they can be convinced to pass legislation that preserves network neutrality.  To the libertarian viewpoint, any governmental stance on network neutrality legislation is a bad idea because even good government policies can be reversed to favor bad outcomes by the next guy to come into the office, and that the market can exert enough pressure on the operators of public Internet gateways to favor a neutral stance. 

A moderate, of course, would suggest that it doesn’t really matter how we preserve a neutral Internet, so long as we end up with one.

So, with that in mind, please understand that the coverage of the issue on the blog will focus on the impact that policies, events, and decisions by all players in this space will have on network neutrality and on network performance.  That our goal will focus on trying to explain how the network will change under different plans, rather than arguing for or against any particular viewpoint.

Except for our unabashedly pro-puppy stance.  Anyone who doesn’t love puppies is a commie mutant traitor.

Here’s an interesting question – What’ll happen to application performance if the Obama healthcare plan passes?

I’m not just talking about the impact it would have on the networks of the medical industries, but across the entirety of the U.S. economy. 

Now, the Obama plan is both controversial and the coverage and interpretations are steeped in misinformation.  I’ve known supporters that believe that the plan will give them a free robot and puppy, and detractors who fear that they’ll have to install a crystal in the palm of their hands that will start blinking when they reach “lastday.”

But whatever the actual result of the plan is, people who think the public option would be sufficient for their needs (and who thought that independently purchased private insurance isn’t) might seriously consider quitting their jobs and starting their own businesses.  And among the many people starting a corner pizza store or barbershop or Spatula City franchise, there’s got to be a few talented people starting their own tech startups delivering cloud apps or Web apps, which have lower barriers to entry than desktop or server application development. 

Some of these applications may have compelling features – so companies may switch over to these Web apps.  Granted, this is already happening today, but the point I’m trying to make is that this single piece of legislation which has nearly nothing to do with networks may cause a very rapid jump in the number of cloud apps you have to support in your organization. Kinda freakonomicsy, but there you go.