Thanks to Slashdot, I think I’ve found the source of the badly mangled story from Australia I reported from yesterday: Nemertes Research, which is quoted in this Times Online article.  (It would have been easier had Seven News just mentioned the name of the research group in their original coverage.)

And it’s a shame too, because it deserves to be taken more seriously than “I thought Cyberspace just came from the air” – now that the original study has been located, it is, in fact, serious. 

To sum up: Nemertes believes that predicted demand for bandwidth will outstrip the predicted capacity of the Internet infrastructure, and will do so within the next couple of years, making the Internet unsuitable for business purposes. 

Because the idea of an “exaflood” has been used before by non-independent think tanks, paid by vested interests before, I approached the idea with some skepticism, but Nemertes Research knows what they’re doing, and as far as I can tell, did a very methodical job. 

Granted, Nemertes, like most think tanks, is in the future predicting business to some extent, and even if they’re coming to the best conclusion that can be obtained from the information they have now, there is no such thing as an unavoidable problem with network infrastructure.  In fact, this is a problem with an obvious solution: Increase the capacity of Internet infrastructure to keep up with the demand.  If demand increases exponentially and investment in infrastructure increases linearly, then yes, it’s time for exponential investment in infrastructure. 

I’ll leave future prediction to Karnak the Magnificant.  But what I can say is that the possibility of Internet brownouts just serves as a reminder not to place all of your infrastructure out on the cloud – that leased lines and private WANs that your company can monitor and control from end to end make it easier to retain and improve network performance than relying on the Internet. 

There are several misconceptions about the Internet. For example, some believe that that the Internet is a very light black box about the size of a handbag with a single light, where the Elders of the Internet keep watch over it from Big Ben.

I never get tired of linking to that video. In fact, the entire TV show “The IT crowd” is based around the idea that to most of the world, networking is a mystery, the Internet a “magic” box that gives you sneezing pandas on demand, and no one cares. Until, of course, it’s time to PANIC!

In Australia, Channel Seven’s “Sunrise” morning news show had a computer expert with, Pete Blasina, talking about how the Internet is “filling up.”

According to the news report, a report where the anchor freely admits that he “doesn’t know how this Internet business works, I thought it just went through the air,” I “learned” the following things:

• Scientists (who are not named in the piece) are warning that video sharing sites such as YouTube are putting a “major strain on cyberspace.”


• By 2012, the Internet could get “full.”


• Blasina is surprised that the Internet hasn’t “run out of space” earlier, and the fact that it hasn’t done so is “remarkable.”


• The Internet is about 13 years old, and it hasn’t broken down once. (For those counting, that means the Internet was invented in 1996.)


• The Internet is probably “the most perfect machine we’ve built as humans.” (It clearly outshines, of course, the lever, inclined plane, wheel and axle, screw, wedge, or pulley.)


• With “video going down the Internet, and so much entertainment and social networking, it’s virtually at capacity now.”


• “It’s called Cyberspace, so we think it just appears out of the air.”


• The problem with The Internet being “full” is not the pipes, because “the pipes are fine.”


• “Optic fiber is infinitely extensible.”


• “The issue is that all the information has to be resident somewhere.”


• “The issue is with the switches that transmit down the fiber connection.”


• “There are massive server farms, or computer farms, where the information is resident. And that’s where the clog happens.”


• “We almost now are at the point where we need to go back down to the foundation and rebuild the infrastructure from the ground up.”

At the risk of insulting Australia Channel Seven’s news staff, this is frozen concentrated stupid juice.

Seriously, if this wasn’t an actual news show, it would sound like an Onion skit.

The segment goes on for four minutes and in that time, the expert asserts the Internet will become “full” but doesn’t quite explain why, blaming YouTube and video applications for the problem, then blaming server storage space for the problem, then blaming problems with delay in the switches, then goes back full circle and talks about YouTube and other video applications and the massive amounts of data they deal with.

Now, as a network performance vendor, one of the reasons we have an end-to-end solution and integrated suite of tools is because we’ve heard of situations where problems are first blamed on the network group, then on the server group, then on the application group. However, this is the first time I’ve seen a single person flip single-handedly from one to the other to the next. And, like adding a marichino cherry on this banana split of confusion, he does it all without actually mentioning what the problem is.

He doesn’t even mention who is saying the Internet is getting full by 2012. “Who” is literally the first thing taught in journalism school, followed by the next five things, “What, When, Where, Why, and How.”

It is bad enough that there is so much complex information out there that is misunderstood and mishandled. Oversimplification is one thing, but at least oversimplification tells you what the problem is and a simple, easy to understand, wrong solution. I don’t even know what the hell the problem is or what they’re trying to say. I’d offer to summarize and re-write it to explain the problem, but I don’t even have enough information to even do that.

But I can get outraged about it because I know enough about computers to know that that was pure nonsense; what worries me is that “the internet will get full” becomes conventional wisdom because that’s all that the non-savvy TV viewer will take away from this piece. After all, Blasina must know what he’s talking about, because “he’s on television.”

And while I’ve never been on TV, I have acted in Pete Blasina’s capacity, not just as editor of this blog and contributor to HardOCP. I’ve also been a tech expert on the Marcus Lush radio programme in New Zealand, and have been interviewed by the New York Times TechTalk podcast.

Maybe I need a pink Hawaiian shirt.

According to a DailyKos/Research 2000 poll, 51% of Texas Republicans approve of Texas Governor Rick Perry’s suggestion that Texas may need to leave the United States, and 48% of Texas Republicans believe Texas would be better off as an independent nation, with 4% not sure.

Overall, only 58% of all Texan respondents disapproved of leaving the U.S. And remember, because of the mechanics of the Winner-Take-All/First-Past-The-Post political system, only the opinions of the majority of the party that wins actually gets taken into consideration. In Texas, that means the Republicans.

What this means is that the possibility of Texas secession is, technically, “plausible,” assuming that the U.S. will just let Texas up-and-walk-away. I mean, I’m using the term “plausible” in the same way that the Mythbusters use “plausible” to describe scenarios such as “two civil war bullets fusing when they hit each other in mid-air.”

On the other hand, Texas isn’t that popular with the rest of the Union, for various reasons. Texans annoy non-Texans because they think that we think they’re better than them, and non-Texans annoy Texans because we’re better than them. So they could just let us go with a simple, “Don’t let the door hit you in the butt on the way out.”

So, as an IT professional, what do you have to do in order to prepare for the scenario where all of a sudden, you have to pass customs to get from Texarkana’s Waffle House to Texarkana’s IHOP?

First of all, if your company does any sort of business with the federal government (of the U.S.), you will likely have to relocate your company headquarters to another state. (You have 49 to choose from, but I prefer Iowa. Iowa doesn’t get enough love.)

But it’s not likely that all of your employees, nor all of your equipment, can move immediately; bringing the importance of WAN Optimization to the fore. It’s really no different from server consolidation; in both cases, the people who need to use the resources are physically further away from the resources they need to use. In order to do that, you need a highly functioning WAN; and you need to monitor the WAN performance to make sure you’re getting the most from optimization solutions.

Additionally, in addition to the employees that can’t immediately move out of the Chucktatorship, there will be employees that immediately do move. They will probably need VPN access through the Internet to telecommute – either that, or virtualized remote desktops. Again – you need to be able to adjust quickly to the new network performance needs.

As for NetQoS, we’re probably going to end up staying in Texas, though I’m probably going to be telecommuting, personally. Have no fear, even if the country devolves into a second Civil War, I’ll be here with you, providing the vital services of an infotainment vendor blogger during the nation’s darkest hour.

From Scandinavia, most likely…

People have a tendency to strongly prefer avoiding losses to acquiring gains.  For example, which would make you feel better: getting a $10 discount, or getting a $10 gift card?

If you answered the latter, you’re obviously not human and should report to the nearest alien processing facility where you will be “massaged” in the Dissect-o-topsy 3000.

For those readers not keen on sucking out human brains through a sippy-straw, this phenomenon may explain why network security always tends to be “sexier,” getting more attention, than network management and application performance.   The heroes are never the guys who save the company $1M over 5 years, they’re the guys that prevent a $100,000 loss from a malicious hacker. 

But there are certain cultural and practical considerations as well.  For example, very few people know what “latency” is.  Instead, a decision-making executive is more likely to know – or think they know - what a computer “hacker” is. (Let’s not get into the hacker vs. cracker nomenclature debate right now.)

In short, it can be a lot easier to get funding to prevent a potential loss than it is to invest in a known gain.

But there are other considerations too – regulations designed to protect, not the company and it’s assets, but everyone else.  Security Expert Bruce Schneier put it this way:

If ChoicePoint has lousy security and someone steals our identity information, we are harmed. But to ChoicePoint, it's an externality. ChoicePoint isn't a charity, and it's not going to improve its security out of the goodness of its heart. If we want ChoicePoint to protect our data, we're going to have to force them. We need to raise the cost of their having lousy security, so it'll be cheaper for them to have good security.

At least, that's the idea behind regulation. Unfortunately, reality isn't nearly as simple as the theory. When you're talking about regulation, the devil is in the details.

With regulation, an emphasis on security over performance starts to make more sense, because while poor network performance can cost a company over the long term, it usually doesn’t have as much of a problem for the general public as a data breach would.  It’s just the nature of the business. 

Even so, we see more crossover between the performance and security spaces daily.  Anomaly detection can be used to detect performance problems as the first tiny bits of evidence pop up – or malicious activity as it occurs.  Retrospective analysis provides both performance data and forensic evidence for intrusion. 

And of course, if your network is performing poorly, it’s nice to know whether that’s because of short-sightedness or sabotage. 

At any rate, maybe we’re a bit too worried about network security when there are other real problems.  After all, it wasn’t a hacker that took out Bear Stearns, Fannie Mae, Freddy Mac, Lehman Brothers, or Merill Lynch. 

By Stephen Creel

I just completed my 6^th annual Symposium as a NetQoS employee and I want to thank our customers for making this another success. I was thinking back to my first one (2004) and there are some pleasing contrasts between that initial experience and what we accomplished this week. The early Symposia were intimate enough to be held inside our offices and we had a few dozen people spend a few days with us. I drove the van. We’ve grown to the point where we are hosting hundreds of customers, have three full days of sessions with four parallel tracks, six keynote presentations, a hands on lab, a usability lab, and a bouncy castle (just kidding on the last one)—all that necessitates a full-fledged conference venue and I don’t think there’s a better one in Austin than Barton Creek.  

We believe this year’s Symposium contained the strongest agenda yet. In addition to the sessions presented by NetQoS employees relating best practices, how to’s and product updates, speakers from Microsoft, Cisco and EMC delivered popular presentations ranging from unified communications to unified computing to operations excellence.

Of course, events like these are not possible without customers like you that are dedicated to optimizing how well your organization can deliver applications. For those of you that could not make it to the event this year, we will get the presentations on the self-service portal as soon as possible and we hope to see you at a local workshop.  Hopefully the economy will continue to improve and we will see all of you next year in Austin.

I’ve said enough. Here are a few comments from this year’s attendees:

• “I enjoyed the usability – being able to have input on planned features. I suggest an increase of the men and resources in the hands-on lab. It was full every time I looked.”
  
  
• “Sessions are getting better; More technically detailed. I continue to be more and more impressed with your products as they develop year over year.”
  
  
• “I really like Zach Belcher’s presentations. I think I have pulled the most useful information from them. The length was perfect. Any longer and topics and classes would have become a jumble.”
  
  
• “The presentations were packed with useful information and new product features just keep getting better each year. Great job!”
  
  
• “Please warn attendees about rattlesnakes on the nature trail.”
  

---------
Stephen Creel is the Director of Account Management at NetQoS.

While it’s a bit soon to be revisiting Time Warner’s bandwidth caps, there have been a few developments which make this story more interesting and more relevant.

One of the biggest is the idea that Time Warner had tied planned DOCSIS 3.0 upgrades to the incorporation of bandwidth caps – and that, as Austin and Rochester and the other test markets didn’t accept bandwidth caps, TW would not use Austin and Rochester as testbeds for DOCSIS 3.0.

Gizmodo calls TW “bad losers,” but I’m not sure I agree. While congestion isn’t a problem on current connections, a 50Mbps downstream connection might, indeed, result in some congestion if Time Warner doesn’t make infrastructure upgrades. DOCSIS 3.0 may be cheap, but it probably will lead to increased traffic, requiring more bandwidth on the backend.

I am keen to point out, however, that this is because Time Warner is increasing the amount of bandwidth per customer, not because of increased data. So, I can see why they would tie the upgrade to the increased revenue resulting from bandwidth caps.

Just one little problem: At 50Mbps, you blow through 20GB/hr. If you cap data, there’s really no reason to actually go ahead and get faster service.

So, Time Warner is, if you follow the logic here… putting caps on customers… to pay for upgrades… which customers can’t use… because they’ve put caps on customers…

(There’s a hole in the bucket, Time Warner, Time Warner. There’s a hole in the bucket, Time Warner, a hole.)

Saul Hansell from the New York Times pointed out that backbone Internet providers actually refuse to sell by the gigabyte to ISPs like Time Warner (as well as other businesses,) and that the costs are bandwidth, not data, related. It is fair to assume that increasing bandwidth to customers will require increased bandwidth costs.

But the more interesting thing is that Hansell reveals some of the information about what DOCSIS 3.0 actually is:

“But the last link, running from a neighborhood office or a small device hung on a phone pole—runs over cable TV or phone wires. In a cable system, there is a fixed amount of bandwidth that is shared among all the customers in a node, often about 500 homes.

That capacity, in current technology, provides about 38 megabits per second to share. That means if four homes are all downloading very long files at 10 Mbps, a fifth customer going online, will start to slow down everyone’s connections…

…The other way that cable companies are increasing capacity is by using new technology known as Docsis 3. This is a standard that allows companies to use more video channels for Internet service. The current standard uses one video channel. The first generation of Docsis 3 service combines four 38-Mbps channels into a pool of roughly 152 Mbps that can be divided among customers. Cable companies can decide whether to use that capacity to offer higher speeds to customers or to increase the number of customers who can be served at slower speeds, avoiding the need to split nodes.[Emphasis added]”

Here’s where things get tricky; if you’re limited to 38Mbit/s for 500 homes total, isn’t it a bit irresponsible to offer 15Mbit connections to individual houses?

Imagine, if you will, a bank that leveraged 7.5 billion dollars and had 3.8 million in assets. Such a bank would be… well, actually, it’d probably be pretty typical. So forget that metaphor.

But at 152Mbps, the 500 home ratio starts to feel a little roomier. Even if all 500 homes got online at once, you’d still get 311kbps; or 38kB/s.

Hansell points out that it could get even roomier:

“But most cable systems are in the process of converting to an all-digital format from the current approach that mixes analog signals (which can be watched without a set-top box on an older “cable-ready” television) with digital signals. This is mainly being driven by the need for extra capacity to handle high definition programs. A company can send 10 standard-definition channels or 2 high-definition channels in the space of one analog channel. All that means is that there is not a shortage of channels for use by Internet data, at least for a while.”

This moves Time Warner’s suspension of the DOCSIS 3.0 rollout from the “darn” to the “huh?” category. Even if faster speeds aren’t available right now, wouldn’t it make more sense to keep the current broadband offerings and rollout DOCSIS 3.0 anyway, so that congestion in that last mile won’t be as pronounced?

Now, this next part is pure speculation, but perhaps there’s a method to Time Warner’s madness. (Granted, there’s more “madness” than “method” in here, but follow along anyway.) Verizon and AT&T, using fiber optic technology, offers 50Mbit and 25Mbit connections. This pressures Time Warner to say that they were working on >20Mbit solutions of their own. So in a choice between using DOCSIS 3.0 to offer greater speed to customers or using the new technology to fight congestion, Time Warner decided to do the former.

The only problem is that, just as 7 and 15Mbit connections to 500 houses with 38Mbits of capacity were overselling, so would 20-50Mbit connections to 500 houses with 152Mbits of capacity be overselling. Overselling 7 and 15Mbit connections worked mostly because most customers do not actually use the capacity that they’re being sold. (Even a 300GB/mo “heavy user” like myself only consumes 5% of the capacity he was sold.) But with more customers, more applications, and more demand, that assumption doesn’t necessarily hold true in the future.

If customers actually used the Internet they were sold, there would likely be no way that customers wouldn’t be affected – except if they could be assured that customers couldn’t actually use the bandwidth that Time Warner was offering. Thus, the data caps – set low enough so that Time Warner can sell 50Mbps Internet – like its competitors do – without having to actually deliver the capabilities of 50Mbps Internet – like its competitors do. Without caps, Time Warner’s entire plan for DOCSIS 3.0 falls apart.

Time Warner could, of course, if they invested the time and research into it, convert more analog cable TV channels into increased bandwidth. What might be a better plan for Time Warner is simply to bite the bullet, use DOCSIS 3.0 to relieve congestion rather than increase speed, and increase speed incrementally when the capabilities get there. As the Rochesterians switching to Frontier in droves show, people are willing to sacrifice speed for data; if Time Warner has an uncapped 7Mbit plan while its competitors offer capped plans of any speed, they’ll not want for customers.

Of course, that assumes competition – and the other major problem is that Time Warner is actually trying to lobby North Carolina’s state government to literally outlaw municipal broadband.  The city of Wilson – 50 miles east of Raleigh – has figured out how to provide 10 down, 10 up fiber connections for $35, compared to Time Warner’s 10 down, 2 up cable Internet connection for $57, in a program they call “Greenlight.”

Hmm… anyone know how to get something like “Greenlight” started in Austin?

by Patrick Ancipink

There’s been a lot of discussion this week at Symposium about dealing with the daily realities of virtualization. The issues cut across technology and politics and the cost savings from virtualized servers basically guarantee continued virtual machine (VM) adoption and sprawl.

Perhaps the most colorful statement I heard was from a network engineer characterizing how the server and application teams approached him on provisioning more VMs: “Just give me an IP address and stop asking questions.”

Ay caramba!

In some organizations it seems the cost savings of VMs are being used as a blunt instrument to justify rapid VM provisioning and throw process and performance implications aside. The risks of unpleasant surprises and painful performance degradation seem to be rising in parallel with VM growth that eschews planning and testing. After some discussion about the Wild West of VM sprawl that some of our customers are experiencing, a network engineer reflected back on our astronaut’s keynote: “How can we work the plan when there is no plan?”

Luckily we heard some better news in a keynote this morning from George Kurian, VP and GM for Cisco’s Application Delivery Business Unit. It always warms my NetQoS heart to listen to George describe why visibility and management are mandatory components of optimizing the application delivery network, and it was reassuring to hear about what Cisco is doing to address the complexity and headaches of virtualization sprawl.

Specifically, providing “fiber channel over Ethernet” promises to deliver a “lossless fabric” and denser switches and servers can cut in half the amount of network connections you have to manage in the virtual data center. With this type of simplification and efficiency--and the attendant savings in power and cabling and labor—virtualization can scale more gracefully than it does today. Working with standards groups to keep the virtual stack as open as possible sounds like the right path. (I know, I know, standards groups have a pretty poor track record in networking and management, but in this situation I think it’s a better approach than building a brittle, more proprietary stack that inhibits flexibility.)

George talked about some cool new stuff like Priority Flow Control and Virtual Network Link coming in the Nexus product line, but I can’t really do justice to it and you can read about that on your own.

Now, where’s the IP address I asked for?

By Patrick Ancipink

After CEO Joel Trammell’s welcome address yesterday morning, Colonel Mike Mullane, a veteran NASA astronaut with several Space Shuttle missions under his belt, reminded us of the importance of “planning the work, and working the plan” and of the dire consequences of not doing that.

Applying that principle to managing the network for application delivery is a bit less dramatic than space travel, but the importance of teamwork, responsibility and not falling prey to the “normalization of deviance” translates easily enough.

The normalization of deviance was particularly interesting for me to ponder. How many times have I been part of something where a compromise was made and we all said “just this once and never again,” but sure enough that exception becomes part of the standard operating procedure?

The teamwork theme carried through this morning when three gentlemen from NBC Universal told a very impressive story about delivering 3.4 petabytes of video over 17 days from the Beijing Olympics back to NBC headquarters at 30 Rock in Manhattan. (There are 1024 terabytes in a petabyte.)   The video had to be available for live and on demand viewing and for producers and editors to create highlight shows.

The planning was massive, coordinating over twenty vendors to work together trying to predict how interested their audience would be in the event overall, not to mention which events and when. While they could hope for a Michael Phelps sweep or American gymnastics success to create demand (and boost ratings), the big unknowns are what make the Olympics so compelling—that is, the “unscripted human emotion” that creates the history and themes that resonate for years after. You just can’t plan for the murder of an American tourist who happened to be the father-in-law of the men’s volleyball coach and then have that team go on to win an improbable gold medal. So much more attention was paid to that team and their games then could have been predicted, and the network had to accommodate delivering the attendant video.

Tomorrow, Cisco and Microsoft execs will take the microphone to discuss, respectively, where next-gen data centers and unified communications will take us.

Oracle and Sun have just announced that Oracle acquired Sun in a $7.4 billion deal.

Well, I’ll be the first to admit that I don’t know what impact this will have on the network – the database stuff usually ends up in the realm of the “developers” and so far all I can tell is that people are panicing over Sun’s Open Source projects, such as Java and MySQL

That is the big question – MySQL has been the go-to database for any project requiring a database, but doesn’t necessarily need to scale.  Used in everything from cloud computing to MythTV installations, the major concern is that the open source MySQL might be killed by Oracle, which makes proprietary databases. 

Now, while MySQL is already open, there are ways to cripple, if not kill, open-source projects once they’ve been acquired – though the best example of that I can give you is when Hasbro released D&D 4. 

MySQL (the company), was, if you recall, acquired by Sun, and similar (unfounded) concerns erupted as no one knew what Sun would be doing with the new intellectual property – and eventually MySQL developers and company officers had to explain what was going on.  Perhaps Sun will have to do so as well.

NetQoS uses MySQL Enterprise edition in our network monitoring and reporting products, so this may affect us, but we don’t quite know how yet – let’s wait for the dust to settle. 

For those attending NetQoS Symposium 2009, we’ve got some information for you. First, check out the Symposium Agenda on the NetQoS Symposium 2009 site for a complete schedule of events.  But here are some highlights:

First, this Sunday at 6:00, the Welcome Reception will begin, as well as the Hands-on Lab, where you can get a chance to install, configure, and deploy our products in a lab environment.

Monday morning will have a welcome address by NetQoS CEO Joel Trammell, but I’m pretty sure the highlight will be astronaut Col. Mike Mullane, who will be presenting “The Normalization of Deviance.” Those expecting social commentary will be disappointed to note that Mullane is, in fact, referring to the concept where lower standards of performance are accepted so often that that lower standard becomes the norm.

Tuesday’s highlights will include a keynote by Lenny Guardino at NBC Universal, talking about his experiences with preparing NBC for the networking demands of high definition live coverage of the Olympics from Beijing.

And Warren Barkley from Microsoft and George Kurian from Cisco – and if you haven’t heard of those two companies, we invite you to Google them – will be delivering keynotes on Wednesday morning.

For live updates, you’re going to want to subscribe to our NetQoSLive twitter feed if you’re not already a subscriber.

We hope to see you there!