We’ve produced a follow-up to our earlier “The Network Company” video, this time looking at LAN vs. WAN application coding, TCP Slow Start, and WAN Optimization. Instead of giving you a detailed run-down, I’m just going to go ahead and embed it right here.
I love any day when I get to smash citrus with a large blunt object at work...
Hello. Is there anybody in there? Just nod if you can hear me. Is there anyone home?
Network World, via Slashdot, has a story by John Cox talking about how a number of companies with Wireless LANs are finding that as Wi-Fi became more prevalent, they don’t actually use wired networking equipment quite as much; in fact, the article says that in these companies, “50% to 90% or more of Ethernet ports now go unused”
So the question is put to the reader: is it time to disconnect those Ethernet wires from our veins and go to Wi-Fi rehab?
Probably not. Even under the best conditions, wireless links typically lose more packets than wired links, and “best conditions” under WiFi usually aren’t real-world conditions. Intereference from phones, radios, and pesky little obstacles like walls can degrade or corrupt signals, causing dropped packets, increasing retransmissions, increasing latency. And there’s also the delay increased with the various wireless encryption schemes. (Of course, you could just not encrypt your over-the-air traffic… but I don’t think that story will have a happy ending.)
There are other concerns as well, with reliability and security, but performance is also an issue. As far as browsing the Web goes, WiFi is fine, and the advantages and economies of scale may mean some cutting back and replacing wires with waves. But choosing between wired and wireless Ethernet should depend on the application’s needs. If you need 24/7 reliability, you’re looking at wired connections.
Most of the examples in the article deal with colleges rolling out networks for student dorms, and students, after all, prize mobility and are more likely to have laptops than the desktop-based solutions of most corporate users. Even so, I think that the physics labs will want a port or two around, you know, just in case CERN decides to send all the data they get from the Higgs-Boson to research universities around the world.
"The driver always comes down to economics, in my experience," St. Bonaventure's Kellogg says. "That's almost always the reason something either gets axed or pushed through. If you can't show the benefit for the extra cost, it just isn't going to happen."
At any rate, whether going with wired or wireless connections, it’s important to monitor the impact of any changes you make to the network and to baseline performance before you make the change. Before figuring out how much it would cost to install new wireless switches, it pays to check out the performance of the current network.
Thanks to Slashdot, I think I’ve found the source of the badly mangled story from Australia I reported from yesterday: Nemertes Research, which is quoted in this Times Online article. (It would have been easier had Seven News just mentioned the name of the research group in their original coverage.)
And it’s a shame too, because it deserves to be taken more seriously than “I thought Cyberspace just came from the air” – now that the original study has been located, it is, in fact, serious.
To sum up: Nemertes believes that predicted demand for bandwidth will outstrip the predicted capacity of the Internet infrastructure, and will do so within the next couple of years, making the Internet unsuitable for business purposes.
Because the idea of an “exaflood” has been used before by non-independent think tanks, paid by vested interests before, I approached the idea with some skepticism, but Nemertes Research knows what they’re doing, and as far as I can tell, did a very methodical job.
Granted, Nemertes, like most think tanks, is in the future predicting business to some extent, and even if they’re coming to the best conclusion that can be obtained from the information they have now, there is no such thing as an unavoidable problem with network infrastructure. In fact, this is a problem with an obvious solution: Increase the capacity of Internet infrastructure to keep up with the demand. If demand increases exponentially and investment in infrastructure increases linearly, then yes, it’s time for exponential investment in infrastructure.
I’ll leave future prediction to Karnak the Magnificant. But what I can say is that the possibility of Internet brownouts just serves as a reminder not to place all of your infrastructure out on the cloud – that leased lines and private WANs that your company can monitor and control from end to end make it easier to retain and improve network performance than relying on the Internet.
There are several misconceptions about the Internet. For example, some believe that that the Internet is a very light black box about the size of a handbag with a single light, where the Elders of the Internet keep watch over it from Big Ben.
I never get tired of linking to that video. In fact, the entire TV show “The IT crowd” is based around the idea that to most of the world, networking is a mystery, the Internet a “magic” box that gives you sneezing pandas on demand, and no one cares. Until, of course, it’s time to PANIC!
In Australia, Channel Seven’s “Sunrise” morning news show had a computer expert with, Pete Blasina, talking about how the Internet is “filling up.”
According to the news report, a report where the anchor freely admits that he “doesn’t know how this Internet business works, I thought it just went through the air,” I “learned” the following things:
At the risk of insulting Australia Channel Seven’s news staff, this is frozen concentrated stupid juice.
Seriously, if this wasn’t an actual news show, it would sound like an Onion skit.
The segment goes on for four minutes and in that time, the expert asserts the Internet will become “full” but doesn’t quite explain why, blaming YouTube and video applications for the problem, then blaming server storage space for the problem, then blaming problems with delay in the switches, then goes back full circle and talks about YouTube and other video applications and the massive amounts of data they deal with.
Now, as a network performance vendor, one of the reasons we have an end-to-end solution and integrated suite of tools is because we’ve heard of situations where problems are first blamed on the network group, then on the server group, then on the application group. However, this is the first time I’ve seen a single person flip single-handedly from one to the other to the next. And, like adding a marichino cherry on this banana split of confusion, he does it all without actually mentioning what the problem is.
He doesn’t even mention who is saying the Internet is getting full by 2012. “Who” is literally the first thing taught in journalism school, followed by the next five things, “What, When, Where, Why, and How.”
It is bad enough that there is so much complex information out there that is misunderstood and mishandled. Oversimplification is one thing, but at least oversimplification tells you what the problem is and a simple, easy to understand, wrong solution. I don’t even know what the hell the problem is or what they’re trying to say. I’d offer to summarize and re-write it to explain the problem, but I don’t even have enough information to even do that.
But I can get outraged about it because I know enough about computers to know that that was pure nonsense; what worries me is that “the internet will get full” becomes conventional wisdom because that’s all that the non-savvy TV viewer will take away from this piece. After all, Blasina must know what he’s talking about, because “he’s on television.”
And while I’ve never been on TV, I have acted in Pete Blasina’s capacity, not just as editor of this blog and contributor to HardOCP. I’ve also been a tech expert on the Marcus Lush radio programme in New Zealand, and have been interviewed by the New York Times TechTalk podcast.
Maybe I need a pink Hawaiian shirt.
People have a tendency to strongly prefer avoiding losses to acquiring gains. For example, which would make you feel better: getting a $10 discount, or getting a $10 gift card?
If you answered the latter, you’re obviously not human and should report to the nearest alien processing facility where you will be “massaged” in the Dissect-o-topsy 3000.
For those readers not keen on sucking out human brains through a sippy-straw, this phenomenon may explain why network security always tends to be “sexier,” getting more attention, than network management and application performance. The heroes are never the guys who save the company $1M over 5 years, they’re the guys that prevent a $100,000 loss from a malicious hacker.
But there are certain cultural and practical considerations as well. For example, very few people know what “latency” is. Instead, a decision-making executive is more likely to know – or think they know - what a computer “hacker” is. (Let’s not get into the hacker vs. cracker nomenclature debate right now.)
In short, it can be a lot easier to get funding to prevent a potential loss than it is to invest in a known gain.
But there are other considerations too – regulations designed to protect, not the company and it’s assets, but everyone else. Security Expert Bruce Schneier put it this way:
If ChoicePoint has lousy security and someone steals our identity information, we are harmed. But to ChoicePoint, it's an externality. ChoicePoint isn't a charity, and it's not going to improve its security out of the goodness of its heart. If we want ChoicePoint to protect our data, we're going to have to force them. We need to raise the cost of their having lousy security, so it'll be cheaper for them to have good security.
At least, that's the idea behind regulation. Unfortunately, reality isn't nearly as simple as the theory. When you're talking about regulation, the devil is in the details.
With regulation, an emphasis on security over performance starts to make more sense, because while poor network performance can cost a company over the long term, it usually doesn’t have as much of a problem for the general public as a data breach would. It’s just the nature of the business.
Even so, we see more crossover between the performance and security spaces daily. Anomaly detection can be used to detect performance problems as the first tiny bits of evidence pop up – or malicious activity as it occurs. Retrospective analysis provides both performance data and forensic evidence for intrusion.
And of course, if your network is performing poorly, it’s nice to know whether that’s because of short-sightedness or sabotage.
At any rate, maybe we’re a bit too worried about network security when there are other real problems. After all, it wasn’t a hacker that took out Bear Stearns, Fannie Mae, Freddy Mac, Lehman Brothers, or Merill Lynch.
By Stephen Creel
I just completed my 6th annual Symposium as a NetQoS employee and I want to thank our customers for making this another success. I was thinking back to my first one (2004) and there are some pleasing contrasts between that initial experience and what we accomplished this week. The early Symposia were intimate enough to be held inside our offices and we had a few dozen people spend a few days with us. I drove the van. We’ve grown to the point where we are hosting hundreds of customers, have three full days of sessions with four parallel tracks, six keynote presentations, a hands on lab, a usability lab, and a bouncy castle (just kidding on the last one)—all that necessitates a full-fledged conference venue and I don’t think there’s a better one in Austin than Barton Creek.
We believe this year’s Symposium contained the strongest agenda yet. In addition to the sessions presented by NetQoS employees relating best practices, how to’s and product updates, speakers from Microsoft, Cisco and EMC delivered popular presentations ranging from unified communications to unified computing to operations excellence.
Of course, events like these are not possible without customers like you that are dedicated to optimizing how well your organization can deliver applications. For those of you that could not make it to the event this year, we will get the presentations on the self-service portal as soon as possible and we hope to see you at a local workshop. Hopefully the economy will continue to improve and we will see all of you next year in Austin.
I’ve said enough. Here are a few comments from this year’s attendees:
---------
Stephen Creel is the Director of Account Management at NetQoS.
While it’s a bit soon to be revisiting Time Warner’s bandwidth caps, there have been a few developments which make this story more interesting and more relevant.
One of the biggest is the idea that Time Warner had tied planned DOCSIS 3.0 upgrades to the incorporation of bandwidth caps – and that, as Austin and Rochester and the other test markets didn’t accept bandwidth caps, TW would not use Austin and Rochester as testbeds for DOCSIS 3.0.
Gizmodo calls TW “bad losers,” but I’m not sure I agree. While congestion isn’t a problem on current connections, a 50Mbps downstream connection might, indeed, result in some congestion if Time Warner doesn’t make infrastructure upgrades. DOCSIS 3.0 may be cheap, but it probably will lead to increased traffic, requiring more bandwidth on the backend.
I am keen to point out, however, that this is because Time Warner is increasing the amount of bandwidth per customer, not because of increased data. So, I can see why they would tie the upgrade to the increased revenue resulting from bandwidth caps.
Just one little problem: At 50Mbps, you blow through 20GB/hr. If you cap data, there’s really no reason to actually go ahead and get faster service.
So, Time Warner is, if you follow the logic here… putting caps on customers… to pay for upgrades… which customers can’t use… because they’ve put caps on customers…
(There’s a hole in the bucket, Time Warner, Time Warner. There’s a hole in the bucket, Time Warner, a hole.)
Saul Hansell from the New York Times pointed out that backbone Internet providers actually refuse to sell by the gigabyte to ISPs like Time Warner (as well as other businesses,) and that the costs are bandwidth, not data, related. It is fair to assume that increasing bandwidth to customers will require increased bandwidth costs.
But the more interesting thing is that Hansell reveals some of the information about what DOCSIS 3.0 actually is:
“But the last link, running from a neighborhood office or a small device hung on a phone pole—runs over cable TV or phone wires. In a cable system, there is a fixed amount of bandwidth that is shared among all the customers in a node, often about 500 homes.
That capacity, in current technology, provides about 38 megabits per second to share. That means if four homes are all downloading very long files at 10 Mbps, a fifth customer going online, will start to slow down everyone’s connections…
…The other way that cable companies are increasing capacity is by using new technology known as Docsis 3. This is a standard that allows companies to use more video channels for Internet service. The current standard uses one video channel. The first generation of Docsis 3 service combines four 38-Mbps channels into a pool of roughly 152 Mbps that can be divided among customers. Cable companies can decide whether to use that capacity to offer higher speeds to customers or to increase the number of customers who can be served at slower speeds, avoiding the need to split nodes.[Emphasis added]”
Here’s where things get tricky; if you’re limited to 38Mbit/s for 500 homes total, isn’t it a bit irresponsible to offer 15Mbit connections to individual houses?
Imagine, if you will, a bank that leveraged 7.5 billion dollars and had 3.8 million in assets. Such a bank would be… well, actually, it’d probably be pretty typical. So forget that metaphor.
But at 152Mbps, the 500 home ratio starts to feel a little roomier. Even if all 500 homes got online at once, you’d still get 311kbps; or 38kB/s.
Hansell points out that it could get even roomier:
“But most cable systems are in the process of converting to an all-digital format from the current approach that mixes analog signals (which can be watched without a set-top box on an older “cable-ready” television) with digital signals. This is mainly being driven by the need for extra capacity to handle high definition programs. A company can send 10 standard-definition channels or 2 high-definition channels in the space of one analog channel. All that means is that there is not a shortage of channels for use by Internet data, at least for a while.”
This moves Time Warner’s suspension of the DOCSIS 3.0 rollout from the “darn” to the “huh?” category. Even if faster speeds aren’t available right now, wouldn’t it make more sense to keep the current broadband offerings and rollout DOCSIS 3.0 anyway, so that congestion in that last mile won’t be as pronounced?
Now, this next part is pure speculation, but perhaps there’s a method to Time Warner’s madness. (Granted, there’s more “madness” than “method” in here, but follow along anyway.) Verizon and AT&T, using fiber optic technology, offers 50Mbit and 25Mbit connections. This pressures Time Warner to say that they were working on >20Mbit solutions of their own. So in a choice between using DOCSIS 3.0 to offer greater speed to customers or using the new technology to fight congestion, Time Warner decided to do the former.
The only problem is that, just as 7 and 15Mbit connections to 500 houses with 38Mbits of capacity were overselling, so would 20-50Mbit connections to 500 houses with 152Mbits of capacity be overselling. Overselling 7 and 15Mbit connections worked mostly because most customers do not actually use the capacity that they’re being sold. (Even a 300GB/mo “heavy user” like myself only consumes 5% of the capacity he was sold.) But with more customers, more applications, and more demand, that assumption doesn’t necessarily hold true in the future.
If customers actually used the Internet they were sold, there would likely be no way that customers wouldn’t be affected – except if they could be assured that customers couldn’t actually use the bandwidth that Time Warner was offering. Thus, the data caps – set low enough so that Time Warner can sell 50Mbps Internet – like its competitors do – without having to actually deliver the capabilities of 50Mbps Internet – like its competitors do. Without caps, Time Warner’s entire plan for DOCSIS 3.0 falls apart.
Time Warner could, of course, if they invested the time and research into it, convert more analog cable TV channels into increased bandwidth. What might be a better plan for Time Warner is simply to bite the bullet, use DOCSIS 3.0 to relieve congestion rather than increase speed, and increase speed incrementally when the capabilities get there. As the Rochesterians switching to Frontier in droves show, people are willing to sacrifice speed for data; if Time Warner has an uncapped 7Mbit plan while its competitors offer capped plans of any speed, they’ll not want for customers.
Of course, that assumes competition – and the other major problem is that Time Warner is actually trying to lobby North Carolina’s state government to literally outlaw municipal broadband. The city of Wilson – 50 miles east of Raleigh – has figured out how to provide 10 down, 10 up fiber connections for $35, compared to Time Warner’s 10 down, 2 up cable Internet connection for $57, in a program they call “Greenlight.”
Hmm… anyone know how to get something like “Greenlight” started in Austin?
by Patrick Ancipink
There’s been a lot of discussion this week at Symposium about dealing with the daily realities of virtualization. The issues cut across technology and politics and the cost savings from virtualized servers basically guarantee continued virtual machine (VM) adoption and sprawl.
Perhaps the most colorful statement I heard was from a network engineer characterizing how the server and application teams approached him on provisioning more VMs: “Just give me an IP address and stop asking questions.”
Ay caramba!
In some organizations it seems the cost savings of VMs are being used as a blunt instrument to justify rapid VM provisioning and throw process and performance implications aside. The risks of unpleasant surprises and painful performance degradation seem to be rising in parallel with VM growth that eschews planning and testing. After some discussion about the Wild West of VM sprawl that some of our customers are experiencing, a network engineer reflected back on our astronaut’s keynote: “How can we work the plan when there is no plan?”
Luckily we heard some better news in a keynote this morning from George Kurian, VP and GM for Cisco’s Application Delivery Business Unit. It always warms my NetQoS heart to listen to George describe why visibility and management are mandatory components of optimizing the application delivery network, and it was reassuring to hear about what Cisco is doing to address the complexity and headaches of virtualization sprawl.
Specifically, providing “fiber channel over Ethernet” promises to deliver a “lossless fabric” and denser switches and servers can cut in half the amount of network connections you have to manage in the virtual data center. With this type of simplification and efficiency--and the attendant savings in power and cabling and labor—virtualization can scale more gracefully than it does today. Working with standards groups to keep the virtual stack as open as possible sounds like the right path. (I know, I know, standards groups have a pretty poor track record in networking and management, but in this situation I think it’s a better approach than building a brittle, more proprietary stack that inhibits flexibility.)
George talked about some cool new stuff like Priority Flow Control and Virtual Network Link coming in the Nexus product line, but I can’t really do justice to it and you can read about that on your own.
Now, where’s the IP address I asked for?
By Patrick Ancipink
After CEO Joel Trammell’s welcome address yesterday morning, Colonel Mike Mullane, a veteran NASA astronaut with several Space Shuttle missions under his belt, reminded us of the importance of “planning the work, and working the plan” and of the dire consequences of not doing that.
Applying that principle to managing the network for application delivery is a bit less dramatic than space travel, but the importance of teamwork, responsibility and not falling prey to the “normalization of deviance” translates easily enough.
The normalization of deviance was particularly interesting for me to ponder. How many times have I been part of something where a compromise was made and we all said “just this once and never again,” but sure enough that exception becomes part of the standard operating procedure?
The teamwork theme carried through this morning when three gentlemen from NBC Universal told a very impressive story about delivering 3.4 petabytes of video over 17 days from the Beijing Olympics back to NBC headquarters at 30 Rock in Manhattan. (There are 1024 terabytes in a petabyte.) The video had to be available for live and on demand viewing and for producers and editors to create highlight shows.
The planning was massive, coordinating over twenty vendors to work together trying to predict how interested their audience would be in the event overall, not to mention which events and when. While they could hope for a Michael Phelps sweep or American gymnastics success to create demand (and boost ratings), the big unknowns are what make the Olympics so compelling—that is, the “unscripted human emotion” that creates the history and themes that resonate for years after. You just can’t plan for the murder of an American tourist who happened to be the father-in-law of the men’s volleyball coach and then have that team go on to win an improbable gold medal. So much more attention was paid to that team and their games then could have been predicted, and the network had to accommodate delivering the attendant video.
Tomorrow, Cisco and Microsoft execs will take the microphone to discuss, respectively, where next-gen data centers and unified communications will take us.
Oracle and Sun have just announced that Oracle acquired Sun in a $7.4 billion deal.
Well, I’ll be the first to admit that I don’t know what impact this will have on the network – the database stuff usually ends up in the realm of the “developers” and so far all I can tell is that people are panicing over Sun’s Open Source projects, such as Java and MySQL
That is the big question – MySQL has been the go-to database for any project requiring a database, but doesn’t necessarily need to scale. Used in everything from cloud computing to MythTV installations, the major concern is that the open source MySQL might be killed by Oracle, which makes proprietary databases.
Now, while MySQL is already open, there are ways to cripple, if not kill, open-source projects once they’ve been acquired – though the best example of that I can give you is when Hasbro released D&D 4.
MySQL (the company), was, if you recall, acquired by Sun, and similar (unfounded) concerns erupted as no one knew what Sun would be doing with the new intellectual property – and eventually MySQL developers and company officers had to explain what was going on. Perhaps Sun will have to do so as well.
NetQoS uses MySQL Enterprise edition in our network monitoring and reporting products, so this may affect us, but we don’t quite know how yet – let’s wait for the dust to settle.
