by Brian Boyko
Editor, Network Performance Daily

Enterprises are seeing more adoption of 10 gigabit Ethernet according to a report by Network Instruments, and reported on their Network Observations blog that nearly one quarter of businesses are implementing 10G networks by the end of the year. The larger the company, the more likely a 10G rollout.

There’s certainly evidence of a trend, but is that evidence of a need-based demand? LAN technology at the gigabit Ethernet level typically has low latency – and I don’t see 10G Ethernet helping with that much if at all. Gigabit Ethernet is still a heck of a lot of bandwidth, especially compared to the bandwidth offered by WAN solutions. In any LAN/WAN/LAN traffic path, it’s almost always the WAN that proves to be the bottleneck.

But it is possible, with large VoIP networks, that you could be overloading the LAN capacity and decide to move to 10G for that reason. This could possibly explain why big companies are more likely to have 10G than smaller companies – because if you’re not hitting the bottleneck on the LAN, 10G doesn’t really help you deliver the applications any faster or effectively.

What I think is more likely is that 10G has hit a price point where it costs about as much to roll out 10G as it does the older technologies. Instead of 10G taking over the market from companies migrating from 1G, instead it seems that when companies choose to build new systems, they’re choosing to build them in 10G instead of 1G.

But again, it comes down to application delivery. And if we’re not delivering applications faster, the question is then asked – is there any application that is not feasible to execute on a 1G network for which a 10G network would be suitable?

Then I remembered that I’m a geek, and I like my toys.

Specifically, when I move into my new apartment next month, I’ll be back on my own router hardware. My current place has Ethernet built in – it’s a feature that saves me $50 a month, but the complex houses its own routers, which I have no capability to port-forward, which means that I can’t set up a remote desktop connection so that I can check on my home computer from work. And looking forward to being able to do that again reminds me that perhaps one of the new applications that could propel an adoption to 10G might be combining virtualization with remote desktop software – that is, making the end users work from their desk computers on a virtualized environment on a server. This means that you get more life out of older but still usable desktop hardware. According to the FAQ from RealVNC, at 100Mbps per connection, “most tasks will be indistinguishable performed remotely from if they were performed locally” Still, 100Mbps fills up a 1Gbps LAN pretty quickly. However, a 10Gb LAN might be able to accommodate this new application.

There are limitations – anything using full screen video or animation (a movie, or a 3-D environment) where there are rapid changes of every pixel will require even more bandwidth before it gets “choppy” – which will probably sink my plans of playing Half Life 2 on my Mac via a remote desktop connection to a PC. But this is certainly one of those “think about it” half baked ideas that may become reality in the near future.

We’ve mentioned numerous times about broadband penetration and speed lagging behind countries more rural and less populated – in other words, countries the U.S. has no excuse lagging behind.

Ars Technica recently put out an article detailing what differences in national broadband policy exist that have enabled other nations to surpass the U.S.’s broadband capability. Japan and France have local loop unbundling – that allows for more competition among ISPs.  They also both deploy fiber instead of copper even if it doesn’t show an immediate profit, and competing ISPs are rolling out new fiber infrastructure instead of just leasing lines. 

Japan, France, Sweden, and Canada all treat broadband as a “core infrastructure element” – that is, it is treated as vital to the functioning of the national economy as good roads, bridges, tunnels, and electrical grids.

In all fairness, the U.S. can claim the same thing.  The U.S. may have no broadband policy, may be looking to traffic shaping to solve problems that would be better addressed by more fiber rollouts (oh, and by the way, there’s a new $800,000 deep packet inspection device on the market today to help service providers monitor and shape traffic), and may be relying on increasingly obsolete technologies – but at least we treat it the same as we do our roads, bridges, tunnels, and electrical grids. 

Which is to say, not very well at all.  The American Society of Civil Engineers gave the United States infrastructure a “D” in 2005, down from a score of “D+” in 2003 – and to fix those problems would require $1.6 trillion over five years.  Since then, not much has been done, according to this CBS video reposted on RawStory.com.

Instead, the government is considering plans to lease highways to private companies – using tolls to provide a “free market” solution to the infrastructure problem – but which will ultimately be a government sanctioned private monopoly over certain sections of blacktop. It is difficult to see how this would improve infrastructure, rather than simply allowing private companies to charge the maximum people will pay for the minimum infrastructure service people will put up with.

So, as far as treating broadband infrastructure like the rest of America’s infrastructure, it seems we already do that.  But what needs to be clear is that broadband infrastructure is infrastructure – that is, it is just as important for the rural area to get good broadband as it was for them to get good roads back during the Eisenhower administration. 

In a macabre way, this limited broadband is good for vendors; if broadband was plentiful there wouldn’t be so great a demand for WAN Optimization tools, for example.  Sure, WAN Optimization is a good idea anyway but it is the high cost of bandwidth that spurs demand forward.  It is becoming harder to maintain performance not just because of the various new demands on the network but also because the infrastructure across the country is simply inadequate – thus the demand for network performance monitoring tools.  Increasing bandwidth doesn’t always solve the network problem but insufficient bandwidth always creates one.

We've recently covered Bell Canada throttling P2P service. Today, in this podcast, we speak to Professor Michael Geist, Canada Research Chair in Internet and E-commerce Law at the University of Ottawa, regarding the controversial move by Bell Canada to use traffic shaping on wholesale service providers.

A transcript of this podcast will be provided at the earliest opportunity.

Traffic shaping is not a tool of the devil, nor do we believe the solution to bandwidth problems is simply to provision more dark fiber and build more underground fiber optic lines. But as time has gone on, the issues around network neutrality have become more pronounced.

For example, Bell Canada has been throttling P2P service, much like Comcast in the United States. However, what makes this different is that Bell Canada is in a position much like AT&T – in that throttling the network on the backbone affects all the people – including people who are not Bell’s customers – along the line.

Worse still, Bell has been reselling the capacity to provide ADSL service to smaller ISPs without letting the services know that the bandwidth is throttled for certain applications. One of those smaller ISPs, Teksavvy, said: “We are not throttling anything and as far as I am aware will never throttle anyone. We don't believe in it.”– so the idea that Bell will leave them with no choice in the matter is a little worrisome. There isn’t much choice in the matter – the only other big broadband provider in Canada is Rogers Cable, which also throttles traffic.

There are arguments that “net neutrality” will be solved by the forces of the free market – that is, if one ISP throttles, they can go to their competitors. The problem is that, in this case, this is exactly what savvy customers were doing by moving from larger companies, like Bell and Rogers, to smaller companies like Teksavvy. From the consumer, it’s reducing their choice. For the small ISP, it’s could be considered downright anticompetitive, and the Canadian Association of Internet Providers applied for relief before the Canadian Radio-television and Telecommunications Commission that would require Bell Canada to cease and desist.

We contacted Rocky Gaudrault, CEO of Teksavvy Solutions, but because this was now a legal matter, he explained that he was unable to comment. It was clear that he is passionate about the issue, but Teksavvy’s staff keeps him from speaking out by supplying him with timbits and beer to keep his mouth and hands busy.

Particularly interesting is this comment by a Slashdotter – both Bell and Teksavvy charge on a “tiered and metered” basis – which pretty much cuts through the false choice between deep packet inspection and metered bandwidth; Bell has both. (One profanity-laden post implied that the only reason that Bell Canada did this was to coldly eliminate the most compelling competitive advantage that smaller ISPs had – Bell had throttled traffic, small ISPs didn’t.)

The upshot is that network neutrality concerns have been brought to Canada’s Parliament during Question Time. (link via Prof. Michael Geist at the University of Ottowa, who we hope to have an interview with on Monday.) It’s unsurprising because these matters do not just affect consumers but large enterprises as well - an unannounced and sudden change in the QoS policies of the backbone provider is exactly the type of thing that can foul up capacity planning, VoIP switchover, teleconferencing, etc. Especially worrisome are those technology companies who rely on some form or another of P2P traffic to help cut their bandwidth costs.

Deep packet inspection is a powerful tool, and used in the right hands, in the right way, it can help make QoS planning easier, can help streamline business critical applications, can provide overall better end-user response times, and may indeed be a great technological boon.

But we can’t see any benefit in this case for throttling the traffic of resold bandwidth, and for not disclosing the changes in advance. If businesses that control backbone traffic want to avoid governmental regulation, they need to show that they can be responsible with the power they have and use it in a manner which is neither anti-competitive nor deceptive to wholesale resellers and end-user customers.

I'll be frank - I couldn't think of a good idea for an article today. There are a couple of interesting links in the news, of course, which I could share with you. And we will get to those in a second, but…

…truth is, I wanted to get a little introspective about things.

This blog is based on Movable Type v.3, and while we can talk about what I should have done, switching to a different system, such as WordPress really didn't make a whole lot of sense.

The one problem that MT had was that we were getting deluged with spam. Hundreds of spam messages a day.

Now, there is a setting that is supposed to catch junk posts. However, this was worse than useless, as it didn't catch a great deal of the junk messages - and it was classifying some good messages as junk mail. In fact, I think it might have been classifying most good messages as junk mail, which may be one of the reasons that we didn't get many comments on this blog for - oh, the first 16 months or so.

A few weeks ago, we moved to a CAPTCHA based system, using ReCAPTCHA. This has been working well - we've gotten more comments, more frequently, and spam is almost entirely gone. Yeah, I know CAPTCHAs are a pain, but it's the only solution we can think of at this time.

Still, 16 months of bad comment moderation may have discouraged regular readers from becoming regular commenters. What I'd like to ask is that, if you've tried to comment in the past but got discouraged, try it now. And if it still doesn't work, for whatever reason, feel free to send me an e-mail to my work address, brian.boyko@netqos.com. I really could use your suggestions for stories to investigate or issues in technology to talk about.

That said, here are those interesting links:

New York Times: Hopes for Wireless Cities Fade as Internet Providers Pull Out:

EarthLink announced on Feb. 7 that "the operations of the municipal Wi-Fi assets were no longer consistent with the company's strategic direction." Philadelphia officials say they are not sure when or if the promised network will now be completed.

For Cesar DeLaRosa, 15, however, the concern is more specific. He said he was worried about his science project on global warming.

"If we don't have Internet, that means I've got to take the bus to the public library after dark, and around here, that's not always real safe," Cesar said, seated in front of his family's new computer in a gritty section of Hunting Park in North Philadelphia. His family is among the 1,000 or so low-income households that now have free or discounted Wi-Fi access through the city's project, and many of them worry about losing access that they cannot otherwise afford. Philadelphia officials say service will not be disconnected.

"We expect EarthLink to live up to its contract," said Terry Phillis, the city's chief information officer.

The problem was that EarthLink's plans required more routers than they initially predicted, which makes me wonder if those predictions were tested on smaller scales first. However, there is no problem with the technology - it performs as advertised. The problem is that there's no real clear way to make a profit from that technology - which, to me, makes it an ideal service that the municipality should provide, rather than outsourcing it to a private company.

George Ou: Fixing the unfairness of TCP congestion control:

George Ou at ZDNet claims that "swarming" is causing a significant bandwidth problem, and goes to great lengths to explain why, in a four page article.

Simply by opening up 10 to 100 TCP streams, P2P applications can grab 10 to 100 times more bandwidth than a traditional single-stream application under a congested Internet link. Since all networks have a bottleneck somewhere, a small percentage of Internet users utilizing P2P can hog the vast majority of resources at the expense of other users. The following diagram illustrates the multi-stream exploit in action where User A hogs more and more bandwidth over User B by opening more and more TCP streams.

But as I read it, it seemed a bit dubious to me. After all, if my multiple TCP-stream connection on my home computer allowed me to have multiple bandwidth links, wouldn't that mean that a download on BitTorrent of a Linux distribution operating at max capacity would be faster than a single TCP stream and FTP connection to a server? In practice, I've found that both speeds are roughly equal - except when there's a lot of demand on the server side; like the first few days after a new Ubuntu version comes out. Then the multiple TCP streams come in handy because they are coming from multiple TCP connections to different locations. But it's impossible for the multiple TCP connections to take up more bandwidth than had been allocated by the service provider under their QoS policies.

Where there is some validation to this is when the pipe gets completely congested to the point that the available bandwidth per user is less than the bandwidth allocation provided by the ISP to the individual users. In other words, it only occurs when the provider has under-provisioned for the network demand and is delivering less than promised to begin with.

Ou suggests an update to the TCP/IP stack that prevents this problem, but for ISPs, the solution is simpler. Either add more bandwidth so that you can deliver the service promised, or promise less if you can't deliver.

PC World: Tech Workers favor McCain, Obama:

Not getting into politics, but this little fact from the article is interesting:

The survey, of 600 self-identified IT workers, found that 27 percent have used the Internet to contribute to a political campaign. By comparison, less than 0.3 percent of U.S. residents have contributed more than US$200 to a U.S. political campaign during the 2008 election cycle.

Which implies that the techies, who by definition are likely to be Internet savvy, are highly politically motivated and therefore very interested in events from the 2008 presidential race.

Hmm, did you experience a bump in recreational network traffic around the time of Obama's speech on race?

We're releasing NetQoS VoIP Monitor v1.1. Biggest changes: SIP (Session Initiation Protocol) support, automatic and on-demand problem investigation, and capacity planning reports.

I want to start with SIP support, because there's an interesting related story that caught my attention when it came out on Slashdot.

One of the odd things about SIP is that it is, to some extent, a peer-to peer based protocol. The advantage of this is that it only requires a simple core network, with all the fiddly bits distributed to the network edge. This makes SIP more scalable than other protocols. You can see why our customers think SIP support is important and why NetQoS worked to put it into this release.

But as a side effect of the way the technology was designed, SIP's peer to peer network means that it can be difficult to route emergency calls because of the mobility of IP endpoints and the fact that SIP has no network location capability - you'll remember that Vonage got into a little bit of trouble a while back because it couldn't consistently promise E911 support. (That has since been fixed.)

SIP also establishes a VoIP connection directly between the two calls out at the edge. Once the call is set up, the data does not pass through any sort of central server owned or controlled by the VoIP provider. That makes it harder for the government to legally (or whatever) intercept calls.

I mention this because the actual documents governing the rules behind U.S. government interception of VoIP was leaked to Wikileaks on the 15th of March. Now, this is nothing new - CALEA requires VoIP providers to maintain wiretapping capability - just like the plain-old-telephone-service providers are. It's interesting, however, to see the documents. Or at least it might be to somebody else who is interested in network security and encryption.

But from a performance angle, the CALEA requirements for wiretapping are directly in opposition to the efficiency of a SIP VoIP network - that is, if a service provider must be in the middle of every call, it eliminates the benefits of the P2P structure. That adds a lot of network overhead.

The other new features in VoIP Monitor v1.1 are generally less conversation sparking - but no less important. Most of our other products, such as SuperAgent, have both automatic and on-demand problem investigation and capacity planning reports. These capabilities have been added to VoIP monitor in the new version.

Automatic investigations occur when a VoIP performance threshold - such as delay to dial tone - is exceeded. Then VoIP Monitor traces the call signaling path and compares it to the automatically generated baselines.

As far as capacity planning reports helps go, VoIP Monitor v1.1 providers enhanced reports on call volume, call quality, call failures, grade of service and gateway utilization. It provides a view of the effect of different call volumes. With this information, IT professionals can view capacity for specific locations or gateways or for the enterprise as a whole - the utilization reports are especially useful when negotiating contracts with service providers.

We have a demo of VoIP Monitor v1.1 up and running at VoiceCon in Orlando, at booth #1305, if you're attending.

Verizon and NBC are working on serving up TV shows to home computers. The problem is, high definition video, (and I've done some HD video work for the Web - shameless plug), takes a whole mess of bandwidth.

Now, the obvious solution for NBC would be to move to some sort of peer-to-peer distribution system, right? I mean, it works for Linux distros.

The problem is that a normal peer-to-peer connection doesn't distinguish between the cheap local links - that is, links on the same ISP, in roughly the same geographic area - from the expensive remote links. So while P2P provides a more cost effective solution, it doesn't provide the most cost-effective solution for the ISP.

A third party, Pando, has developed a P2P system for pre-authorized, pre-approved content, and has come up with a way to force peer to peer connections to look for local nodes first. This increase the efficiency of the system, lowers the cost, and generally increases the performance of the streaming/downloading video.

This is exactly the type of thing we talk about when we say that how the application is coded can have a huge impact on the application performance over the WAN. Sometimes instead of needing more bandwidth, you need to find a way to make the apps work more efficiently.

In this case, decentralized P2P systems developed after the destruction of Napster. Though they were much less likely to get shut down by the RIAA, they were also much less efficient. This dominated development of P2P applications for years. But for offering only pre-authorized content, a centralized system - especially one that takes advantage of the structure of the physical network, makes a certain bit of sense.

NBC will be offering Verizon customers their shows via Pando's P2P service - which they're calling P4P, later this year. The name is a logical outgrowth, P2P, or "peer to peer," versus P4P, or "peer for peer." P3P was disregarded because it sounded too much like PCP. And if a kid with a lisp goes around school saying: "I downloaded the latesth Methallica album on P3P" and a teacher hears: "I downloaded the latest Metallica album on PCP," well, that's just not going to be a story that ends well, now, is it?

There's only one problem with Pando's plan: Each ISP will have to give up information about its subscribers in order to participate - that is, the Pando platform requires knowing which nodes are "local" and which nodes are "remote" in order to optimize for the local connections:

For other ISPs to reap the benefits Verizon did in the test, they too would have to share information about their networks with file-sharing companies, and that they normally keep that information close to their chests.

''That's one of the objectives we have to solve -- how are we going to consolidate this data and distribute it?'' Pasko said, adding that the result of the test gives ISPs plenty of incentive to collaborate.

(Okay, maybe there's two problems: No offense to NBC, but when your biggest hit is a veritable case study in game theory… you need some new shows.)

by Brian Boyko
Editor, Network Performance Daily

The release of Adobe AIR today might just bring about major changes - both good and bad - for network performance. AIR is a way to produce Web apps that can be run as desktop apps. It is cross-platform and relies, like Java, on a just-in-time compiler and an interpreter of application bytecode. There are interpreters for Windows and OSX, and a Linux interpreter in development.

"It allows Web application developers - or just application developers - to use the Internet technologies they know, whether it's Flex and ActionScript to target the Flash part of AIR, or Javascript/HTML/CSS to target the AJAX part of AIR," said Phil Costa, director of product management at Adobe. "It allows them to take those applications and run them on the desktop."

Costa explained that through AIR, (depending on what the application does and how it is coded,) companies may theoretically experience a lowered amount of data throughput and an improved network performance.

"Today a huge number of corporate networks are moving towards browser based applications, and one of the extra bandwidth requirements that it puts upon the network is that every time you access a [Web based] application, you need to download it. Whether that's HTML or Javascript, or all kinds of Flex and Flash content, that needs to be pulled over the network. Having the application installed locally avoids that. All that will be going forth is the actual data that you're trying to access."

"We've done tests with some of our customers where they've seen our bandwidth [usage] go down for Internet applications in general, because unlike a Web site, which creates both the content and the formatting of the content, most AIR apps are just passing the information back and forth instead of refreshing the page each time."

"Now, depending on what the application does, it may actually add [to] bandwidth requirements for the network as well. One of the things that applications do, is run in the background and connect permanently to a data source's real time streams, or frequently check for data. That could increase the bandwidth requirements. But that's more about what the application specifically does than anything specific about AIR."

AIR's capabilities allow for offline usage as well, which will likely prompt more demand for online apps as the major drawback of SAAS - inaccessibility - is mitigated.

"In addition to giving the developers and then end-user of the application the convenience of launching the [Web] application like any other desktop application," said Costa, "it gives them additional capabilities that they didn't have when they were targeting the browser, such as local storage, either in flat-files or structured storage like a SQL database, which is embedded in there, or drag-and-drop integration with the file system, and cut-and-paste as well as the ability to take data or content offline, and run it when they're on an airplane or just not connected to the network."

"The runtime provides a whole set of APIs for notifying the application when it is on and offline, and so the developer can implement behavior that accounts for that; in many cases what we see is that the developers are caching some of the information offline, so that if the user takes it offline, it will still be available."

"To give you an example… one of our customers, Anthropologie, built an online catalog that lets people browse through things they have, and they built an AIR version which lets customers make little notes to themselves about the product, and rather than store them on the Anthropologie Web site, it stores them locally. The customer can put notes on things the same way they put stickie notes on an actual physical catalog, and they don't have to share that information with the Web site, so it's private to them. It also means, from Anthropologie's standpoint, that they don't have to create massive databases to store that information."

Costa said that Adobe hopes that there will be AIR apps on mobile phones, something that there's no specific date on, but which is on the Adobe roadmap.

CIA Plan #328 to remove Castro from power in Cuba has succeeded. This plan, also known as "wait until El Presidente gets old and retires," now suggests that there will be vast geopolitical changes.

However, while Castro may be retiring, the communist regime he's supported for nearly half a century isn't going away, and whoever wins the elections in Cuba, the policies vis-à-vis the U.S. probably won't change that much until and unless the U.S. foreign policy towards Cuba drastically changes after the next U.S. election.

Which is a shame, really, because when old hardware (I have no problem calling Castro "old hardware,") is too troublesome to maintain than to replace, it's usually a good time to re-evaluate the way that your company, or, in some cases, your tropical island nation, does business.

Now, whether Castro is retiring because of his natural health, or because the CIA slipped him a cigar with a really, really slow acting poison back in 1963, it doesn't matter. The point is that we are so used to the way things are, we often don't pause to consider what could be. We don't understand the true cost of maintaining legacy hardware versus trying the new, and don't give much thought to better alternatives if "things are working well enough."

IT professionals have been thwarted many times by using legacy management tools that are outmoded, increasingly and annoyingly cumbersome to maintain and administer, or no longer deliver the value for which they were acquired. Beyond the obvious maintenance and license costs, there are also the opportunity costs related to speed, quality, scalability and efficiency. When you combine high maintenance costs with opportunity costs, the financial penalty for the status quo begins to add up.

Now, no one should advocate tossing out a product or technology just because it's a little long in the tooth- in fact, long, huge, revolutionary solutions when small fixes are the most efficient solution can be a serious problem that takes time and energy away from IT. But, there is nothing wrong with taking time to assess the situation and ask whether we're better served by what is or what could be.

With mysteries abounding about the undersea cables cut in the Middle East, Network Performance Daily talked to Eric Schoonover, a senior analyst at TeleGeography, a market research firm specializing in telecommunications supply, demand, and pricing. We wanted to get to the bottom of what's happening with the undersea cables and widespread network outages, and see if there's any truth to the various rumors floating around.

Network Performance Daily: Could you tell me a little bit about the effects of the undersea Internet cable cuts?

Schoonover: The undersea cables that were cut are part of the global network and in fact a heavily used part of it. And as such when they were cut, it limited the amount of capacity connecting the Middle East to Europe. I'm specifically referring to the cuts on January 30th - the first two. And because of that, the Internet and things that relied on the communications to Europe, you know, phone traffic and business-to-business type communications were severely hampered until the carriers that were affected were able to find alternative routing.

Network Performance Daily: When they were able to find alternative routing - was that immediate? Did the traffic find they couldn't get connectivity and just routed around it, or did someone have to pull a switch somewhere?

Someone had to pull a switch. With this amount of capacity, in terms of percentage, there's not that level of restoration available on the direct route. So, for instance, I know an affected carrier that has been quite vocal about the things they have done to restore capacity to their customers - even to the point of having to enter into some short-term contracts to transit traffic around the other side of the world, you know, via India, Sinagpore, Japan to the U.S.

So it does take a little bit of time. And each carrier that was affected responded a little differently in a little different time as well. So anywhere between a few hours to a few days to get service back, depending on the type of carrier and their relationships with the wholesale providers in the area.

Network Performance Daily: Has this increased network latency for those kind of connections?

Schoonover: Absolutely. Two kinds of factors increasing the latency - anytime you go the other way around the world from the Middle East, it's going to add a little bit of distance and distance equals latency, because of that "physics" thing. The other thing is that 75 percent of the capacity connecting the Middle East to Europe was cut, which, when you try to move that type of demand around, then you're going to create congestion on the remaining line.

Between those two factors there is a higher amount of latency and it does take some creativity on the part of the carriers to keep their business customers operating and keep their voice calls at a higher performance level.

The thing to suffer the most would be the Internet. Because that's not as latency sensitive as voice or real-time business communications, the carriers allow it to be more affected by the problems than the other services.

Network Performance Daily: Is there any basis for any sort of conspiracy theory here at all?

Schoonover: No, I don't think so, really. Cables are damaged with relative frequency, and I think that this is more along the lines of coincidence that there were a few different incidences within a couple days than anything else.

Network Performance Daily: What about the two main lines?

Schoonover: Well the two main lines were close enough that it probably was the same event. Whatever cut one most likely cut the other one as well. I know that the initial speculation was that a ship had dragged anchor across the two cables which would very easily snap them. That was later refuted by the Egyptian regulator. You can then look at things like seismic or geological events, something like that.

But most likely because those cables went down together, and they were so close - most likely that's one event.

The other cable breaks in the gulf - there's two others - were separate events that happened within a few days of the initial one.

Network Performance Daily: So if this stuff happens all the time, what can companies do to preserve mission-critical network connectivity and performance?

Schoonover: Finding restoration paths and having existing agreements for having restoration in place is very important, and many carriers have diversity in route and upstream providers, as well as the option to exercise a backup plan. And as we've seen, even if it takes a few hours to a day to get things back up and running with some amount of regularity, that's a result of having these pre-existing redundant relationships available.

I think businesses are getting smarter about that and I think carriers as well, particularly after the Taiwan Earthquake from December [2006], that cut a significant amount of capacity in the inter-Asia region. A lot of businesses quickly realized that their disaster-recovery plans were not sufficient, and went about getting better ones.

The fact that businesses have been able to recover relatively speedily is indicative of good planning to a large extent.

Network Performance Daily: How important are these sea cables to global communication?

Schoonover: Very. A lot of people don't realize, but undersea cables are the backbone of the global communications network. Obviously Europe has a lot of terrestrial cable as does the U.S., but as soon as you need to cross an ocean, the bulk of the traffic is travelling via submarine cable, not satellite.

Network Performance Daily: Well, why couldn't we just use satellite?

Schoonover: Higher latency, less capacity, and more expensive.

Network Performance Daily: What's the most important thing that people are learning from this incident?

Schoonover: I think there's a fragility to any sort of infrastructure, and I think you can take away that businesses and carriers do need to prepare for the unexpected. With the Taiwan earthquake taking seven of eight cables, and this taking two of three on a particular route, there has to be physical redundancy, both geographical and capacity.

But that being said, the carriers knew that and they're working towards it. There's at least four cables being planned and built on the exact same route that the cables that were cut are on. It'll be another year or two until the new cables are operational, but the demand for this type of thing was known and is being addressed, it's just that the timeline didn't work in the favor of the Internet users in the Middle East.

Network Performance Daily: The whole thing - just to get this whole conspiracy thing out of the way - what would it actually take to knock Iran's communication infrastructure off the Internet?

Schoonover: Well, it would take a lot more than what's been done. Really, when you look at Iran's connectivity, while they have been affected by the cable cuts, they are not the most affected country. They have terrestrial connections to surrounding countries, satellite connectivity, and redundant submarine connectivity.

Really, what's been done, if it were a directed attack, it has not been particularly effective.

• Network World: What those oceanic cable cuts mean to you.


• Wired/Threat Level: Cable Cut Fever Grips the Web

How is your company weathering the cable cuts? If you've been affected, let us know how you got back up and running by leaving us a comment below.