Here’s one of the big problems with experimenting to find a faster, more robust Internet architecture: Since there isn’t another computer network out there as big as the Internet, when you want to make grand changes to the way the Internet works, what do you test it out on before deploying?

This is a real problem, because a single accident taking out a trunk line can cause significant disruption – like, for example, those undersea cables cut by ship anchors.

Bennett Daviss at New Scientist wrote about these problems in an article entitled “Building a crash-proof Internet.” The main problem: While the Internet was designed to route around damaged lines and offline servers, the routers we have today are relatively slow at finding new paths. Improving all of the Internet’s routers individually with newer software to find paths quicker would cost billions, if not trillions. While the million dollar losses incurred by Internet outages are not small change, compared to the costs of upgrading, living with the temporary losses seems a viable option.

Additionally, even if the money could be put up front, any “improvements” may actually break stuff, creating the problem that the solution is supposed to prevent. You’d test before deployment if we were talking about a smaller scale enterprise network, but this is the Internet we’re talking about. There just isn’t any other network out there big enough to properly test for scale.

At least, not yet. The National Science Foundation is building an “at-scale” computer network for testing Internet technologies. Part of that network is an open-source program called OpenFlow, hosted at Stanford University, which can be added to standard routers, creating a remote interface to the router's flow table - the rules for handling traffic.

Having the ability to handle flow tables remotely may be a bit of a security risk, but it means that you can let human beings – or supercomputers designed to do nothing but crunch the numbers on routing tables – make routing decisions and send them back to the router. (I wonder if, theoretically, you could use data that one router gathers to help other routers make routing decisions faster.)

Additionally, OpenFlow can be used to spread communication between two specific computers over multiple paths, which may increase reliability and spread congestion more evenly. It can also be used to sort different types of traffic to use different paths – giving VoIP traffic to the lowest-latency path, and, say, non-streaming movie downloads to the largest-throughput path… or giving e-mails more priority than MP3 files.

This just seems like an interesting piece of technology which may impact the way that we observe Internet traffic in the mid-to-long term – so it might be worthwhile to keep an eye on it.