Going to Cisco Live? Check out these sessions on how to guarantee optimized services in virtual environments.

By Denise Dubie

Network managers in the know realize they must master the art of optimized application performance just as they conquered Cisco router configuration. The application performance related job duty fell in the laps of network gurus years ago when it became clear that the network wasn’t always to blame for poor application performance -- but that network engineers possibly held the best perspective on how to optimize bandwidth and other network resources to ensure business-critical apps performed as expected.

Poor application performance to blame for lost productivity, increased costs

Next week at Cisco Live attendees will get a chance to learn more about how to manage application performance from the network perspective.

Continue reading "Application performance, network engineers and Cisco Live" »

A study by L.E.K. consulting, a business strategy and marketing consulting firm, recently conducted a survey on media consumption habits; and what they found turned out to be a bit of a shock. 

According to the survey, 32% of users listen to an average of 5.8 hours of Internet radio a week.  That’s huge.

The reason it is huge is because unlike a lot of high-throughput downloads; streaming radio tends to be a constant drain on bandwidth.  Sure, a 5GB file is a lot to transfer, but it saturates the available bandwidth in the pipe for a limited amount of time.  On the other hand, 160mbps streaming audio improperly configured into a high QoS priority knocks out 160mbps of your total bandwidth.  Multiply that stream by the number of users streaming; and you can see why a new interest in streaming Internet radio is something to take note of. 

The other major thing from the report worth mentioning is that “e-readers,” like the Kindle, have been encouraging people to consume more written words – digitally downloading them.  While only 10% of consumers own e-readers, 48% of those who do report reading more books, and only 7% decreased their book reading.  This also extended to magazines and newspapers as well. 

Why?  According to the survey, 40% said that e-books are more affordable, and that drives their consumption, while 47% indicated that “more interesting books were being released.”  Considering that the publishing industry hasn’t hit a halcyon boom, it’s more likely that e-books enable readers to more easily find books they would be interested in, through searching, recommendations, etc.    

What’s interesting about this shift is that we’re increasingly in a world where if the medium can be digital, the medium will be digital.  And this requires thinking about the network in ways that a lot of enterprises haven’t thought about the network before. 

It used to be that the network was what enabled computers to talk to each other – it still is, of course – but the language of computers was dull and uninspired; in the beginning, only business apps transmitted through the network, because the business apps were what the network was used for. 

But to most end users today, the network isn’t just for one limited purpose.  It’s not even for a variety of purposes.  The Internet is the tube which gives us information, entertainment, conversation, and sustenance, in the form of productivity.  It is, quite frankly, the most important thing in many people’s lives. 

So the stewards of the network have to consider that they hold an awesome responsibility, and that it might be time to stop thinking of the network as just a business tool, and instead, think about it as the circulatory system of human culture. 

Hold on – I think I just had a hippie moment brought on by high stress levels and lack of sleep.  It’s okay though.  I’m sure you get the idea. 

Distributed computation has been around a while in different forms – Beowulf clusters, for example, - but Ian Clarke, the developer of Freenet and founder of Revver, has started working on a programming language, based on Scala, called “Swarm,” which he hopes will create a distributed programming language that can run on almost any operating system.

Because it runs on an application level, any computer can be a part of Swarm. You run Swarm on any computer you like, and you can access the computation of other computers running Swarm on the network; or, theoretically, on the public Internet. And Swarm allows a programmer to code an application for multiple CPUs and multiple computers with the same code that you could code for one CPU on one computer.

Now, there are projects such as SETI@Home or Folding@Home which do similar grid-computing tasks, but both are based on a model of breaking up the data to bite-sized chunks, moving that data to individual machines, where the information is processed, and then resending the output back to the central server.

Swarm is trying to flip that on its head. With Swarm, you can run the program wherever the data resides. So if you had a piece of data on Computer A, and a piece of data on Computer B, and you wanted to do a calculation that required both A and B’s data, you wouldn’t need to copy the data over the network – the program would execute on both A and B, returning the result of the calculations on B’s data to Computer A. Swarm is designed to manage which software runs with which data on which computer – without the programmer having to think about it beforehand.

Combine this with the latest advances in dynamic allocation of virtual servers according to need, and you start to really chip away at a whole bunch of scalability problems that have traditionally plagued massively-multi-user-applications… that is, Web apps.

Now, here’s the question: CPU latency is measured in picoseconds. Network latency is measured in milliseconds. The question is: How do you figure out what computations will actually benefit from being offloaded to another computer? – i.e., which computations are so far back in the stack that it would be better for them to go for a round trip across the Ether than to just wait patiently for the stack to clear? It seems to me that network latency monitoring would be very important for such an application.

For example, let’s use some of the NetQoS Network Estimation Tools (shameless plug) to determine how fast we can theoretically get a calculation going over the network. So, figuring a router latency of 0.5 milliseconds on both ends, a server latency of 2ms, a link speed of 64000, and a (very short) link distance of 10 miles – you’re looking at 132 ms of latency altogether – assuming point-to-point protocol.

In that 132ms, a 2.4 GHz quad-core computer can perform 1.26 billion calculations locally. That seems like a lot – and it is. But you actually start saving time once you hit 1.26 billion plus one calculations. For some applications, that might be worth it.

But other than pure speed, there’s another reason to consider running Swarm – and that is that applications coded with Swarm should have the ability to continue running on other servers – preserving the application in the case of fault or insufficient resources on the primary computer.

Right now, Swarm is more theory than fact, and there’s a lot of work to be done before it can be practical. But anything that requires less data to be sent over the network is something to keep an eye one when trying to preserve network performance.

Part 2: Killing Network Performance

So, let's take a look at what Time Warner is doing and compare it to some other offerings. First, Time Warner has four plans, at four speeds with four caps. While a TW spokesman has said that people will be able to mix-and-match data caps with bandwidth plans in Austin, we'll use the Beaumont plan as an example.

Let's make some quick assumptions. First, let's assume that no user wants to go “over” their cap – that is, that whatever they're paying now without a cap, is what they want to pay, and they don't want to pay more. Second, let's assume that TW's prices do not change significantly. Third, let's assume that Time Warner makes good on their promise to bring out a “100 GB” cap for the Austin market. Considering the rate changes in Beaumont, these are probably good assumptions to make. 

Time Warner Cable Vs. The 56.6k Modem. 

Time Warner advertises speeds of 1920kB/s, 1024kB/s, 640kB/s and 96kB/s respectively for it's four tiers of bandwidth service.

However, remaining under the cap means that, on average, you only have an effective speed of 250bytes/sec under a 5 gig plan, 500bytes/sec under a 10 GB plan, 1.01kB/sec under a 20 gig plan, 2.02kB/sec under a 40 gig plan – the largest possible plan in Beaumont – and 5.06kB/sec under a theoretical 100GB plan.

5.06kB/s... versus an advertised 1920kB/s. Now would be the perfect time to point out that the top speed of a 56.6k modem is 7.08kB/sec.

Time Warner Cable Vs. “The Average U.K. Teen”

Now, you may argue with me that these are only averaged numbers, and that a 5.0kB/s might very well mean a burst of 50kB/s for one second, followed by 0kB/s for nine more.  Quite right. 

You may also suggest that “no one uses the Internet 24 hours a day,” and aside from 24 hour coffeeshops, work-in-shift development teams, roommates with day-shift and night-shift jobs, and professional networking bloggers, I would also agree with you. 

So, how much does the average Internet user actually use the Internet? I couldn't find that after some weak Googling. However, I was able to find out that the average U.K. Teen uses the Internet for 31 hours a week . I like using that number as a base, because it seems adequately conservative when trying to measure the amount that any particular household – perhaps filled with more than one teen – will use the Internet.

But even averaging out the numbers over 31 hours a week, rather than over a full month, it's still disappointing. The 56.6k modem still beats out the 5, 10, and 20 gigabyte plans at 1.47kB/s, 2.04kB/s, and 5.87kB/s. The 40 gig plan does not fare much better at 11.74kB/s – certainly beating out the modem, but not in any way being a speed you'd call “broadband.” The theoretical 100 GB cap results in 29.36kB/s average, which sounds good to a person just now getting off dial-up, but even the lowest Time Warner broadband speed is advertised at 96kB/s; the top tier, once again, is advertised at 1920kb/s.

Oh, remember one of the tricky things about using the “average” U.K. teen as the benchmark.  That means that about half of U.K. teens likely use more than 31 hours a week. 

Time Warner Cable vs. The Ghost of Internet Past

In fact, the speeds only start to become competitive if you use the Internet about 5.5 hours a week.

At 5.5 hours a week, you can get 66.20kB/s average without going over your 40GB cap, 165.49 for the theoretical 100GB cap. 20, 10, and 5 GB caps clock in at 33.10kB/s, 16.55kb/s, and 8.27kb/s respectively. None of these speeds are anywhere close to what we've become accustomed to from “broadband,” and none of these speeds are acceptable – the 5GB plan barely better than a 56.6k modem even at the highly reduced Internet usage of 5.5 hours a week.

You may ask yourself why I chose “5.5 hours a week,” specifically, as my benchmark. That's because 5.5 hours a week is the amount the average American or Canadian spent on the Internet in 1996!

I think that that’s a particularly revealing number.  After all, in order to stay under the cap, many (if not most) people will have to use the Internet less.  But what these caps do is limit TWC customers to the habits of the Internet’s early days, before YouTube, Hulu, Netflix, Skype or Vonage. 

What data caps actually do is take us back more than a decade in our cultural and technological development of the Internet. It is an attempt by Time Warner to stuff the genie back in the bottle – to cripple the Internet as a possible competitor to the television (and cable telephone) business by causing us to curb our habits back to the “bad old days” of the 1990s... and, of course, to grab an awful lot of money from those of us who simply can’t bear living in the past. 

Any customer who finds any of these bandwidth plans “acceptable for what they do on the Internet” should, instead, look into much cheaper, less stressful dial-up Internet service.

The counterargument for charging for consumption.

You could ask the question: that if we pay for gasoline, electricity and water based on what we consume, why shouldn’t we charge for the Internet the same way?  But this is based on a logical fallacy. 

We already pay for what we consume on the Internet.  We pay for bandwidth.  A 728kbps connection from Time Warner costs $30, while a 15Mbps connection from Time Warner costs $60.  And this is fine because bandwidth, not data, is the limited resource. 

Allow me to explain for those in our audience (Hi, Mom!) not familiar with networking technology:  When you lay down a network "pipe," that pipe can only send so many bits a second - this is called "bandwidth." When everyone wants to use the pipe at once, "congestion" occurs and not everyone can get all the bandwidth that they want. 

So what most Internet companies do is charge you based on the amount of bandwidth you consume. They do this by putting you into one of a number of "tiers," or "speeds." If you're willing to pay more for more bandwidth, then you get more bandwidth. This is how Time Warner – and practically every other broadband company in the United States – does things now. 

But data - data is not a limited resource. Unlike water, gasoline, electricity, there is literally an infinite amount of data over the Internet.

Grabbing data does not reduce the amount of data available to others - even though this is essentially the argument cable companies put forth when they try to blame "heavy users" for the need for data caps. What Time Warner is trying to do is create an artificial scarcity where none exists, and then charge for it.

The electric company, in essence, charges you, indirectly, for the limited resource of the coal, oil, uranium, etc, to fire the plant.  Even with wind power, you’re paying for the limited resource of wind that can be captured by windmills – wind may be limitless, but we can’t currently use 99.99% of it. We can only use the 00.01% that runs into the windmill. 

I didn't see “Quantum of Solace,” but if this helps you to wrap your head around it, what Quantum was trying to do with the Bolivian water supply, Time Warner is trying to do with our “data supply.” (Why yes, data caps are a move worthy of a Bond villain.

Time Warner vs. Comcast

Comcast also has a 250 GB cap.  No one is thrilled about it, but it is certainly more reasonable than a 40 or 100 GB cap, by far. At 250GB, the effective performance hit is 73.41kb/s for the 31-hour user. This is still poor compared to uncapped services, but at least, at 250 GB, most people won't have to worry about how much Internet they're using.

That is another thing – when you go over your 250GB cap on Comcast, you either get a warning notice, or you get your service cut – the latter certainly annoying, but at least you can go to another service. When you go over your Time Warner cap, you're being charged, GB by GB.

Because the caps are so low, it's pretty clear that that is exactly what Time Warner intends to have happen – that is, the system designs to place you into overages.

And in this system, you pay for every bit. Every annoying video ad that you don't want. Every zombie malware host probing your connection. Every pop-up ad. Even, dare I say it, every Rick-Roll.

We take a look at the costs that Time Warner customers will have to eat under the new plan in Part Three of this article.

This is the penultimate punch line in today’s “Penny-Arcade” Webcomic. It is in reference to a company called “OnLive” which promises to use virtualization and cloud computing to provide broadband, server-side gaming in the browser. The server renders the entire game, then sends back 60 images per second to a browser window. This means, theoretically, that older hardware can “play” complex games.

The big concern, of course, is latency - how many milliseconds does it take from the time you press the button until the button press is registered on the game server, and how many milliseconds does it take for the computer to display the results? In a video interview, OnLive’s CEO, Steve Perlman, handwaved those concerns away:

“If we had a significant amount of lag, it would be unusable. So we had to develop a new technology that would allow the game to run – a lot of the game to run – in the server center. And then really to send tiny pieces of the game down through your DSL and cable modem connection, very very rapidly, with no lag… your computer screen or your TV screen updates so fast that perceptually, it’s as if the game is running right there.”

The video game industry is particularly noteworthy for hype. For example, the Phantom console promised direct game downloads in 2004 – that turned out, of course, to be a scam. And let’s face it – there’s no such thing as “no lag” unless you’re planning to roll out TCP over Quantum Entanglement.

But there’s no doubt that the idea that OnLive proposes is feasible. That is, there’s very little difference between this technology and Cisco Telepresence, only, of course, instead of displaying an image of the opposite partner in the conversation, the image displayed is a computer-simulated hallucination that was pre-rendered and flattened from three dimensions to two. There really is no technical difference between the two.

Yes, it could be done.

But the question is, of course, will something that should work in theory work when it is field tested on the much less predictable conditions of the larger Internet (rather than on corporate LANs or WANs). Ultimately, OnLive relies entirely on network performance to remain feasible, at a time when many broadband providers are purposefully degrading Internet access in one way or another to prevent overcongestion.

The Cisco EnergyWise solution, which we wrote about earlier when it was unveiled at CiscoLive! in Barcelona, is starting to gather some attention.  For example, the Forrester Infrastructure Blog recently wrote about how EnergyWise can change basic assumptions within the ruling theory of IT.  (And yes, I wrote that all out so that I wouldn’t have to write a phrase as clichéd as “paradigm shift.”)

Still, when the shoe fits – the idea of “Green IT” has always been about reducing the amount of power consumption of IT itself; lowering utility costs.  Using IT to reduce the amount of power consumption for the company as a whole is an abrupt step change, advancing and augmenting the capabilities and responsibilities of IT in a revolutionary manner.  (Or… I guess you could call it a “quantum leap” if you had to…)

(Darn.  I’ve been working in this industry long enough that I’m beginning to think in buzzwords.  But I digress.)

Point is that we too often have been thinking of energy efficiency and network performance as – if not inversely proportional, than at least two goals that can interfere with each other – in that sometimes companies may choose to sacrifice one of them in order to improve the other, though most of the time, it’s mostly about finding a happy middle ground. 

But what’s happening with Cisco EnergyWise is that energy efficiency is now being shifted from a concern to a networked application.  The network can become the core of energy efficiency, rather than just a cause of energy consumption. 

This places IT teams in the precarious position of being responsible not just for the computers but for the facilities – heating, cooling, and lighting.  In other words, IT teams have just been given a hell of a lot of – and forgive the bad pun – power. 

Bad pun, yes, but resistance to IT taking over responsibilities for facilities will likely be strong because in an organization responsibility often is power.   When bad times come, and layoffs with them, companies will be more hesitant if someone is the only person in the company, “who knows how to turn on the darn lights…”

In an unrelated note, Network Instruments just released a new filter for use with GigaStor and Observer devices specifically designed to track the Conficker/Downadup worm.  More information on GigaStor can be found here. 

Steve Taylor and Larry Hettick at Network World recently talked to the CEO of VBrick systems, which sells video over IP appliances – and suggested that corporate networks should be engineered for video first and data second.  More importantly, he suggested that companies will start to engineer for video first and data second this year – 2009. 

“Last December (in 2008) I met with 30 CIOs and they were all planning for it to happen this year. CIOs view this as a low-cost way to increase productivity. Obviously there are travel cost savings, but increasingly [IPV] is being used for reassuring concerned employees [with improved communications].” Graziani also noted that the increased bandwidth capacity and improvements in video encoding / decoding technology have also contributed to making 2009 the year for a tipping point.

Don’t get me wrong, improvements in video encoding and decoding technology have made better quality video with smaller file sizes and requiring less bandwidth, and the importance of video in corporate communications is increasing.  Combine that with a desire to replace expensive travel with relatively inexpensive teleconferencing during an economic crisis that would make Cthulhu rising from the depths seem like a gleeful distraction, and you can see why the timing’s right. 

But considering that data applications for the network include such things as order processing, record retrieval, and report generation – I still have the feeling that companies will continue to see data as a priority for the foreseeable future.

Now, you could argue that when VoIP came around, it was more important that the CEO’s phone rings when he gets a call than processing X orders in Y milliseconds, when Y+100 milliseconds would do just fine.  But video over IP is, right now, a nice-to-have.

Now, I’m not saying that Video over IP isn’t important and effective at improving productivity and employee communications.  I’m just saying that there are other network aspects that are higher priority.  I mean, if you had to pull the plug on one of these services, which one would you choose?

A) VoIP
B) Data Applications
C) Video IP

I’d choose C, any day.  Granted, this is a false choice – a properly managed network should be able to handle all three.  Again, I’m not taking issue with the use of Video IP, simply the priority assigned to it in Graziani’s remarks.

The difference in priorities may be explained by the idea that there are some businesses out there that are so video-intensive that they might prioritize this differently.  However, we can’t imagine this being more than a small percentage. 

With that said, the leap from a Data/Voice network to a Data/Voice/Video network is not nearly as challenging as the leap from Data-only to Data/Voice.  For one, it’s simply the same problem – a new type of traffic on the network that’s latency sensitive.  The main difference from the networking perspective is that VideoIP is more like VoIP’s big brother, with larger bandwidth needs.  So I do agree with Graziani that we will see more VideoIP in 2009.  The timing’s right, the technology is here.   Now we just need to make sure that we have some way to monitor the new communications traffic. 

By Kevin Davis
Adapted from “Sources of Latency” Whitepaper

When network users call the Help Desk to report poor application performance, you don’t typically hear things like “The router’s CPU is too busy!,” “The network utilization is above 70%!,” or “The carrier path has failed-over to a sub-optimal path.” Instead, what you’re likely to hear is “The network is slow” or “The calls on my IP phone sound terrible.”

Complaints that end-users lodge are nearly always based their quality of experience using the application. And their quality of experience is almost always reliant on time.

Anytime a significant delay occurs in the delivery of network data, application performance suffers. Depending on the type of application and how it works, variances in network delay can have a severe impact on application performance thereby degrading end-user’s experiences.

Two important measurements of time intervals in network transmission systems are referred to as “latency” and “jitter”. Understanding latency and jitter sources and how their values vary in network architectures is critical to engineering application performance and optimizing information resources. For many regular readers, this will be old-hat, but we’ll go over it again.

Network latency is the amount of time it takes for a packet to be transmitted end-to-end across a network and is composed of five variables:

Network Latency = (Distance Delay) + (Serialization Delay) + (Queue Delay) + (Forwarding Delay) + (Protocol Delay)

Serialization Delay refers to the amount of time it takes for a network interface (such as a router’s interface or computer’s NIC) to perform bitwise transmission of a frame unto the outbound media, Forwarding Delay is the amount of time it takes a network device to process a frame/packet by performing a destination address lookup and forwarding the frame/packet to the outbound interface, and Protocol Delay is the amount of time that access or transmission algorithms may contribute to the delay of a network frame, and is typically introduced at the endpoints of the data transmission system.

Serialization delay, on a per-packet basis, becomes insignificant at data rates above 1.544 Mbits/s – or a T1. Forwarding delay is typically insignificant in modern routers and switches (when appropriately configured – significant delay can occur in misconfigured routers.) And Protocol delay typically occurs at the access layer or the end points. So the two major variables that have the most effect on network latency are Distance Delay and Queue Delay.

Distance Delay is simply the minimum amount of time that it takes the electrical signals that represent bits to travel down the physical wire. Optical cable sends bits at about ~5.5 µs/km, copper cable sends it at ~5.606 µs/km, and satellite sends bits at ~3.3 µs/km. (There are a few additional microseconds of delay from amplifying repeaters in optical cable, but compared to distance, the delay is negligible.)

Distance delay can have a significant impact on application performance for applications that require a large number of network round trips in order to complete a transaction – for example, custom transactional based applications, database queries, and VoIP, which begins do degrade when one-way end-to-end latency exceeds 200-220 milliseconds.

One of the biggest sources of end-user ire are database queries designed to run over a LAN ported to the WAN. For example if a user executes a SQL database query that requests 100 rows of a database table, one row at a time, over a link with a latency due to distance of 60 ms, it would take approximately 6 seconds (60 ms * 100 turns) to complete the transaction. The same query executed by a user on a LAN connected to the same database server would take less than 2-3 ms to be completed, as the latency due to distance across the LAN is insignificant.

Queue Delay is the amount of time a packet must spend in a network buffer waiting its turn to be transmitted. Network interfaces transmit one frame at a time, typically one bit at a time. As such, when two or more packets are forwarded to a network interface at the same time, or close to the same time – one packet is transmitted while the others are put in a queue on the interface buffer to await their turn at the interface. Packets that are put into the queue must wait until they can be transmitted, adding milliseconds of delay.

Increases in Queue Delay can be measured and detected by monitoring traffic along a given network path. Typically, most intermittent increases in latency above the baseline distance latency can be attributed to network congestion. (In order to reduce the possibility of excessive queue delay, application servers that are members of the same application architecture should be placed on the same Ethernet switch and on the same VLAN to ensure they do not have to compete for uplink bandwidth when problems like the one pictured above occur.)

Worse still, if the problem gets worse and packets wait in increasingly longer lines within the queue, the buffer may become full and the packets may be dropped. Packet drop, in turn, causes TCP connections to throttle back on the rate of transmission.

Those are some of the main causes of latency – but what about jitter?

Jitter is a term that refers to the variance in the arrival rate of packets from the same data flow, and abnormal jitter values can negatively impact real-time applications like VoIP and video. Jitter is typically created by three different mechanisms in a network: variance in Serialization Delays due to variance in packet sizes, variance in per-packet Queue Delay due to packet spacing from multiple sources at a common outbound interface, or packets taking different routes from source to destination – perhaps due to per-packet load sharing or routing issues.

The most effective way to deal with jitter is by using low-latency queuing for VoIP and video traffic on network interfaces with large serialization and/or queue delays. In addition, endpoints (such as IP phones) can use jitter buffers or playout delay buffers in order to deliver received packets at a constant rate to the end consumer. These buffers are typically 30-50 ms in depth, and thus they attempt to manage jitter values within these values on any single one-way path. While these buffers technically add 30-50ms in latency, they significantly reduce jitter. Since human beings don’t start to notice latency in VoIP or VideoIP applications till it hits about 200ms, if latency can be kept to under 150 milliseconds, then jitter can be significantly reduced using this method.

by Brian Boyko
Editor, Network Performance Daily

Time Warner Cable has rolled out its plan to cap the data of high-speed Internet subscribers in Beaumont, Texas, a town about 20 miles west of the Louisiana border.

The plans include $29.95/mo for 768kb/s downstream and a 5GB monthly cap - or $54.90/mo at 15Mb/s and 40GB monthly cap - with $1 additional charge for each GB above the cap. 

For comparison, the same service in Austin is $29.95/mo for 768kb/s downstream with no cap, or $59.95 (or $5.05 more) for 15Mb/s downstream with no cap. Here's Ars Technica quoting Kevin Leddy:   

Kevin Leddy, Time Warner Cable executive vice president of advanced technology, told the Associated Press that the variable billing model is being adopted to address the disparity in bandwidth consumption among Time Warner Cable users. Five percent of the subscribers are consuming half of the local line capacity, Leddy says.

Yes, the old "X% of users are using X% of bandwidth" argument, with an implication that those top 5% are hurting the ISP’s network performance.

I think by now we can shoot this out of the sky - not that there aren't bandwidth hogs, but mentioning that the top 5% of users are consuming 50% of the bandwidth is pretty much saying: "Apparently, Internet usage follows a power law curve." 

The Power Law Curve, or "Pareto Principle," or "80/20 rule," is part of the Internet.  Roughly 20% of people who participate on Forum X will leave 80% of the comments, roughly 20% of the gamers on World of Warcraft will log 80% of the game hours, and there's been an entire philosophy of thought called "The Long Tail" about how the power log curve affects many aspects of Internet business.  Business, by the way, has known about the 80/20 rule for a long time - which is why 20% of a supermarket's products will result in 80% of it's sales, or in a small business, why 20% of customers will provide 80% of the revenue.

So let's put this old argument to bed - 5% of the users will consume 50% of the capacity.  If you removed those 5% of the customers from the pool, chances are that the new top 5% - or what used to be the second 5% - will now consume roughly 50% of the capacity! 

Of course, this doesn’t stop Time Warner from offering 15Mb download speeds to customers who pay for the service. You’d think that if Time Warner was really concerned about network congestion, that they would scale down the bandwidth that they offer, rather than the data that people download.  Because data – data is an infinite resource.  There’s no limit to the number of bytes out there that you can download.  What is a limited resource is bandwidth – that is, the amount of data traveling along the same pipe at the same time.  And caps simply don’t help with that. 

Whatever Time Warner’s actual rationale, it has absolutely nothing to do with network performance.  (Which is why you’re finding this discussed on Network Performance Daily.  I know.  Irony can be so ironic sometimes.)

Instead, and this is a wild guess, I think that Time Warner wants bandwidth caps because it is working to preserve an old social order upon which the Time Warner enterprise is built, all while being pressured by the new social order that is emerging. 

The Boredom Killing Machine

Do cable companies with caps establish them with the express hope that those heavy-usage customers will move to other services, removing the need for them to spend money on upgrading their infrastructure?  Well, this is practically axiomatic: by removing those customers that are least profitable, they make more money.  What about the idea of anti-competitive behavior – as services like Hulu, NetFlix, AppleTV, and others use the Internet to deliver video-on-demand? I could understand a cable company being nervous about that. 

But there's more to it than that, and to dismiss this as merely the work of the “evil cable company” is to dismiss the bigger picture and ignore something more fundamental.

There is an entire paradigm shift that is occurring with the rise of broadband, and to understand it, we have to go back seventy years. 

In the 1938, the Fair Labor Standards Act was passed.  This act provided for the federal minimum wage, but what we want to look at is that it also established the standard of the 40 hour work week.

There were a number of changes that occurred because of that 40 hour work week.  Workers had something they never had before – an abundance of free time.  And for the most part, they didn’t know what to do with it. 

Similarly, trends towards suburbanization continued – helped by FHA loans and other programs, but also, a move to the suburbs required an increase in commuting time.  You could live near where you worked, but it was cheaper to live in the suburbs.  It meant you spent more time driving to and from work, but because of the 40 hour work week, people had more time than they had money. 

Additionally, Robert Putnam noticed in his book, “Bowling Alone,” that while his main thesis was that, past 1965, Americans were spending less time together engaged in group activities, from post WWII to around 1965, public participation in groups waxed.  But, starting in 1965, it declined sharply.

1965 was also the year when television reached 90% household penetration in America.

People spent more time in group activities before 1965 because – well, there was nothing else to do with the free time that they had been given.  (Yes, this is a simplification, but this is a relatively short article.)  And people spent less time in group activities after 1965 because they finally found a way to get rid of all that excess free time. 

The television. 

The television is not primarily a communication device – it only broadcasts one way.  I think an argument can be made that it is not an educational device. 

The television is a free-time killing machine.  It eliminates boredom.  It simply gives people the ability to shed themselves of the excess free time which was previously impossible. 

The Fertile Soil of the Suburban Mind

The 40-hour work week, suburbia and television are all related trends.

Now, the 40-hour work week, suburbia, and television, are all changing, for related reasons. 

According to the Los Angeles Times, 40% of America works 50 hours or more each week.  There’s also been a trend towards “re-urbanization,” due to desires to have shorter commute times, fewer gas bills, and a realization among young professionals that the social scene is better in the cities than out in the suburbs.  More free time and better ways to spend it.  But not all of us live in the cities - yet. 

The real threat to television that the Internet poses is not that you can watch the same shows on the computer that you could on the TV.  It is because the Internet was able to get people with varied and disparate interests to communicate and to organize.

When people who don’t participate in the Internet ask: "Where do people get the free time to create YouTube videos, or the free time to edit Wikipedia into a massive resource, or to spend hours on Slashdot replying to comments with jokes about Soviet Russia, or time to organize a protest complete with "V" masks?” - that free time is coming from the time once killed in front of the tube.

In other words, people are watching TV less - and when they do watch TV, it's usually a conscious choice to watch a particular TV show rather than a result of a lifetime habit of zoning out in front of the TV and watching "whatever's on."  Indeed, PVRs and TV-on-DVD are markets which evolved specifically to take advantage of this niche.  You can talk about the "convenience" of time-shifting, but the true effect that TiVo has had on American TV is that people don't have to watch crap unless they want to.

Clay Shirky, who gets credit for espousing these ideas in the first place, said:

"Desperate Housewives essentially functioned as a kind of cognitive heat sink, dissipating thinking that might otherwise have built up and caused society to overheat.

And it's only now, as we're waking up from that collective bender, that we're starting to see the cognitive surplus as an asset rather than as a crisis. We're seeing things being designed to take advantage of that surplus, to deploy it in ways more engaging than just having a TV in everybody's basement....

...So how big is that surplus? So if you take Wikipedia as a kind of unit, all of Wikipedia, the whole project--every page, every edit, every talk page, every line of code, in every language that Wikipedia exists in--that represents something like the cumulation of 100 million hours of human thought. I worked this out with Martin Wattenberg at IBM; it's a back-of-the-envelope calculation, but it's the right order of magnitude, about 100 million hours of thought.

And television watching? Two hundred billion hours, in the U.S. alone, every year. Put another way, now that we have a unit, that's 2,000 Wikipedia projects a year spent watching television. Or put still another way, in the U.S., we spend 100 million hours every weekend, just watching the ads. This is a pretty big surplus. People asking, "Where do they find the time?" when they're looking at things like Wikipedia don't understand how tiny that entire project is, as a carve-out of this asset that's finally being dragged into what Tim calls an architecture of participation."

So, what is the use of a bandwidth cap?  A bandwidth cap limits the amount of free time that can be spent on the Internet, leaving people once again, with a surplus of free time. 

Free Time, by Hook or by Crook.

I did some math here – remember, lowercase "b" represents "bits", uppercase "B" represents "bytes."

Here’s Time Warner’s “low-end” Beaumont plan.  
$29.95/mo at 768kb/s and 5GB monthly cap.

768kb/s = 96kB/s
96kB/s = 0.09375MB/s
0.09375MB/s = 0.000091552734375 GB/s
5GB / (0.000091552734375 GB/s)=
54613.33(repeating) seconds. =
910.22(repeating) minutes =
15.1703703(repeating) hours.

On the low-end plan, you have 15.1 hours of using your Internet connection to its fullest potential, before you hit the cap. (And you incur an additional dollar of bandwidth cost every additional 3 hours, 2 minutes.)

Now, here’s the “high-end” Beaumont plan.
$54.90/mo at 15Mb/s and 40GB monthly cap.

15Mb/s = 1.875MB/s
1.874MB/s = 0.0018310546875 GB/s
40GB / (0.0018310546875 GB/s) =
21845.33(repeating) seconds
364.088(repeating) minutes
6.06814814 (repeating) hours.

On the high-end plan, the "turbo" plan, you have 6.07 hours of using your Internet connection to its fullest potential before you hit the cap. (And you incur an additional dollar of bandwidth cost every additional 9 minutes, 6 seconds.)

By limiting the amount of data that can be downloaded, what caps really do is limit the amount of time, and therefore cognitive capital that one can spend on either the Internet or on "real world" activities enhanced or facilitated by the Internet. 

If you can't use the Internet, after all, because you've either gone over your bandwidth cap or you're afraid of going over your bandwidth cap, what are you going to do?  For most people, the answer will probably be "watch TV." 

And most cable providers are vertically integrated – Time Warner also owns numerous other interests, many of which are dependent on you sitting down in front of a television, including HBO and HBO Films, Adult Swim, Cartoon Network, Boomerang, truTV, TBS, CNN, TBS, TNT, a slew of production companies for TV, and a slew of production companies through Warner Bros. 

Television production companies are going to be faced with a dilemma as the boomers die and the millennials take charge: A population that won’t bother with watching the crap.  This will force production values up.  Additionally, with fewer and fewer viewers during fewer and fewer hours, broadcasters can demand less from advertisers.  Will this kill television?  No.  But the television industry is built almost entirely on the idea that people will watch because they have nothing better to do.

Now, people have something better to do.

After the Shift

Human beings are creatures of habit.  The baby boomers are much less likely to use the Internet to create, to organize, or to participate – they’re much more likely to simply use the Internet as a passive, one-way conduit of information, where you read, listen, or watch.  The younger generations are much more likely to use the Internet to converse – to engage, to participate, to do.

The “5%” that’s often touted out aren’t just the most savvy.  They are early adopters.  They are pioneers of a way of life that future generations will see as routine.  Eventually, that 5% will grow into 10%, then 20% - until we can’t think of a way of life without broadband.  (Some of us already can’t!)

Instituting bandwidth caps probably will not work in the long run – it is “stuffing the genie back into the bottle,” or “fighting the tide of history,” or whatever cliché you want to assign to it.  It is a desperation move. 

We’ve mentioned numerous times about broadband penetration and speed lagging behind countries more rural and less populated – in other words, countries the U.S. has no excuse lagging behind.

Ars Technica recently put out an article detailing what differences in national broadband policy exist that have enabled other nations to surpass the U.S.’s broadband capability. Japan and France have local loop unbundling – that allows for more competition among ISPs.  They also both deploy fiber instead of copper even if it doesn’t show an immediate profit, and competing ISPs are rolling out new fiber infrastructure instead of just leasing lines. 

Japan, France, Sweden, and Canada all treat broadband as a “core infrastructure element” – that is, it is treated as vital to the functioning of the national economy as good roads, bridges, tunnels, and electrical grids.

In all fairness, the U.S. can claim the same thing.  The U.S. may have no broadband policy, may be looking to traffic shaping to solve problems that would be better addressed by more fiber rollouts (oh, and by the way, there’s a new $800,000 deep packet inspection device on the market today to help service providers monitor and shape traffic), and may be relying on increasingly obsolete technologies – but at least we treat it the same as we do our roads, bridges, tunnels, and electrical grids. 

Which is to say, not very well at all.  The American Society of Civil Engineers gave the United States infrastructure a “D” in 2005, down from a score of “D+” in 2003 – and to fix those problems would require $1.6 trillion over five years.  Since then, not much has been done, according to this CBS video reposted on RawStory.com.

Instead, the government is considering plans to lease highways to private companies – using tolls to provide a “free market” solution to the infrastructure problem – but which will ultimately be a government sanctioned private monopoly over certain sections of blacktop. It is difficult to see how this would improve infrastructure, rather than simply allowing private companies to charge the maximum people will pay for the minimum infrastructure service people will put up with.

So, as far as treating broadband infrastructure like the rest of America’s infrastructure, it seems we already do that.  But what needs to be clear is that broadband infrastructure is infrastructure – that is, it is just as important for the rural area to get good broadband as it was for them to get good roads back during the Eisenhower administration. 

In a macabre way, this limited broadband is good for vendors; if broadband was plentiful there wouldn’t be so great a demand for WAN Optimization tools, for example.  Sure, WAN Optimization is a good idea anyway but it is the high cost of bandwidth that spurs demand forward.  It is becoming harder to maintain performance not just because of the various new demands on the network but also because the infrastructure across the country is simply inadequate – thus the demand for network performance monitoring tools.  Increasing bandwidth doesn’t always solve the network problem but insufficient bandwidth always creates one.