By Kevin Davis
Adapted from “Sources of Latency” Whitepaper

When network users call the Help Desk to report poor application performance, you don’t typically hear things like “The router’s CPU is too busy!,” “The network utilization is above 70%!,” or “The carrier path has failed-over to a sub-optimal path.” Instead, what you’re likely to hear is “The network is slow” or “The calls on my IP phone sound terrible.”

Complaints that end-users lodge are nearly always based their quality of experience using the application. And their quality of experience is almost always reliant on time.

Anytime a significant delay occurs in the delivery of network data, application performance suffers. Depending on the type of application and how it works, variances in network delay can have a severe impact on application performance thereby degrading end-user’s experiences.

Two important measurements of time intervals in network transmission systems are referred to as “latency” and “jitter”. Understanding latency and jitter sources and how their values vary in network architectures is critical to engineering application performance and optimizing information resources. For many regular readers, this will be old-hat, but we’ll go over it again.

Network latency is the amount of time it takes for a packet to be transmitted end-to-end across a network and is composed of five variables:

Network Latency = (Distance Delay) + (Serialization Delay) + (Queue Delay) + (Forwarding Delay) + (Protocol Delay)

Serialization Delay refers to the amount of time it takes for a network interface (such as a router’s interface or computer’s NIC) to perform bitwise transmission of a frame unto the outbound media, Forwarding Delay is the amount of time it takes a network device to process a frame/packet by performing a destination address lookup and forwarding the frame/packet to the outbound interface, and Protocol Delay is the amount of time that access or transmission algorithms may contribute to the delay of a network frame, and is typically introduced at the endpoints of the data transmission system.

Serialization delay, on a per-packet basis, becomes insignificant at data rates above 1.544 Mbits/s – or a T1. Forwarding delay is typically insignificant in modern routers and switches (when appropriately configured – significant delay can occur in misconfigured routers.) And Protocol delay typically occurs at the access layer or the end points. So the two major variables that have the most effect on network latency are Distance Delay and Queue Delay.

Distance Delay is simply the minimum amount of time that it takes the electrical signals that represent bits to travel down the physical wire. Optical cable sends bits at about ~5.5 µs/km, copper cable sends it at ~5.606 µs/km, and satellite sends bits at ~3.3 µs/km. (There are a few additional microseconds of delay from amplifying repeaters in optical cable, but compared to distance, the delay is negligible.)

Distance delay can have a significant impact on application performance for applications that require a large number of network round trips in order to complete a transaction – for example, custom transactional based applications, database queries, and VoIP, which begins do degrade when one-way end-to-end latency exceeds 200-220 milliseconds.

One of the biggest sources of end-user ire are database queries designed to run over a LAN ported to the WAN. For example if a user executes a SQL database query that requests 100 rows of a database table, one row at a time, over a link with a latency due to distance of 60 ms, it would take approximately 6 seconds (60 ms * 100 turns) to complete the transaction. The same query executed by a user on a LAN connected to the same database server would take less than 2-3 ms to be completed, as the latency due to distance across the LAN is insignificant.

Queue Delay is the amount of time a packet must spend in a network buffer waiting its turn to be transmitted. Network interfaces transmit one frame at a time, typically one bit at a time. As such, when two or more packets are forwarded to a network interface at the same time, or close to the same time – one packet is transmitted while the others are put in a queue on the interface buffer to await their turn at the interface. Packets that are put into the queue must wait until they can be transmitted, adding milliseconds of delay.

Increases in Queue Delay can be measured and detected by monitoring traffic along a given network path. Typically, most intermittent increases in latency above the baseline distance latency can be attributed to network congestion. (In order to reduce the possibility of excessive queue delay, application servers that are members of the same application architecture should be placed on the same Ethernet switch and on the same VLAN to ensure they do not have to compete for uplink bandwidth when problems like the one pictured above occur.)

Worse still, if the problem gets worse and packets wait in increasingly longer lines within the queue, the buffer may become full and the packets may be dropped. Packet drop, in turn, causes TCP connections to throttle back on the rate of transmission.

Those are some of the main causes of latency – but what about jitter?

Jitter is a term that refers to the variance in the arrival rate of packets from the same data flow, and abnormal jitter values can negatively impact real-time applications like VoIP and video. Jitter is typically created by three different mechanisms in a network: variance in Serialization Delays due to variance in packet sizes, variance in per-packet Queue Delay due to packet spacing from multiple sources at a common outbound interface, or packets taking different routes from source to destination – perhaps due to per-packet load sharing or routing issues.

The most effective way to deal with jitter is by using low-latency queuing for VoIP and video traffic on network interfaces with large serialization and/or queue delays. In addition, endpoints (such as IP phones) can use jitter buffers or playout delay buffers in order to deliver received packets at a constant rate to the end consumer. These buffers are typically 30-50 ms in depth, and thus they attempt to manage jitter values within these values on any single one-way path. While these buffers technically add 30-50ms in latency, they significantly reduce jitter. Since human beings don’t start to notice latency in VoIP or VideoIP applications till it hits about 200ms, if latency can be kept to under 150 milliseconds, then jitter can be significantly reduced using this method.

As you might notice, we’ve got a new “Twitter Live Feed” on the left sidebar of Network Performance Daily – this is being maintained by the thoroughly awesome Chandra Hosek, who gets to add the moniker “Twitter Wrangler” to her already impressive resume.  You can also follow our Twitter at http://twitter.com/NetQoSLive using whatever twitter-following app you prefer. 

You’ll notice that we’ve put this way down on the sidebar, so you might have to look for it.  This is for a very important reason: Twitter’s servers have this annoying habit of occasionally seizing up and drooling on themselves at random intervals. 

See, we would have liked to put the Twitter feed prominently on the page, but because many browsers, including Firefox 3, load the page line by line, any content below after the slow loading Twitter feed resolves gets delayed.  Placing Twitter at the top of the page would mean the entire site would take seconds to load. 

This is actually more than just a specific concern for us – one of the advents of the 21^st century is the idea that we get added value from our Web applications by combining with other Web applications – mashups. 

Many Web 2.0 services, Google maps, data from Craigslist, etc, provide APIs so that third parties can use the applications to develop their own applications or add functionality to their Web apps and Web sites – creating mashups of multiple applications.

So now, instead of having all your apps running in the same place, the apps are running from multiple different places which network engineers have little control over.  And in many cases this can result in a delay to the end-user – because your performance is based on the performance of other computers, other networks, other hardware. 

Some of the apps that contribute to these mashups may be in turn based on multiple services themselves.  The problem can compound itself. 

So this is a concern that should be on any network engineer’s mind when talking to the application developers. 

It’s Download Day.  At 10:00 a.m. PDT, or noon, for us in Austin, Firefox 3.0 was released to the public in what the Mozilla foundation has dubbed “download day.” In fact, they’re attempting to set a Guinness World Record for “most downloads in a 24 hour period.” 

So, it was a bit of a concern to us because with all those people downloading Web browsers, there would be sure to be traffic spikes on our network. But the “Download Day” promotion is such a huge success that Mozilla is having trouble keeping their own server up. 

At 10:16 a.m. PDT, I can see a “The server at www.spreadfirefox.com is taking too long to respond” error.  Mozilla.org is also unable to resolve. 

At 10:30 a.m. PDT, it’s still not connecting, and I decide to stop hitting refresh and go and eat lunch. Mmm.  Roast Beef. 

At 11:30 a.m. PDT, Spreadfirefox.com is still not resolving, but Mozilla.org does.  That doesn’t last, however, as I go to download Firefox, I get a “Http/1.1 Service Unavailable” error.   I bring up a copy of “Waiting for Godot” in another browser window.

It is 12:00 noon on the Pacific.  Spreadfirefox.com is still not resolving. 

12:30 p.m. PDT.  Still not working.  I clean off my work desk, something I’ve been putting off for a wh—ew, is that mayonnaise?  (I hope that’s mayonnaise.)

1:00 p.m. PDT. No Firefox, but My desk is now clean.  (My closet is now dangerous.)  Time to catch up on my RSS feeds to find out if there are any interesting leads that I can investigate. Hmm.  Wine 1.0 is out, but that really doesn’t have a lot to do with network performance.  Reddit seems have problems with Firefox too.  But somebody has to be getting the browser – there’s over 8000 downloads a minute according to the counting tracker.  Wait.  Some users report the counts running backward… what, are people uploading it back?

1:45 p.m. PDT. Aha!  Finally.  The page resolves and I begin my download… and it redirects me to Firefox 2.0.0.14.  Great.

1:55 p.m. PDT. I download Opera 9.5.

2:00 p.m. PDT. Mozilla’s page finally shows a link to Firefox 3.0 – but still shows the logo for Firefox 2.  The 7.1 MB download starts at around 50kBytes/s – which is pretty lame for the usual 700kBytes/s I can get when I download from work. 

2:15 p.m. I install Firefox 3.0 and launch it.  It’s nice.  It’s certainly more responsive and uses less memory.  However, my Tab Mix Plus extension isn’t compatible, and furthermore, there’s no option to undo closed tabs.  All in all, a disappointment – if it were a restaurant, it would be infamous for slow service and bad food.

Leaving aside the whole “Undo Closed Tabs” issue, you would think that an organization actively trying to beat the world record for the most downloads in a 24 hour period might, you know, be prepared enough to make sure the servers don’t go down?

Additionally; Mozilla has been promoting “Download Day” for some time now, so it makes sense for IT departments to be prepared for the onslaught of downloads coming into the network from users upgrading their PCs to the latest version of the browser – and keep track of the impact that traffic has on the user experience for more mission-critical apps.

Apple’s new 3G iPhone will soon be issued to most of you.  Ownership of the Apple 3G iPhone is mandatory.  This message is brought to you by the Ministry of Cellphones, MiniCel. 

APPLE IS PEACE.
FREEDOM IS NO THIRD PARTY APPS.
NOT INCLUDING AN ACCESSIBLE BATTERY COMPARTMENT IS STRENGTH.

The one concern is that AT&T’s offerings limit the iPhone’s data transfer rate to 1.4Mbits/s downstream.  That’s better than EDGE, but isn’t 3G supposed to be able to produce 3.6Mbits/s, with claims that 3G would hit 7Mbits/s soon? 

From Gizmodo:

Turns out, according to AT&T people we talked to, 1.4Mbps is the capped bandwidth for all mobile smartphones on the network for a few reasons….

A major one is battery life—the faster you burn, the faster your battery dies, so going full steam at 3.6Mbps would cut you well short of that nice round five hours. A second one is cell site congestion and backhaul (carrier-speak for size of the wired dataline that connects cell sites to the actual telecom infrastructure). While everyone at AT&T, from the top down, is adamant that AT&T is "comfortable" with their ability to meet the huge data draw once 3G iPhones hit the streets, it's not like the pipe is unlimited.

Ah – this is understandable.  A glut of new iPhone users will create strain on the network, and so, in order to prevent the network from being overwhelmed, it is limiting the bandwidth – that is, the throughput of the connection – in order to ensure that every customer has a decent service rate. 

This seems to be an effective plan to dealing with bandwidth shortage on overcrowded networks. 

But… hmm, who else has been complaining about overcrowded, oversubscribed networks lately?  Who was it that was talking about ‘bandwidth hogs’ and tried a different solution?  I just can’t seem to put my finger on it…

Data capping is seems to be ISPs stock solution for dealing with what they claim to be oversubscribed networks.  Of course, we’ve showed that time and time again, data limits don’t solve bandwidth problems.  QoS policies and per-user bandwidth limits – like the one AT&T is rolling out for the iPhone 3G – do. 

Which leads me to suspect that the big “bandwidth shortage” for broadband providers is something that’s almost entirely overblown – perhaps even fictional - to derive a new revenue stream from consumers while hindering the competition to services that compete with other subsidiaries of ISPs parent companies.  Because if there was actually a bandwidth shortage, wouldn’t ISPs be doing what AT&T is doing with the 3G network?  Wouldn’t they choose the more effective solution to the problem? 

Something’s not quite right here.

The article by Nick Carr that we referenced on Monday has come out.

The article, “Is Google Making Us Stupid,” written by Nick Carr is not only in the Atlantic but is also the cover story, written as “Is Google Making Us Stoopid.”

We understand that Carr is a good writer, that the article is solidly researched, and there’s no doubt that with changes in the technology of communication and mediums of communication, there will be changes both in writing and in readership.

However, “Is Google Making Us Stupid?” is a provocative headline that really has little to do with the original concept of the article. Even if Google and Internet communication has reduced our attention spans (to a degree greater than would have occurred naturally had there not been public acceptance of the Internet) having a reduced attention span does not necessarily make us less intelligent and less able. It just makes us more distracted.

What makes us stupid, however, is oversimplification.

You could argue that the Internet changes the way we think, but to characterize this in a headline as “Google making us stupid” does a disservice to Carr’s article, a disservice to Google, and a disservice to people. If the Internet really does change the way we think then it’s more important that we examine the ways in which it does and understand what we can do to use our brains most effectively in this new way – by reducing it to a yes/no (rhetorical) question, it leads to a false dichotomy that the Internet is either good (if the answer is no) or bad (if the answer is yes.)

At any rate, giving this story the cover of the Atlantic during a time when there are so many much more important stories not covered is just plain… well, not smart.

by Patrick Ancipink

Things used to be easy.

No, wait.  Things never used to be easy.  In fact, they were horribly complex and frustrating to the point where engineers pull their hair out.  But now we usually expect around 99.99umpteen% uptime from our network equipment. 

So frustration today often stems from the new tasks that enterprise IT engineers are expected to handle beyond the routers and switches.  Application delivery controllers, WAN Optimization controllers, and more latency sensitive applications such as VoIP and Teleconferencing simply mean that the IT teams are being tasked with problems that require them to think in new ways about what it means to be in IT.

If you’ve been to any networking convention or conference, you’ve probably heard “in IT you either develop applications or deliver applications” more times than you’ve seen the Brady Bunch episode in which Marcia gets hit in the face with a football.  That’d doesn’t make it any less true. 

Ann Bednarz, writing for Network World, suggests that companies take research firm Gartner’s advice and look to hire “application delivery architects and engineers.” The idea is that there should be at least one person in the IT department whose full time job is worrying about application delivery and tuning on a WAN – someone who can converse with application developers and security teams and end users. 

At NetQoS, we’re trying to help companies get the information they need to either designate and train an existing member of the IT staff for these new responsibilities, or at least know what to look for when hiring for an Application Delivery Engineer position.

For example, some things we’re doing right now include our NetAnalyst training based on real-world examples on resolving complex network application issues, and integrating our multiple products together in the NetQoS Performance Center. 

But there are some more subtle ways in which we’re hoping to get this point across.  We argue that the most important metric for network performance management is application response time.  And while there’s many things that can affect application response time, the most basic is that your best possible application response time is limited by the latency of the connection (especially in financial applications,) multiplied by the number of connections that the application has to make.  Network engineers often focus on only one aspect of that formula, latency – while application developers only focus on the other aspect – the connections.  (That’s if they bother to think about the impact of the app on the network at all. And if they do, their test environment sorely lacks any similarity to the real world WAN.)  

So the value of developing the role of the Application Delivery Engineer, someone who can coordinate the two halves of that Application Response Time equation, becomes clear. 

Usenet may be going the way of Gopher, as Time Warner, Sprint, and Verizon, rather than taking measures to selectively target the 88 newsgroups listed by the office of New York Attorney General Andrew Cuomo that contained child pornography, they’ve decided that USENET as a whole is pretty much not worth the trouble.  Time Warner will remove it’s free-to-subscribers USENET servers, Sprint is cutting off the alt.* hierarchy, and Verizon has made no definite plans but it’s likely they they would cut off the alt.* hierarchy as well.

This means, in order to get rid of “alt.illegalstuff.ick.ick.ick,” they will also be shutting down groups like alt.certification.cisco.

Keep in mind that none of these companies are blocking access, as has been erroneously reported on social news services like Reddit.  There’s nothing stopping the NNTP protocol – it’s just that these cable companies will no longer be maintaining their own newsgroup servers.  That means you’ll still be able to get to alt.certification.cisco from Google Groups for free – but Google Groups does not contain all of the alt.* hierarchy, nor does it handle binary attachments.  For that, customers will need to sign up an account with a third-party USENET provider for a fee.

Even so, removing the entirety of USENET seemed like overkill if the issue was a handful of groups, and there are certainly free speech implications.  So we talked to Jeff Simmermon, Time Warner Cable’s director of digital communication.  Simmermon said that the decision to eliminate newsgroups from their service offerings came down to three factors: “a. Very few customers use them; b. the technology is outdated; c. the risk is not worth the reward.”

There’s no doubt that A is true; USENET is certainly a “power user” tool.  Outdated technology? Possibly – although people have been decrying the end of USENET for years, ever since the invention of PHP forum boards.  The third: Risk is not worth the reward… this is actually quite interesting.

USENET is one of the older services – old enough that USENET has actually been tested in the court as a common carrier – that is, so long as an ISP does not edit the content of USENET, they cannot be held responsible for the content of USENET. So I would imagine that the risk would be very, very low – which gives you some idea of what Time Warner thinks of the possible rewards. 

Mr. Simmermon said that customers would not be credited for the removal of what had previously been a free part of their Internet service, “to the best of [his] knowledge.”

Time Warner subscribers will be notified of the change, in a final bit of irony, through postings at the help.rr.com newsgroup.

You know, one of the difficulties in being a tech blogger/columnist/writer is that when something comes along that’s big news – like yesterday’s announcement of an actually-affordable iPhone with a 3G network and enterprise application support (specifically, Microsoft Exchange and Cisco IPsec VPN,) everyone else has already covered it to death. 

Seriously, Network World.  Did we really need three front-page stories on the new iPhone?  Three?  Four if you count that duplicated story?

There’s no doubt that iPhone’s new enterprise integration is big news – in fact, we covered it when it was first announced in March.  So we could go over the implications of consumer devices on the enterprise network, but we’ve already done so.  Repeatedly. 

Anyway, with stories about the iPhone already coming up as #1 on the google search for “Jesus Phone,” I don’t think I have anything insightful to add about its second coming.  So I won’t. 

Okay, maybe this one troubling tidbit from Matt Buchanan at Gizmodo:

Supposedly the network will be ready, even if the supply won't be. I asked him four different ways if it was ready for the onslaught of millions of 3G data phones and he said "absolutely" each time, and that they've planned for it. What's unclear is how many units they've planned for the first day. He said they expected "high" demand but nobody knows what the "full demand" will be, in response to my question about meeting demand.

Discuss?

[Ed. Note: The article referenced in this post has since been published online on the Atlantic Monthly Web site.]

Nick Carr, author of “The Big Switch” and “Is IT Obsolete?” has written “Is Google Making Us Stupid?,” and it has been published in the July issue of the Atlantic Monthly. 

Sadly, I called up my local bookstore and they only currently carry the June issue of the Atlantic Monthly, so I can’t give you a well informed critique of Carr’s thoughts.  However, the article is quoted – minimally – by Matt Asay of C|Net.

Of course, there’s a certain amount of irony that Asay seems to use very limited excerpts from Carr’s article to decry the “soundbite culture.” 

Then again, I’m about to give you my thoughts on an article that Asay has read and I haven’t, so if Asay’s article is ironic, this one is downright hypocritical.  So be it.  This seems to be the most important direct quote from Carr’s original article:

The Internet promises to have particularly far-reaching effects on cognition....The Internet, an immeasurably powerful computing system, is subsuming most of our other intellectual technologies. It's becoming our map and our clock, our printing press and our typewriter, our calculator and our telephone, and our radio and TV.

When the Net absorbs a medium, that medium is recreated in the Net's image. It injects the medium's content with hyperlinks, blinking ads, and other digital gewgaws, and it surrounds the content with the content of other media it has absorbed. A new e-mail message, for instance, may announce its arrival as we're glancing over the latest headlines at a newspaper's site. The result is to scatter our attention and diffuse our concentration.

Of course you could make the opposite point.  With RSS feeds, news aggregators, and “long tail” blogs, there is also a point to be made that instead of distracting us and diffusing our concentration, we end up hyper-focused on one or two topics to the complete exclusion of everything else.  (The “scattering” effect actually came up quite a bit in my grad-level journalism classes as a defense of the dying newspaper – that you get to see articles you may not have been interested out of the corner of your eye while you read articles that you are interested in.)

But as I mentioned, I haven’t read the entire article; so instead of taking apart Carr’s argument – let’s put forward a new one. 

The limits on network performance then in turn limit the ability to communicate complex thought. 

Let’s start with Twitter.  A twitter post is to information what bumper stickers are to philosophy, at 140 characters, there’s not much that can be done.  But Twitter already suffers from network performance problems and outages presumably related to scale.  If Twitter allowed longer posts, that increases the amount of data traversing across the network. 

You may be asking: So what?  Twitter, at its core, isn’t very different than the “Friends” feature of LiveJournal – and you can post long posts on Livejournal.  This is mostly true, but there is a major difference between Twitter’s model and LiveJournal.  LiveJournal’s “Friends” posts are pulled out of the database at various different times by various readers who actively “pull” the information to their Web browsers.  Twitter, on the other hand, “pushes” the information like an IM client – and does so simultaneously to multiple users.  Twitter’s big selling point is immediacy - latency needs to be low.  Since a single twitter user can have hundreds or even thousands of subscribers… well, you can see the implications.  Twitter’s performance problems may seem incongruous for such a “simple little app” but are actually quite complex.

So, for right now, 140 characters is all that Twitter can handle.  You can blog, you can email, you can IM to express more complex ideas, but because Twitter requires additional demands on the network, the medium’s ability to express complex thought is limited by the performance of the network.

To take a further point, let’s look at YouTube.  There’s another arbitrary limit – 10 minutes or 100MB of data.  Here, the relationship between performance and the limit are a bit more direct; though other video services allow for longer/bigger videos, none of them have the demand that YouTube has. 

But the relationship between complex thought and YouTube is a bit less direct – certainly a complex thought can be expressed in 10 minutes.  Perhaps not completely examined like a book – but certainly expressed.  And comparatively, the 10 minute YouTube video delivers subtlety and nuance to the point where it is replacing the 10 second sound-bite usually found on television.  In this case, the medium’s ability to express complex thought is limited by the performance of the network but is still more informative than the alternative. 

Then again, it’s all about how we use the information; if we used every bit of information in a Cisco Telepresence rig to send text, there would be no human that would be able to parse that much text, that quickly.  The 100MB used for those 10 minutes of YouTube video could also hold the entire text of War and Peace 32 times over. 

To talk about whether the Internet makes you stupid (as SomethingAwful.com has been decrying for years) is to oversimplify a complex idea.  If, during the course of one’s Internet browsing, one is easily distracted when looking for information; this distraction will interfere with your ability to think about things in depth.  On the other hand, if one thinks about things in depth and does not allow for some distraction, one can end up with a deep, but not particularly broad amount of information.  Neither one really decreases your actual intelligence; it’s just the way that one looks at different subjects. 

Eventually I’ll manage to read Mr. Carr’s article and address these points in more depth.  Right now, however, I’m forced to conclude that Google is not making us stupid.

4Chan is making us stupid.   

by Brian Boyko
Editor, Network Performance Daily

Time Warner Cable has rolled out its plan to cap the data of high-speed Internet subscribers in Beaumont, Texas, a town about 20 miles west of the Louisiana border.

The plans include $29.95/mo for 768kb/s downstream and a 5GB monthly cap - or $54.90/mo at 15Mb/s and 40GB monthly cap - with $1 additional charge for each GB above the cap. 

For comparison, the same service in Austin is $29.95/mo for 768kb/s downstream with no cap, or $59.95 (or $5.05 more) for 15Mb/s downstream with no cap. Here's Ars Technica quoting Kevin Leddy:   

Kevin Leddy, Time Warner Cable executive vice president of advanced technology, told the Associated Press that the variable billing model is being adopted to address the disparity in bandwidth consumption among Time Warner Cable users. Five percent of the subscribers are consuming half of the local line capacity, Leddy says.

Yes, the old "X% of users are using X% of bandwidth" argument, with an implication that those top 5% are hurting the ISP’s network performance.

I think by now we can shoot this out of the sky - not that there aren't bandwidth hogs, but mentioning that the top 5% of users are consuming 50% of the bandwidth is pretty much saying: "Apparently, Internet usage follows a power law curve." 

The Power Law Curve, or "Pareto Principle," or "80/20 rule," is part of the Internet.  Roughly 20% of people who participate on Forum X will leave 80% of the comments, roughly 20% of the gamers on World of Warcraft will log 80% of the game hours, and there's been an entire philosophy of thought called "The Long Tail" about how the power log curve affects many aspects of Internet business.  Business, by the way, has known about the 80/20 rule for a long time - which is why 20% of a supermarket's products will result in 80% of it's sales, or in a small business, why 20% of customers will provide 80% of the revenue.

So let's put this old argument to bed - 5% of the users will consume 50% of the capacity.  If you removed those 5% of the customers from the pool, chances are that the new top 5% - or what used to be the second 5% - will now consume roughly 50% of the capacity! 

Of course, this doesn’t stop Time Warner from offering 15Mb download speeds to customers who pay for the service. You’d think that if Time Warner was really concerned about network congestion, that they would scale down the bandwidth that they offer, rather than the data that people download.  Because data – data is an infinite resource.  There’s no limit to the number of bytes out there that you can download.  What is a limited resource is bandwidth – that is, the amount of data traveling along the same pipe at the same time.  And caps simply don’t help with that. 

Whatever Time Warner’s actual rationale, it has absolutely nothing to do with network performance.  (Which is why you’re finding this discussed on Network Performance Daily.  I know.  Irony can be so ironic sometimes.)

Instead, and this is a wild guess, I think that Time Warner wants bandwidth caps because it is working to preserve an old social order upon which the Time Warner enterprise is built, all while being pressured by the new social order that is emerging. 

The Boredom Killing Machine

Do cable companies with caps establish them with the express hope that those heavy-usage customers will move to other services, removing the need for them to spend money on upgrading their infrastructure?  Well, this is practically axiomatic: by removing those customers that are least profitable, they make more money.  What about the idea of anti-competitive behavior – as services like Hulu, NetFlix, AppleTV, and others use the Internet to deliver video-on-demand? I could understand a cable company being nervous about that. 

But there's more to it than that, and to dismiss this as merely the work of the “evil cable company” is to dismiss the bigger picture and ignore something more fundamental.

There is an entire paradigm shift that is occurring with the rise of broadband, and to understand it, we have to go back seventy years. 

In the 1938, the Fair Labor Standards Act was passed.  This act provided for the federal minimum wage, but what we want to look at is that it also established the standard of the 40 hour work week.

There were a number of changes that occurred because of that 40 hour work week.  Workers had something they never had before – an abundance of free time.  And for the most part, they didn’t know what to do with it. 

Similarly, trends towards suburbanization continued – helped by FHA loans and other programs, but also, a move to the suburbs required an increase in commuting time.  You could live near where you worked, but it was cheaper to live in the suburbs.  It meant you spent more time driving to and from work, but because of the 40 hour work week, people had more time than they had money. 

Additionally, Robert Putnam noticed in his book, “Bowling Alone,” that while his main thesis was that, past 1965, Americans were spending less time together engaged in group activities, from post WWII to around 1965, public participation in groups waxed.  But, starting in 1965, it declined sharply.

1965 was also the year when television reached 90% household penetration in America.

People spent more time in group activities before 1965 because – well, there was nothing else to do with the free time that they had been given.  (Yes, this is a simplification, but this is a relatively short article.)  And people spent less time in group activities after 1965 because they finally found a way to get rid of all that excess free time. 

The television. 

The television is not primarily a communication device – it only broadcasts one way.  I think an argument can be made that it is not an educational device. 

The television is a free-time killing machine.  It eliminates boredom.  It simply gives people the ability to shed themselves of the excess free time which was previously impossible. 

The Fertile Soil of the Suburban Mind

The 40-hour work week, suburbia and television are all related trends.

Now, the 40-hour work week, suburbia, and television, are all changing, for related reasons. 

According to the Los Angeles Times, 40% of America works 50 hours or more each week.  There’s also been a trend towards “re-urbanization,” due to desires to have shorter commute times, fewer gas bills, and a realization among young professionals that the social scene is better in the cities than out in the suburbs.  More free time and better ways to spend it.  But not all of us live in the cities - yet. 

The real threat to television that the Internet poses is not that you can watch the same shows on the computer that you could on the TV.  It is because the Internet was able to get people with varied and disparate interests to communicate and to organize.

When people who don’t participate in the Internet ask: "Where do people get the free time to create YouTube videos, or the free time to edit Wikipedia into a massive resource, or to spend hours on Slashdot replying to comments with jokes about Soviet Russia, or time to organize a protest complete with "V" masks?” - that free time is coming from the time once killed in front of the tube.

In other words, people are watching TV less - and when they do watch TV, it's usually a conscious choice to watch a particular TV show rather than a result of a lifetime habit of zoning out in front of the TV and watching "whatever's on."  Indeed, PVRs and TV-on-DVD are markets which evolved specifically to take advantage of this niche.  You can talk about the "convenience" of time-shifting, but the true effect that TiVo has had on American TV is that people don't have to watch crap unless they want to.

Clay Shirky, who gets credit for espousing these ideas in the first place, said:

"Desperate Housewives essentially functioned as a kind of cognitive heat sink, dissipating thinking that might otherwise have built up and caused society to overheat.

And it's only now, as we're waking up from that collective bender, that we're starting to see the cognitive surplus as an asset rather than as a crisis. We're seeing things being designed to take advantage of that surplus, to deploy it in ways more engaging than just having a TV in everybody's basement....

...So how big is that surplus? So if you take Wikipedia as a kind of unit, all of Wikipedia, the whole project--every page, every edit, every talk page, every line of code, in every language that Wikipedia exists in--that represents something like the cumulation of 100 million hours of human thought. I worked this out with Martin Wattenberg at IBM; it's a back-of-the-envelope calculation, but it's the right order of magnitude, about 100 million hours of thought.

And television watching? Two hundred billion hours, in the U.S. alone, every year. Put another way, now that we have a unit, that's 2,000 Wikipedia projects a year spent watching television. Or put still another way, in the U.S., we spend 100 million hours every weekend, just watching the ads. This is a pretty big surplus. People asking, "Where do they find the time?" when they're looking at things like Wikipedia don't understand how tiny that entire project is, as a carve-out of this asset that's finally being dragged into what Tim calls an architecture of participation."

So, what is the use of a bandwidth cap?  A bandwidth cap limits the amount of free time that can be spent on the Internet, leaving people once again, with a surplus of free time. 

Free Time, by Hook or by Crook.

I did some math here – remember, lowercase "b" represents "bits", uppercase "B" represents "bytes."

Here’s Time Warner’s “low-end” Beaumont plan.  
$29.95/mo at 768kb/s and 5GB monthly cap.

768kb/s = 96kB/s
96kB/s = 0.09375MB/s
0.09375MB/s = 0.000091552734375 GB/s
5GB / (0.000091552734375 GB/s)=
54613.33(repeating) seconds. =
910.22(repeating) minutes =
15.1703703(repeating) hours.

On the low-end plan, you have 15.1 hours of using your Internet connection to its fullest potential, before you hit the cap. (And you incur an additional dollar of bandwidth cost every additional 3 hours, 2 minutes.)

Now, here’s the “high-end” Beaumont plan.
$54.90/mo at 15Mb/s and 40GB monthly cap.

15Mb/s = 1.875MB/s
1.874MB/s = 0.0018310546875 GB/s
40GB / (0.0018310546875 GB/s) =
21845.33(repeating) seconds
364.088(repeating) minutes
6.06814814 (repeating) hours.

On the high-end plan, the "turbo" plan, you have 6.07 hours of using your Internet connection to its fullest potential before you hit the cap. (And you incur an additional dollar of bandwidth cost every additional 9 minutes, 6 seconds.)

By limiting the amount of data that can be downloaded, what caps really do is limit the amount of time, and therefore cognitive capital that one can spend on either the Internet or on "real world" activities enhanced or facilitated by the Internet. 

If you can't use the Internet, after all, because you've either gone over your bandwidth cap or you're afraid of going over your bandwidth cap, what are you going to do?  For most people, the answer will probably be "watch TV." 

And most cable providers are vertically integrated – Time Warner also owns numerous other interests, many of which are dependent on you sitting down in front of a television, including HBO and HBO Films, Adult Swim, Cartoon Network, Boomerang, truTV, TBS, CNN, TBS, TNT, a slew of production companies for TV, and a slew of production companies through Warner Bros. 

Television production companies are going to be faced with a dilemma as the boomers die and the millennials take charge: A population that won’t bother with watching the crap.  This will force production values up.  Additionally, with fewer and fewer viewers during fewer and fewer hours, broadcasters can demand less from advertisers.  Will this kill television?  No.  But the television industry is built almost entirely on the idea that people will watch because they have nothing better to do.

Now, people have something better to do.

After the Shift

Human beings are creatures of habit.  The baby boomers are much less likely to use the Internet to create, to organize, or to participate – they’re much more likely to simply use the Internet as a passive, one-way conduit of information, where you read, listen, or watch.  The younger generations are much more likely to use the Internet to converse – to engage, to participate, to do.

The “5%” that’s often touted out aren’t just the most savvy.  They are early adopters.  They are pioneers of a way of life that future generations will see as routine.  Eventually, that 5% will grow into 10%, then 20% - until we can’t think of a way of life without broadband.  (Some of us already can’t!)

Instituting bandwidth caps probably will not work in the long run – it is “stuffing the genie back into the bottle,” or “fighting the tide of history,” or whatever cliché you want to assign to it.  It is a desperation move.