By Kevin Davis
Adapted from “Sources of Latency” Whitepaper

When network users call the Help Desk to report poor application performance, you don’t typically hear things like “The router’s CPU is too busy!,” “The network utilization is above 70%!,” or “The carrier path has failed-over to a sub-optimal path.” Instead, what you’re likely to hear is “The network is slow” or “The calls on my IP phone sound terrible.”

Complaints that end-users lodge are nearly always based their quality of experience using the application. And their quality of experience is almost always reliant on time.

Anytime a significant delay occurs in the delivery of network data, application performance suffers. Depending on the type of application and how it works, variances in network delay can have a severe impact on application performance thereby degrading end-user’s experiences.

Two important measurements of time intervals in network transmission systems are referred to as “latency” and “jitter”. Understanding latency and jitter sources and how their values vary in network architectures is critical to engineering application performance and optimizing information resources. For many regular readers, this will be old-hat, but we’ll go over it again.

Network latency is the amount of time it takes for a packet to be transmitted end-to-end across a network and is composed of five variables:

Network Latency = (Distance Delay) + (Serialization Delay) + (Queue Delay) + (Forwarding Delay) + (Protocol Delay)

Serialization Delay refers to the amount of time it takes for a network interface (such as a router’s interface or computer’s NIC) to perform bitwise transmission of a frame unto the outbound media, Forwarding Delay is the amount of time it takes a network device to process a frame/packet by performing a destination address lookup and forwarding the frame/packet to the outbound interface, and Protocol Delay is the amount of time that access or transmission algorithms may contribute to the delay of a network frame, and is typically introduced at the endpoints of the data transmission system.

Serialization delay, on a per-packet basis, becomes insignificant at data rates above 1.544 Mbits/s – or a T1. Forwarding delay is typically insignificant in modern routers and switches (when appropriately configured – significant delay can occur in misconfigured routers.) And Protocol delay typically occurs at the access layer or the end points. So the two major variables that have the most effect on network latency are Distance Delay and Queue Delay.

Distance Delay is simply the minimum amount of time that it takes the electrical signals that represent bits to travel down the physical wire. Optical cable sends bits at about ~5.5 µs/km, copper cable sends it at ~5.606 µs/km, and satellite sends bits at ~3.3 µs/km. (There are a few additional microseconds of delay from amplifying repeaters in optical cable, but compared to distance, the delay is negligible.)

Distance delay can have a significant impact on application performance for applications that require a large number of network round trips in order to complete a transaction – for example, custom transactional based applications, database queries, and VoIP, which begins do degrade when one-way end-to-end latency exceeds 200-220 milliseconds.

One of the biggest sources of end-user ire are database queries designed to run over a LAN ported to the WAN. For example if a user executes a SQL database query that requests 100 rows of a database table, one row at a time, over a link with a latency due to distance of 60 ms, it would take approximately 6 seconds (60 ms * 100 turns) to complete the transaction. The same query executed by a user on a LAN connected to the same database server would take less than 2-3 ms to be completed, as the latency due to distance across the LAN is insignificant.

Queue Delay is the amount of time a packet must spend in a network buffer waiting its turn to be transmitted. Network interfaces transmit one frame at a time, typically one bit at a time. As such, when two or more packets are forwarded to a network interface at the same time, or close to the same time – one packet is transmitted while the others are put in a queue on the interface buffer to await their turn at the interface. Packets that are put into the queue must wait until they can be transmitted, adding milliseconds of delay.

Increases in Queue Delay can be measured and detected by monitoring traffic along a given network path. Typically, most intermittent increases in latency above the baseline distance latency can be attributed to network congestion. (In order to reduce the possibility of excessive queue delay, application servers that are members of the same application architecture should be placed on the same Ethernet switch and on the same VLAN to ensure they do not have to compete for uplink bandwidth when problems like the one pictured above occur.)

Worse still, if the problem gets worse and packets wait in increasingly longer lines within the queue, the buffer may become full and the packets may be dropped. Packet drop, in turn, causes TCP connections to throttle back on the rate of transmission.

Those are some of the main causes of latency – but what about jitter?

Jitter is a term that refers to the variance in the arrival rate of packets from the same data flow, and abnormal jitter values can negatively impact real-time applications like VoIP and video. Jitter is typically created by three different mechanisms in a network: variance in Serialization Delays due to variance in packet sizes, variance in per-packet Queue Delay due to packet spacing from multiple sources at a common outbound interface, or packets taking different routes from source to destination – perhaps due to per-packet load sharing or routing issues.

The most effective way to deal with jitter is by using low-latency queuing for VoIP and video traffic on network interfaces with large serialization and/or queue delays. In addition, endpoints (such as IP phones) can use jitter buffers or playout delay buffers in order to deliver received packets at a constant rate to the end consumer. These buffers are typically 30-50 ms in depth, and thus they attempt to manage jitter values within these values on any single one-way path. While these buffers technically add 30-50ms in latency, they significantly reduce jitter. Since human beings don’t start to notice latency in VoIP or VideoIP applications till it hits about 200ms, if latency can be kept to under 150 milliseconds, then jitter can be significantly reduced using this method.

by Brian Boyko
Editor, Network Performance Daily

Time Warner Cable has rolled out its plan to cap the data of high-speed Internet subscribers in Beaumont, Texas, a town about 20 miles west of the Louisiana border.

The plans include $29.95/mo for 768kb/s downstream and a 5GB monthly cap - or $54.90/mo at 15Mb/s and 40GB monthly cap - with $1 additional charge for each GB above the cap. 

For comparison, the same service in Austin is $29.95/mo for 768kb/s downstream with no cap, or $59.95 (or $5.05 more) for 15Mb/s downstream with no cap. Here's Ars Technica quoting Kevin Leddy:   

Kevin Leddy, Time Warner Cable executive vice president of advanced technology, told the Associated Press that the variable billing model is being adopted to address the disparity in bandwidth consumption among Time Warner Cable users. Five percent of the subscribers are consuming half of the local line capacity, Leddy says.

Yes, the old "X% of users are using X% of bandwidth" argument, with an implication that those top 5% are hurting the ISP’s network performance.

I think by now we can shoot this out of the sky - not that there aren't bandwidth hogs, but mentioning that the top 5% of users are consuming 50% of the bandwidth is pretty much saying: "Apparently, Internet usage follows a power law curve." 

The Power Law Curve, or "Pareto Principle," or "80/20 rule," is part of the Internet.  Roughly 20% of people who participate on Forum X will leave 80% of the comments, roughly 20% of the gamers on World of Warcraft will log 80% of the game hours, and there's been an entire philosophy of thought called "The Long Tail" about how the power log curve affects many aspects of Internet business.  Business, by the way, has known about the 80/20 rule for a long time - which is why 20% of a supermarket's products will result in 80% of it's sales, or in a small business, why 20% of customers will provide 80% of the revenue.

So let's put this old argument to bed - 5% of the users will consume 50% of the capacity.  If you removed those 5% of the customers from the pool, chances are that the new top 5% - or what used to be the second 5% - will now consume roughly 50% of the capacity! 

Of course, this doesn’t stop Time Warner from offering 15Mb download speeds to customers who pay for the service. You’d think that if Time Warner was really concerned about network congestion, that they would scale down the bandwidth that they offer, rather than the data that people download.  Because data – data is an infinite resource.  There’s no limit to the number of bytes out there that you can download.  What is a limited resource is bandwidth – that is, the amount of data traveling along the same pipe at the same time.  And caps simply don’t help with that. 

Whatever Time Warner’s actual rationale, it has absolutely nothing to do with network performance.  (Which is why you’re finding this discussed on Network Performance Daily.  I know.  Irony can be so ironic sometimes.)

Instead, and this is a wild guess, I think that Time Warner wants bandwidth caps because it is working to preserve an old social order upon which the Time Warner enterprise is built, all while being pressured by the new social order that is emerging. 

The Boredom Killing Machine

Do cable companies with caps establish them with the express hope that those heavy-usage customers will move to other services, removing the need for them to spend money on upgrading their infrastructure?  Well, this is practically axiomatic: by removing those customers that are least profitable, they make more money.  What about the idea of anti-competitive behavior – as services like Hulu, NetFlix, AppleTV, and others use the Internet to deliver video-on-demand? I could understand a cable company being nervous about that. 

But there's more to it than that, and to dismiss this as merely the work of the “evil cable company” is to dismiss the bigger picture and ignore something more fundamental.

There is an entire paradigm shift that is occurring with the rise of broadband, and to understand it, we have to go back seventy years. 

In the 1938, the Fair Labor Standards Act was passed.  This act provided for the federal minimum wage, but what we want to look at is that it also established the standard of the 40 hour work week.

There were a number of changes that occurred because of that 40 hour work week.  Workers had something they never had before – an abundance of free time.  And for the most part, they didn’t know what to do with it. 

Similarly, trends towards suburbanization continued – helped by FHA loans and other programs, but also, a move to the suburbs required an increase in commuting time.  You could live near where you worked, but it was cheaper to live in the suburbs.  It meant you spent more time driving to and from work, but because of the 40 hour work week, people had more time than they had money. 

Additionally, Robert Putnam noticed in his book, “Bowling Alone,” that while his main thesis was that, past 1965, Americans were spending less time together engaged in group activities, from post WWII to around 1965, public participation in groups waxed.  But, starting in 1965, it declined sharply.

1965 was also the year when television reached 90% household penetration in America.

People spent more time in group activities before 1965 because – well, there was nothing else to do with the free time that they had been given.  (Yes, this is a simplification, but this is a relatively short article.)  And people spent less time in group activities after 1965 because they finally found a way to get rid of all that excess free time. 

The television. 

The television is not primarily a communication device – it only broadcasts one way.  I think an argument can be made that it is not an educational device. 

The television is a free-time killing machine.  It eliminates boredom.  It simply gives people the ability to shed themselves of the excess free time which was previously impossible. 

The Fertile Soil of the Suburban Mind

The 40-hour work week, suburbia and television are all related trends.

Now, the 40-hour work week, suburbia, and television, are all changing, for related reasons. 

According to the Los Angeles Times, 40% of America works 50 hours or more each week.  There’s also been a trend towards “re-urbanization,” due to desires to have shorter commute times, fewer gas bills, and a realization among young professionals that the social scene is better in the cities than out in the suburbs.  More free time and better ways to spend it.  But not all of us live in the cities - yet. 

The real threat to television that the Internet poses is not that you can watch the same shows on the computer that you could on the TV.  It is because the Internet was able to get people with varied and disparate interests to communicate and to organize.

When people who don’t participate in the Internet ask: "Where do people get the free time to create YouTube videos, or the free time to edit Wikipedia into a massive resource, or to spend hours on Slashdot replying to comments with jokes about Soviet Russia, or time to organize a protest complete with "V" masks?” - that free time is coming from the time once killed in front of the tube.

In other words, people are watching TV less - and when they do watch TV, it's usually a conscious choice to watch a particular TV show rather than a result of a lifetime habit of zoning out in front of the TV and watching "whatever's on."  Indeed, PVRs and TV-on-DVD are markets which evolved specifically to take advantage of this niche.  You can talk about the "convenience" of time-shifting, but the true effect that TiVo has had on American TV is that people don't have to watch crap unless they want to.

Clay Shirky, who gets credit for espousing these ideas in the first place, said:

"Desperate Housewives essentially functioned as a kind of cognitive heat sink, dissipating thinking that might otherwise have built up and caused society to overheat.

And it's only now, as we're waking up from that collective bender, that we're starting to see the cognitive surplus as an asset rather than as a crisis. We're seeing things being designed to take advantage of that surplus, to deploy it in ways more engaging than just having a TV in everybody's basement....

...So how big is that surplus? So if you take Wikipedia as a kind of unit, all of Wikipedia, the whole project--every page, every edit, every talk page, every line of code, in every language that Wikipedia exists in--that represents something like the cumulation of 100 million hours of human thought. I worked this out with Martin Wattenberg at IBM; it's a back-of-the-envelope calculation, but it's the right order of magnitude, about 100 million hours of thought.

And television watching? Two hundred billion hours, in the U.S. alone, every year. Put another way, now that we have a unit, that's 2,000 Wikipedia projects a year spent watching television. Or put still another way, in the U.S., we spend 100 million hours every weekend, just watching the ads. This is a pretty big surplus. People asking, "Where do they find the time?" when they're looking at things like Wikipedia don't understand how tiny that entire project is, as a carve-out of this asset that's finally being dragged into what Tim calls an architecture of participation."

So, what is the use of a bandwidth cap?  A bandwidth cap limits the amount of free time that can be spent on the Internet, leaving people once again, with a surplus of free time. 

Free Time, by Hook or by Crook.

I did some math here – remember, lowercase "b" represents "bits", uppercase "B" represents "bytes."

Here’s Time Warner’s “low-end” Beaumont plan.  
$29.95/mo at 768kb/s and 5GB monthly cap.

768kb/s = 96kB/s
96kB/s = 0.09375MB/s
0.09375MB/s = 0.000091552734375 GB/s
5GB / (0.000091552734375 GB/s)=
54613.33(repeating) seconds. =
910.22(repeating) minutes =
15.1703703(repeating) hours.

On the low-end plan, you have 15.1 hours of using your Internet connection to its fullest potential, before you hit the cap. (And you incur an additional dollar of bandwidth cost every additional 3 hours, 2 minutes.)

Now, here’s the “high-end” Beaumont plan.
$54.90/mo at 15Mb/s and 40GB monthly cap.

15Mb/s = 1.875MB/s
1.874MB/s = 0.0018310546875 GB/s
40GB / (0.0018310546875 GB/s) =
21845.33(repeating) seconds
364.088(repeating) minutes
6.06814814 (repeating) hours.

On the high-end plan, the "turbo" plan, you have 6.07 hours of using your Internet connection to its fullest potential before you hit the cap. (And you incur an additional dollar of bandwidth cost every additional 9 minutes, 6 seconds.)

By limiting the amount of data that can be downloaded, what caps really do is limit the amount of time, and therefore cognitive capital that one can spend on either the Internet or on "real world" activities enhanced or facilitated by the Internet. 

If you can't use the Internet, after all, because you've either gone over your bandwidth cap or you're afraid of going over your bandwidth cap, what are you going to do?  For most people, the answer will probably be "watch TV." 

And most cable providers are vertically integrated – Time Warner also owns numerous other interests, many of which are dependent on you sitting down in front of a television, including HBO and HBO Films, Adult Swim, Cartoon Network, Boomerang, truTV, TBS, CNN, TBS, TNT, a slew of production companies for TV, and a slew of production companies through Warner Bros. 

Television production companies are going to be faced with a dilemma as the boomers die and the millennials take charge: A population that won’t bother with watching the crap.  This will force production values up.  Additionally, with fewer and fewer viewers during fewer and fewer hours, broadcasters can demand less from advertisers.  Will this kill television?  No.  But the television industry is built almost entirely on the idea that people will watch because they have nothing better to do.

Now, people have something better to do.

After the Shift

Human beings are creatures of habit.  The baby boomers are much less likely to use the Internet to create, to organize, or to participate – they’re much more likely to simply use the Internet as a passive, one-way conduit of information, where you read, listen, or watch.  The younger generations are much more likely to use the Internet to converse – to engage, to participate, to do.

The “5%” that’s often touted out aren’t just the most savvy.  They are early adopters.  They are pioneers of a way of life that future generations will see as routine.  Eventually, that 5% will grow into 10%, then 20% - until we can’t think of a way of life without broadband.  (Some of us already can’t!)

Instituting bandwidth caps probably will not work in the long run – it is “stuffing the genie back into the bottle,” or “fighting the tide of history,” or whatever cliché you want to assign to it.  It is a desperation move. 

We’ve mentioned numerous times about broadband penetration and speed lagging behind countries more rural and less populated – in other words, countries the U.S. has no excuse lagging behind.

Ars Technica recently put out an article detailing what differences in national broadband policy exist that have enabled other nations to surpass the U.S.’s broadband capability. Japan and France have local loop unbundling – that allows for more competition among ISPs.  They also both deploy fiber instead of copper even if it doesn’t show an immediate profit, and competing ISPs are rolling out new fiber infrastructure instead of just leasing lines. 

Japan, France, Sweden, and Canada all treat broadband as a “core infrastructure element” – that is, it is treated as vital to the functioning of the national economy as good roads, bridges, tunnels, and electrical grids.

In all fairness, the U.S. can claim the same thing.  The U.S. may have no broadband policy, may be looking to traffic shaping to solve problems that would be better addressed by more fiber rollouts (oh, and by the way, there’s a new $800,000 deep packet inspection device on the market today to help service providers monitor and shape traffic), and may be relying on increasingly obsolete technologies – but at least we treat it the same as we do our roads, bridges, tunnels, and electrical grids. 

Which is to say, not very well at all.  The American Society of Civil Engineers gave the United States infrastructure a “D” in 2005, down from a score of “D+” in 2003 – and to fix those problems would require $1.6 trillion over five years.  Since then, not much has been done, according to this CBS video reposted on RawStory.com.

Instead, the government is considering plans to lease highways to private companies – using tolls to provide a “free market” solution to the infrastructure problem – but which will ultimately be a government sanctioned private monopoly over certain sections of blacktop. It is difficult to see how this would improve infrastructure, rather than simply allowing private companies to charge the maximum people will pay for the minimum infrastructure service people will put up with.

So, as far as treating broadband infrastructure like the rest of America’s infrastructure, it seems we already do that.  But what needs to be clear is that broadband infrastructure is infrastructure – that is, it is just as important for the rural area to get good broadband as it was for them to get good roads back during the Eisenhower administration. 

In a macabre way, this limited broadband is good for vendors; if broadband was plentiful there wouldn’t be so great a demand for WAN Optimization tools, for example.  Sure, WAN Optimization is a good idea anyway but it is the high cost of bandwidth that spurs demand forward.  It is becoming harder to maintain performance not just because of the various new demands on the network but also because the infrastructure across the country is simply inadequate – thus the demand for network performance monitoring tools.  Increasing bandwidth doesn’t always solve the network problem but insufficient bandwidth always creates one.

Kevin Davis, a senior consultant at NetQoS, will be presenting a few training sessions at Symposium about SuperAgent, the end-to-end response time module of the NetQoS Performance Center. This will include a training session about how to use time-based network metrics in troubleshooting.  He talks about his upcoming training session below.

In the session, I’m going to be covering the importance of using a time-based metric in troubleshooting, because end-users complain foremost about time.  For example, they’ll say “the application is running slow,” or they believe “the network is slow.”  To users, everything is based on time, that’s what they’re complaining about.  And they’re correct.

It’s very new to many people to think of performance in “time” although that may seem counterintuitive - because most people are used to reading utilization graphs.  With utilization graphs, however, we don’t know if 70 or 80 or 90 percent utilization is necessarily impacting the user experience.  I mean, we buy networking equipment, routers, switches, firewalls, servers, and we want them to be highly – or efficiently - utilized.  Seeing high utilization could indicate a problem – or it could just indicate that you haven’t over-purchased.  So you can have a link at 90% utilization or a router at ninety percent CPU utilization but you won’t know if that’s impacting the end-user without a time based metric.

It’s time-based data that tells you how the users are being impacted.  Sure, the utilization data – the interface utilization, memory utilization, I/O utilization, can often tell what is doing the impact.  But the time base shows you the degree of the impact – the real-world effect on end-users.  With a time-based instrument, such as NetQoS SuperAgent, you can find out where the delay increase is occurring, and whether it’s based in the network, server, or application. 

In fact, you can take a look at time-based data and make a determination very quickly as to which entity is creating the performance issue – the beautiful thing about SuperAgent, in particular, is that it trends by time 24/7, so not only can you determine how your important business applications are being impacted today, but you can go back and look at recurring patterns in performance issues.  You can see if today is worse than yesterday or last week or last month.

In the session, I’ll also be going over how to architect the data center for performance.  Placement of servers that participate in inter-architectures is critical for the health and performance of the application and indeed the data center.  We also talk about how different protocols, for example, Microsoft’s TCP/IP stack, can impact application performance by enhancing or degrading it. 

It’s important for servers that are serving the same application.  For example, a front-end Web server and a back-end Oracle database really should be on the same switch on the same VLAN.  That way they receive optimum service from the network.  If they do leave the switch, they’ll have to contend with bandwidth going up and down the switch links, and they’ll be switched and routed multiple times. 

Based on measurements from customer environments and from our own laboratories, when two servers are on different switches they can have up to 18 milliseconds delay between them.  If we think of that in the terms of network engineers of one millisecond per 100 miles, what in effect we’re doing when we put two different servers on different switches, or two different VLANs on the same switch, we’re making it look like those servers are 1800 miles apart – like one server is in Los Angeles and the other is in Memphis. 

by Patrick Ancipink
Director of Product Marketing, NetQoS

There’s been a lot of growth (and attendant hype) in technology areas like WAN optimization and application acceleration over the past few years, and for good reason. Anything that helps companies speed up and reduce the risk of strategic IT initiatives like consolidating data centers, turning up new branches or serving an increasingly mobile and scattered user community will be popular.

To help with cope with the increasing reliance on the WAN and keep latency in check, there are a dizzying array of vendors and products out there – but if you’re trying to determine precisely which techniques and technologies to implement for your specific needs, the array of vendors quickly goes from “dizzying” to “disorienting” and finally “nauseating.” 

Cisco’s been in this Tilt-a-Whirl™ of a market for a while (and NetQoS has been right there with them) and they’ve taken some big steps recently to provide a more holistic approach that centers on building an “application aware” network, rather than trying to highlight one type of implementation against another for a narrow set of capabilities.

NetQoS started working exclusively with Cisco closely to help customers evaluate, measure, and prove the effectiveness of WAN optimization and application acceleration deployments. As customers are moving from pilot phases into full production, the before/after measurements and comprehensive monitoring are critical to ensure customers are getting the benefits they intended and doing what they need to deliver application performance. 

To help get the word out, Cisco just launched a new section of their web site today that contains a wealth of information about, as they call it, “WAN and Application Optimization.” The downloadable presentation, Cisco WAN and Application Optimization Technical Overview Presentation, puts Cisco technologies (and complimentary ones, NetQoS included) into a useful context with a methodical approach and framework built around four steps: Profile and Baseline, Optimize, Evolve, and Operate. A whole Campbell’s Factory of Cisco alphabet soup technologies are included—WAAS, ACE, NBAR, Netflow, CBQoS, IP SLA, PfR—to show how they work in concert and what role they play in the bigger picture.

There’s also the Cisco WAN and Application Optimization Solution Guide , a very in-depth publication—like 227 pages deep—that is targeted for “technical personnel involved in the specification, design, and implementation of specific WAN and application optimization solutions.” We, here at NetQoS, are proud to have contributed several sections to book regarding the methodology and implementation of network performance monitoring for WAN optimization and application acceleration. 

(If you are looking for some lighter fare, the video on the site tells a nice story in about 6 minutes including an airshow, snowmobiles, windsurfers, and skydiving—interesting choices for demonstrating the criticality of serving video over the WAN.  Then again, some company somewhere has to make the recreational products, I suppose.)

Traffic shaping is not a tool of the devil, nor do we believe the solution to bandwidth problems is simply to provision more dark fiber and build more underground fiber optic lines. But as time has gone on, the issues around network neutrality have become more pronounced.

For example, Bell Canada has been throttling P2P service, much like Comcast in the United States. However, what makes this different is that Bell Canada is in a position much like AT&T – in that throttling the network on the backbone affects all the people – including people who are not Bell’s customers – along the line.

Worse still, Bell has been reselling the capacity to provide ADSL service to smaller ISPs without letting the services know that the bandwidth is throttled for certain applications. One of those smaller ISPs, Teksavvy, said: “We are not throttling anything and as far as I am aware will never throttle anyone. We don't believe in it.”– so the idea that Bell will leave them with no choice in the matter is a little worrisome. There isn’t much choice in the matter – the only other big broadband provider in Canada is Rogers Cable, which also throttles traffic.

There are arguments that “net neutrality” will be solved by the forces of the free market – that is, if one ISP throttles, they can go to their competitors. The problem is that, in this case, this is exactly what savvy customers were doing by moving from larger companies, like Bell and Rogers, to smaller companies like Teksavvy. From the consumer, it’s reducing their choice. For the small ISP, it’s could be considered downright anticompetitive, and the Canadian Association of Internet Providers applied for relief before the Canadian Radio-television and Telecommunications Commission that would require Bell Canada to cease and desist.

We contacted Rocky Gaudrault, CEO of Teksavvy Solutions, but because this was now a legal matter, he explained that he was unable to comment. It was clear that he is passionate about the issue, but Teksavvy’s staff keeps him from speaking out by supplying him with timbits and beer to keep his mouth and hands busy.

Particularly interesting is this comment by a Slashdotter – both Bell and Teksavvy charge on a “tiered and metered” basis – which pretty much cuts through the false choice between deep packet inspection and metered bandwidth; Bell has both. (One profanity-laden post implied that the only reason that Bell Canada did this was to coldly eliminate the most compelling competitive advantage that smaller ISPs had – Bell had throttled traffic, small ISPs didn’t.)

The upshot is that network neutrality concerns have been brought to Canada’s Parliament during Question Time. (link via Prof. Michael Geist at the University of Ottowa, who we hope to have an interview with on Monday.) It’s unsurprising because these matters do not just affect consumers but large enterprises as well - an unannounced and sudden change in the QoS policies of the backbone provider is exactly the type of thing that can foul up capacity planning, VoIP switchover, teleconferencing, etc. Especially worrisome are those technology companies who rely on some form or another of P2P traffic to help cut their bandwidth costs.

Deep packet inspection is a powerful tool, and used in the right hands, in the right way, it can help make QoS planning easier, can help streamline business critical applications, can provide overall better end-user response times, and may indeed be a great technological boon.

But we can’t see any benefit in this case for throttling the traffic of resold bandwidth, and for not disclosing the changes in advance. If businesses that control backbone traffic want to avoid governmental regulation, they need to show that they can be responsible with the power they have and use it in a manner which is neither anti-competitive nor deceptive to wholesale resellers and end-user customers.

Editorial by Brian Boyko
Editor, Network Performance Daily

Ars Technica reported that a memo claiming that Time Warner was going to roll out a "pay-as-you-go" metered scheme for Internet access, rather than today's subscription-based unlimited bandwidth access plans was leaked to BroadbandReports.com. That memo, which since has been removed, claimed that Time Warner was going to try metered/limited access on a trial basis in Beaumont, Texas, and Time Warner representatives have confirmed this plan with Reuters.

As Ars Technica pointed out, Comcast has tried using bandwidth caps and traffic shaping to curb Internet usage among the customers that pay Comcast for Internet access. Comcast, however, has run into trouble because it has not revealed those policies to Comcast's customers. Time Warner will supposedly give customers online tools to monitor bandwidth usage.

Of course, it would be the best solution to increase the capability of the network - ISPs have to play by different rules than corporate networks as they are common carriers. But we don't know whether it is economically feasible for Time Warner's cable division to remain profitable while increasing the bandwidth, and if an unlimited-access plan is not feasible, a pay-as-you-go plan seems at first to be the fairest of the alternatives.

That said, there's something a little, well, strange about this, because the Internet is not a big truck that you just dump something on. It's a series of tubes.

Solving The Wrong Problem

That is, all Internet connections are merely the transfer of little positively and negatively charged electrical bits which stream down the wire. The limitations are not in the availability of the resource but in the capacity of distribution. We are not, in other words, "running out of bandwidth" like we run out of oil, run out of water, or run out of diapers.

What is limited is the capacity of the "pipe." To strain a metaphor, you could push Lake Michigan through a coffee stirring straw, but it would take a very, very long time.

Any pay-as-you-go plan has a fatal flaw - it doesn't make a whole lot of sense to bill people for the data they are downloading because data is not the limited resource!

What is limited is the capacity of the ISP's infrastructure at any particular moment in time, so it would be saner to limit the usage of the pipeline at a particular time. Perhaps to even out the usage of bandwidth, the ISP could provide different speeds for peak and off-peak usage times. Those unhappy with the slow speeds at peak times could pay a premium for a greater share of the pipe.

But wait a minute! ISPs already do this - I know that my Internet connection at home is capped at a certain speed. In fact I could get a faster speed simply by asking for it and paying a premium - no delay nor needed infrastructure upgrades. Just cash.

So the move to a pay-as-you-go plan seems, to be at best a case of solving the wrong problem, and at worst a case of "double dipping" by making people pay for data and bandwidth. (If there are network slowdowns, charging people per-gigabyte won't help much if people are still downloading that gigabyte at the same time of the day, after all.)

Your Experiences May Differ

Unfortunately, I've been on the receiving ends of one of these plans. Recently I was in New Zealand filming a movie about electoral reform. Bland stuff. While I was there, I was planning to upload film to the Internet - sort of a production blog. But I found that I couldn't - the ISP there, New Zealand Telecom, had placed my flatmates and myself on a pay-as-you-go program with a cap of only one gigabyte, and they would not increase the cap until the next billing period, which would have been after I left the country.

One gigabyte. Anything over that amount was downloaded at speeds that I hadn't seen since I bought my last 56.6k modem. That meant that even doing things like normal Web browsing was a particularly hard chore. Uploading film to YouTube was right out. I was even hoping to get some extra work done for Network Performance Daily during that time but found that I simply did not have the ability to do so. I was, in a word, ticked off and frustrated. It certainly made it quite a bit harder for me to use the network - I ended up getting a lot of iced mochas at the local Internet café, as patronage was a prerequisite for Internet service.

Now, I have no idea if Time Warner plans anything like New Zealand Telecom, and Time Warner has more competition - even in Beaumont, TX - than New Zealand Telecom did in Wellington. That may force them to abandon this plan if they find customers cancelling accounts and leaving for competitors.

It is rather important to notice that the last mainstream successful service that charged you based on how much you used it was 1996's AOL.

I've never been to Japan, France, or Korea but I'm told that all of these countries have broadband available at much greater speed, without having to worry about pay-as-you-go plans. So the question is not whether unlimited broadband is technically feasible as more people use broadband, the question is whether companies are willing to make the infrastructure investments necessary. And considering that there will be more competition, not less, as new technologies (like FIOS and WiMax) become available, investing in infrastructure rather than limiting customers seems to be the smarter move in the long term.

But let's say that this plan is a success in Beaumont, and catches on. What's the upshot for enterprise networking?

You Think You Have A Recreational Network Use Problem Now…

If people come to expect that every piece of data that goes through their network is going to cost them extra money, that may mean that all the large data that they were once downloading at home now ends up getting downloaded to the corporate network and taken home via flash drives. In addition to the spike in traffic use, there are also issues with copyright infringement liability, computer security (with flash drives from home possibly containing malware - not to mention that people will probably swap flash drives within the company, spreading infections,) and people looking for large files to download before they go home instead of doing work.

Now, in many ways, the problem with limited bandwidth availability from an ISP may seem similar to limited bandwidth availability on a corporate WAN. But a business has many more options for dealing with slow networks than an ISP does. Businesses can check their application performance and if necessary recode them (many legacy apps designed for a LAN are too "chatty" for the WAN.) They can set QoS policies to make sure certain types of traffic from certain types of applications get priority. Traffic can be rescheduled so that it goes through the system during off-peak times.

Businesses have all these options - including limiting the end-users in a number of different ways - because in a business, the network is there to serve the business. But in an ISP, the network is there to serve the subscribers by providing a common-carrier communications service.

As such, the subscribers of an ISP can and should determine what traffic should be on the network, when, where, and how much. Any methods to alter, curb, slow, or block traffic from the network should be disclosed to the end-user at the very least and should be avoided unless there are no other alternatives - to do otherwise is to create a value judgment on certain types of traffic and to endorse certain types of speech over others.

(Perhaps I'm wrong on this, but…) To my knowledge, no company uses a method similar to "pay-as-you-go" to curb recreational traffic on their networks. They may limit speeds to certain applications, they may block sites, but I don't believe that any company institutes a bandwidth cap on its own employees.

That to me suggests that this plan doesn't have much merit as a solution to ISP oversubscription.

What do you think about Time Warner's plan? Disagree with the author? Feel free to make your opinions heard in our comments section.

By Brian Boyko

If you spend some time poring through RFC documents (something I don’t recommend for the 99% of the population that is still sane) you’ll find tons of improvements, modifications, case-specific optimizations and alternatives to TCP, the workhorse of networking transport protocols since the 1970s.

Seth Noble, President and Founder of Data Expedition, Inc., believes that he can do one better. His company claims that their proprietary transport protocol, MTP, for “Multipurpose Transaction Protocol,” provides a scalable alternative to TCP that uses bandwidth more efficiently. According to Mr. Noble:

“TCP's 1970's data model makes dealing with this problem more difficult than it needs to be. TCP was created with the assumption that packet loss would "rarely" occur, and so it is rather fragile in our modern, congested networks. A lot of very smart people have tried for many years to patch TCP and help it cope, but it still carries its 30 year old legacy with it.”

“MTP/IP was designed from scratch to operate in congested environments where packet loss and other network problems are common. As a result, MTP/IP does an exceptionally good job of quickly AND correctly identifying whether or not data has really been lost and then recovering that data with little or no disruption.”

(Continued...)

Continue reading "Proprietary MTP: an alternative to TCP?" »

For more information on this topic, you can download our Tech Brief on Cisco WAAS, available here

by Steve Fulton

Users expect a ubiquitous and instantaneous network, as well as consistent application performance. This, combined with a proliferation of business critical, Web 2.0, (and recreational) applications that consume precious WAN bandwidth, forces IT to get very creative in squeezing more performance out of existing infrastructure.

Hence the red-hot market for application acceleration and WAN optimization products that address WAN performance problems caused by latency, congestion, and applications (such as WAFS and CIFS) that were designed for the LAN and now have to traverse the WAN due to data center consolidation.

Cisco shook things up in late 2006 with the introduction of WAAS-short for Wide Area Application Services-technology that is transparent to the underlying network infrastructure. According to Cisco, WAAS combines WAN optimization, acceleration of TCP-based applications, and Cisco's Wide Area File Services (WAFS) in a single appliance or blade.

WAAS addresses problems related to traffic congestion that need some sort of optimization done at the branch. It complements Cisco's Application Control Engine (ACE), which is a data center optimization product that integrates server load balancing, application security, and unique virtual partitioning capabilities.

(Continued...)

Continue reading "WAAS Up with Cisco's WAN Optimization Initiative?" »

by Jeff Hicks

There's a number of reasons why a company would move to VoIP. Generally there's been some component of cost-savings - it may be in regular long distance savings, it may be in hardware cost savings (versus a PBX system), it may be that you only have one network infrastructure to deploy and manage.

But it's interesting how in the past couple of years, costs have become less of a factor in the decision process. Long-distance rates have dropped, so the cost factor is not quite as pronounced as it used to be, especially considering short-term rollout costs.

(Continued...)

Continue reading "VoIP Traffic Isn't Just Normal Traffic" »