According to Network World, T-Mobile recently announced “T-Mobile @Home” which allows you to buy VoIP-to-the-home service from T-Mobile for $10/mo – assuming you don’t mind the $50 router and the 2 year lock-in contract. 

The interesting part of the article was that T-Mobile released a study that 97% of customers who had a landline service then got a VoIP service - dropped the landline service completely.  The implications are staggering – traffic once reserved for the land lines will now be thrown in with the rest of the Internet mix.  I’m not quite worried about capacity but rather about latency – will latency on the network remain low enough for a VoIP system to remain superior to the traditional phone network even as it grows?

I’m also wondering what to do with all those miles of obsolete land-line copper cable running to all the houses.  Hmm…

Lindi Horton, Sales Engineer, NetQoS:

Just some quick stuff as I type on my blackberry on the plan back from Networkers today.

A big kudos goes out to [Graphic Designer] Ginger [McBride] for designing a shirt that everyone loves. I have seen a dozen “Certified Network Rockstar” shirts in the airport making them the consummate fashion statement for network engineers everywhere.

As usual the party was top notch. The booth was beautifully laid out and designed. An extra thanks goes to the plush carpet and comfortable seating for the wary engineer.

All of the Cisco teams were very complimentary of the professionalism and competence of NetQoS employees and we were able to garner some great mentions in the WAN Optimization classes due to everyone's hard work.

The rain did nothing to dampen the spirits of these knowledge hungry engineers as we were all shuttled to and from hotels. Cisco certainly knows how to make a good event look effortless.

While the show had a slight feel of smaller budgets and economy wary marketers there was a distinct attempt to make those who made the journey feel at home. It was also interesting to note a severe lack of security vendors while a marked differentiation of performance focused vendors.

This year's Cisco bag is the perfect complement to the network engineer's routine with sturdy ergonomic straps and plenty of compartments for cables, testers, and the ever elusive whiteboard marker and eraser.

Patrick Ancipink, Director of Product Marketing, NetQoS:

Since I’m a pro-ITIL guy with a service management background, I was encouraged by a few extensive, practical conversations about better integrating network performance with ITIL. It demonstrated to me that IT organizations are making headway in breaking down silos and replacing oldy-van-moldy fault and device mindsets with more relevant performance metrics.

David Byrne, General Manager of Managed Services, NetQoS:

I had no idea how “cool” our T Shirts are…everybody wanted one! Great attendance at the booth continuously for our presentations, and John Chambers of Cisco stopped by to say “Hi!”

Photo of the NetQoS Booth from Tim Richardson, Customer Care Consultant

Cathryn Smalley, Marketing Coordinator, NetQoS

I heard lots of people talking about how fun our party was and that we had the best give away. I was mostly behind the scenes and scanning badges. I saw four people wearing our shirts on the plane home. I think the show was a success and people were pleased with it.

Microsoft has, according to ComputerWorld, said that it will no longer sell Windows XP after the June 30, 2008 deadline – or next Monday, as of this blog’s posting.

However, Microsoft has, according to InformationWeek, pledged to support Windows XP until 2014. Microsoft has also, according to NetworkWorld, confirmed a ship date of Windows 7 in January 2010.

Other than the significance of Microsoft landing IT-oriented public relations trifecta, the question is – are networks going to bother with changing over to Windows Vista at all, knowing that XP will be supported for four years after Windows 7 comes out?

The buzz among technology columnists is that Vista has never provided a compelling reason to upgrade for businesses. (While there are benefits to upgrading Microsoft-based servers to Windows Server 2008, Vista is a desktop/client based OS.)

Additionally, considering that any changes can impact the network, changing operating systems across multiple desktops is a significant risk for no benefit. So, even assuming that businesses test before deployment and move into major changes like operating system upgrades with trepidation, four years should be plenty of time to migrate to Windows 7, which should include a more advanced feature set. Why would any company then not choose to skip this generation of desktop Microsoft OSes, wait out 2009 while still getting support, and upgrade to Windows 7 when it comes out?

By David Byrne,
General Manager of Managed Services

Our tools here at NetQoS deliver the capabilities and insight needed to understand what’s going on across the network from a performance perspective to improve application delivery. Some customers find it difficult to maximize the performance of their monitoring tools given the constant change they’re experiencing, which includes everything from new applications to infrastructure changes to staff turnover. And ironically, customers need to fully utilize their performance monitoring tools to minimize the risk of change and ensure performance remains optimal as they deploy new technologies and make alterations to the infrastructure.

While training helps improve knowledge and skills, a managed services approach can make more sense for customers undergoing these challenges.  For example, a company may go through a rapid amount of change in a short period and simply can’t keep up with the performance monitoring on its own.  Or, perhaps a company might have trouble recruiting and retaining technical employees.  Or maybe a company realizes the IT team’s core strength lies outside performance monitoring and doesn’t want their best engineers spending all their time on it.

For these reasons, we’re announcing the NetQoS Managed Services today, which means we can offer customers the benefit of our talent and resources to perform the traditional tasks required to maintain optimal network performance. Because we can deliver our expert knowledge and best practices at a reduced cost compared to a full-time equivalent, many companies will find our Managed Services a compelling option.

Think of all the devices going out on the network today.  Years ago, you didn’t have to worry about things like streaming audio or video or BlackBerry/iPhone compatibility. But change is constant, and right now it’s exploding, which makes putting the right technical resources on the right tasks critical.  For many companies, it doesn’t make sense to have their broadly skilled engineers focus only on one or two tools.   There is a value to focusing on performance, but for many companies, NetQoS Managed Services might be a better way to do it. 

By Kevin Davis
Adapted from “Sources of Latency” Whitepaper

When network users call the Help Desk to report poor application performance, you don’t typically hear things like “The router’s CPU is too busy!,” “The network utilization is above 70%!,” or “The carrier path has failed-over to a sub-optimal path.” Instead, what you’re likely to hear is “The network is slow” or “The calls on my IP phone sound terrible.”

Complaints that end-users lodge are nearly always based their quality of experience using the application. And their quality of experience is almost always reliant on time.

Anytime a significant delay occurs in the delivery of network data, application performance suffers. Depending on the type of application and how it works, variances in network delay can have a severe impact on application performance thereby degrading end-user’s experiences.

Two important measurements of time intervals in network transmission systems are referred to as “latency” and “jitter”. Understanding latency and jitter sources and how their values vary in network architectures is critical to engineering application performance and optimizing information resources. For many regular readers, this will be old-hat, but we’ll go over it again.

Network latency is the amount of time it takes for a packet to be transmitted end-to-end across a network and is composed of five variables:

Network Latency = (Distance Delay) + (Serialization Delay) + (Queue Delay) + (Forwarding Delay) + (Protocol Delay)

Serialization Delay refers to the amount of time it takes for a network interface (such as a router’s interface or computer’s NIC) to perform bitwise transmission of a frame unto the outbound media, Forwarding Delay is the amount of time it takes a network device to process a frame/packet by performing a destination address lookup and forwarding the frame/packet to the outbound interface, and Protocol Delay is the amount of time that access or transmission algorithms may contribute to the delay of a network frame, and is typically introduced at the endpoints of the data transmission system.

Serialization delay, on a per-packet basis, becomes insignificant at data rates above 1.544 Mbits/s – or a T1. Forwarding delay is typically insignificant in modern routers and switches (when appropriately configured – significant delay can occur in misconfigured routers.) And Protocol delay typically occurs at the access layer or the end points. So the two major variables that have the most effect on network latency are Distance Delay and Queue Delay.

Distance Delay is simply the minimum amount of time that it takes the electrical signals that represent bits to travel down the physical wire. Optical cable sends bits at about ~5.5 µs/km, copper cable sends it at ~5.606 µs/km, and satellite sends bits at ~3.3 µs/km. (There are a few additional microseconds of delay from amplifying repeaters in optical cable, but compared to distance, the delay is negligible.)

Distance delay can have a significant impact on application performance for applications that require a large number of network round trips in order to complete a transaction – for example, custom transactional based applications, database queries, and VoIP, which begins do degrade when one-way end-to-end latency exceeds 200-220 milliseconds.

One of the biggest sources of end-user ire are database queries designed to run over a LAN ported to the WAN. For example if a user executes a SQL database query that requests 100 rows of a database table, one row at a time, over a link with a latency due to distance of 60 ms, it would take approximately 6 seconds (60 ms * 100 turns) to complete the transaction. The same query executed by a user on a LAN connected to the same database server would take less than 2-3 ms to be completed, as the latency due to distance across the LAN is insignificant.

Queue Delay is the amount of time a packet must spend in a network buffer waiting its turn to be transmitted. Network interfaces transmit one frame at a time, typically one bit at a time. As such, when two or more packets are forwarded to a network interface at the same time, or close to the same time – one packet is transmitted while the others are put in a queue on the interface buffer to await their turn at the interface. Packets that are put into the queue must wait until they can be transmitted, adding milliseconds of delay.

Increases in Queue Delay can be measured and detected by monitoring traffic along a given network path. Typically, most intermittent increases in latency above the baseline distance latency can be attributed to network congestion. (In order to reduce the possibility of excessive queue delay, application servers that are members of the same application architecture should be placed on the same Ethernet switch and on the same VLAN to ensure they do not have to compete for uplink bandwidth when problems like the one pictured above occur.)

Worse still, if the problem gets worse and packets wait in increasingly longer lines within the queue, the buffer may become full and the packets may be dropped. Packet drop, in turn, causes TCP connections to throttle back on the rate of transmission.

Those are some of the main causes of latency – but what about jitter?

Jitter is a term that refers to the variance in the arrival rate of packets from the same data flow, and abnormal jitter values can negatively impact real-time applications like VoIP and video. Jitter is typically created by three different mechanisms in a network: variance in Serialization Delays due to variance in packet sizes, variance in per-packet Queue Delay due to packet spacing from multiple sources at a common outbound interface, or packets taking different routes from source to destination – perhaps due to per-packet load sharing or routing issues.

The most effective way to deal with jitter is by using low-latency queuing for VoIP and video traffic on network interfaces with large serialization and/or queue delays. In addition, endpoints (such as IP phones) can use jitter buffers or playout delay buffers in order to deliver received packets at a constant rate to the end consumer. These buffers are typically 30-50 ms in depth, and thus they attempt to manage jitter values within these values on any single one-way path. While these buffers technically add 30-50ms in latency, they significantly reduce jitter. Since human beings don’t start to notice latency in VoIP or VideoIP applications till it hits about 200ms, if latency can be kept to under 150 milliseconds, then jitter can be significantly reduced using this method.

By Brian Boyko

We'll get back to network performance, routers, switches, IT and enterprises shortly, but something happened today and I think there’s someone out there that should deserve some recognition.

So, before I tell you this story, I should tell you a little bit about the circumstances surrounding it. I’m 29 years old, weigh in at 300 pounds, and while much of it is fat, a whole bunch of it is muscle too.  Long story short, when people can’t remember my name, they often address me as “big guy.”

I went to the local supermarket to pick up some lunch.  I load up my heavy items in the trunk, and head towards the driver’s side of the car. 

That’s when I get hit with what feels like someone throwing a cactus at the back of my head at high velocity.  When the initial shock wore off, I was able to figure out that it wasn’t, in fact, a drive-by cacti, but a crow who had attacked me for some as-yet-unspecified reason.

My immediate suspicion was that the bird was somehow attracted to – or annoyed by – the mousse in my hair.  See, I’ve got kind of a reputation around here of being the local mad scientist/evil genius, and I’m trying to cultivate mad science hair.  I think I was shooting for something a lot like Ludvig von Beethoven but every time I tried it ended up more like Willard Scott’s Bozo – so I just slicked it back and vowed to try again the next day.  But it didn’t turn out to be the mousse.

Some supermarket attendees who saw the exchange explained to me that on the other side of the car rested a baby bird, who would not move.  As such, the parent birds attacked me for having the audacity to get between them and the baby. 

Various methods of getting to the car were tried, but ultimately failed. I tried to get to the car through the passenger side, only to be warded off by the threatening caw-caws of Poppa Crow, caws that penetrated to my very soul.

I then informed work that I might be a little late in getting back to the office. 

I called Austin’s local non-emergency service – 311 – to get in contact with animal control, hoping that they somehow had Steve Irwin’s lost twin brother on staff who could guide me to getting to my car without provoking the large animal.

Okay, well, strictly speaking, a crow isn’t very large, but it certainly hit above its weight class.  Anyway, I was informed that animal control really doesn’t “do birds.”

It was along this time that I saw, from a distance, a young girl, who couldn’t even be in middle school yet, holding a small handtowel, and approaching the baby bird.  I warned her about the crow that attacked – she just kept on heading towards the baby crow, scooped it up into the towel, and took it to safety.

The little girl, Naomi*, is nine years old. I gave her my business card and earlier today, she e-mailed me and told me that the baby bird was eating, and that the bird trusted her like she was its mom.

I want to thank you, Naomi.  You showed both bravery and compassion that should be applauded. 

(Now, if only I was a bit less of a 300 pound wimp!)

*Because she is a minor, "Naomi's" name has been changed.

As you might notice, we’ve got a new “Twitter Live Feed” on the left sidebar of Network Performance Daily – this is being maintained by the thoroughly awesome Chandra Hosek, who gets to add the moniker “Twitter Wrangler” to her already impressive resume.  You can also follow our Twitter at http://twitter.com/NetQoSLive using whatever twitter-following app you prefer. 

You’ll notice that we’ve put this way down on the sidebar, so you might have to look for it.  This is for a very important reason: Twitter’s servers have this annoying habit of occasionally seizing up and drooling on themselves at random intervals. 

See, we would have liked to put the Twitter feed prominently on the page, but because many browsers, including Firefox 3, load the page line by line, any content below after the slow loading Twitter feed resolves gets delayed.  Placing Twitter at the top of the page would mean the entire site would take seconds to load. 

This is actually more than just a specific concern for us – one of the advents of the 21^st century is the idea that we get added value from our Web applications by combining with other Web applications – mashups. 

Many Web 2.0 services, Google maps, data from Craigslist, etc, provide APIs so that third parties can use the applications to develop their own applications or add functionality to their Web apps and Web sites – creating mashups of multiple applications.

So now, instead of having all your apps running in the same place, the apps are running from multiple different places which network engineers have little control over.  And in many cases this can result in a delay to the end-user – because your performance is based on the performance of other computers, other networks, other hardware. 

Some of the apps that contribute to these mashups may be in turn based on multiple services themselves.  The problem can compound itself. 

So this is a concern that should be on any network engineer’s mind when talking to the application developers. 

Do you feel like we do… about application delivery and acceleration? If so, you’re either quite, quite lonely, or you’re probably headed to the next Cisco Live in Orlando, starting this Sunday, the 22^nd of June, through the 26^th.

We’re going to be there – Ben Erwin will be giving a presentation called “A Strategic Approach to Managing the Application Delivery Network” on Monday at 6:30pm, and will be taking part in a case study on Tuesday at noon in Room 414D, called “Getting the Most from Cisco WAN and Application Acceleration Technologies”

Can’t you just feel the excitement!?

We’ll also have a booth up, with four different presentations: “Manage Yoür Ciscö Wörld with NetQöS,” “Using Netwörk Behaviör Analysis to Impröve End-üser Applicatiön Perförmance,” “Find and Sölve Cömplex Pröblems to Maximize Application Delivery,” and the totally gnarly “Unified Cömmünicatiöns – Managing Yöur Way to the Fütüre.”

And if you’re ready to rock, at the booth, we’ll be giving away “Certified Network Rockstar” t-shirts and blinking guitar pins – and afterwards, we’ll be having our 3^rd Annual Customer VIP Invitation-Only Party at the Hard Rock on Tuesday Night.

There’s a rumor Gene Simmons and Ozzy Osbourne will show up. I should know. I started that rumor. Just now, actually.

We also have a Photobooth, Tattoo Artist, DJ, Food, Drinks, and we’ve set up some stations where you can party like a rockstar and shred on Guitar Hero and Rock Band!

Hell yeah!

It’s Download Day.  At 10:00 a.m. PDT, or noon, for us in Austin, Firefox 3.0 was released to the public in what the Mozilla foundation has dubbed “download day.” In fact, they’re attempting to set a Guinness World Record for “most downloads in a 24 hour period.” 

So, it was a bit of a concern to us because with all those people downloading Web browsers, there would be sure to be traffic spikes on our network. But the “Download Day” promotion is such a huge success that Mozilla is having trouble keeping their own server up. 

At 10:16 a.m. PDT, I can see a “The server at www.spreadfirefox.com is taking too long to respond” error.  Mozilla.org is also unable to resolve. 

At 10:30 a.m. PDT, it’s still not connecting, and I decide to stop hitting refresh and go and eat lunch. Mmm.  Roast Beef. 

At 11:30 a.m. PDT, Spreadfirefox.com is still not resolving, but Mozilla.org does.  That doesn’t last, however, as I go to download Firefox, I get a “Http/1.1 Service Unavailable” error.   I bring up a copy of “Waiting for Godot” in another browser window.

It is 12:00 noon on the Pacific.  Spreadfirefox.com is still not resolving. 

12:30 p.m. PDT.  Still not working.  I clean off my work desk, something I’ve been putting off for a wh—ew, is that mayonnaise?  (I hope that’s mayonnaise.)

1:00 p.m. PDT. No Firefox, but My desk is now clean.  (My closet is now dangerous.)  Time to catch up on my RSS feeds to find out if there are any interesting leads that I can investigate. Hmm.  Wine 1.0 is out, but that really doesn’t have a lot to do with network performance.  Reddit seems have problems with Firefox too.  But somebody has to be getting the browser – there’s over 8000 downloads a minute according to the counting tracker.  Wait.  Some users report the counts running backward… what, are people uploading it back?

1:45 p.m. PDT. Aha!  Finally.  The page resolves and I begin my download… and it redirects me to Firefox 2.0.0.14.  Great.

1:55 p.m. PDT. I download Opera 9.5.

2:00 p.m. PDT. Mozilla’s page finally shows a link to Firefox 3.0 – but still shows the logo for Firefox 2.  The 7.1 MB download starts at around 50kBytes/s – which is pretty lame for the usual 700kBytes/s I can get when I download from work. 

2:15 p.m. I install Firefox 3.0 and launch it.  It’s nice.  It’s certainly more responsive and uses less memory.  However, my Tab Mix Plus extension isn’t compatible, and furthermore, there’s no option to undo closed tabs.  All in all, a disappointment – if it were a restaurant, it would be infamous for slow service and bad food.

Leaving aside the whole “Undo Closed Tabs” issue, you would think that an organization actively trying to beat the world record for the most downloads in a 24 hour period might, you know, be prepared enough to make sure the servers don’t go down?

Additionally; Mozilla has been promoting “Download Day” for some time now, so it makes sense for IT departments to be prepared for the onslaught of downloads coming into the network from users upgrading their PCs to the latest version of the browser – and keep track of the impact that traffic has on the user experience for more mission-critical apps.

Apple’s new 3G iPhone will soon be issued to most of you.  Ownership of the Apple 3G iPhone is mandatory.  This message is brought to you by the Ministry of Cellphones, MiniCel. 

APPLE IS PEACE.
FREEDOM IS NO THIRD PARTY APPS.
NOT INCLUDING AN ACCESSIBLE BATTERY COMPARTMENT IS STRENGTH.

The one concern is that AT&T’s offerings limit the iPhone’s data transfer rate to 1.4Mbits/s downstream.  That’s better than EDGE, but isn’t 3G supposed to be able to produce 3.6Mbits/s, with claims that 3G would hit 7Mbits/s soon? 

From Gizmodo:

Turns out, according to AT&T people we talked to, 1.4Mbps is the capped bandwidth for all mobile smartphones on the network for a few reasons….

A major one is battery life—the faster you burn, the faster your battery dies, so going full steam at 3.6Mbps would cut you well short of that nice round five hours. A second one is cell site congestion and backhaul (carrier-speak for size of the wired dataline that connects cell sites to the actual telecom infrastructure). While everyone at AT&T, from the top down, is adamant that AT&T is "comfortable" with their ability to meet the huge data draw once 3G iPhones hit the streets, it's not like the pipe is unlimited.

Ah – this is understandable.  A glut of new iPhone users will create strain on the network, and so, in order to prevent the network from being overwhelmed, it is limiting the bandwidth – that is, the throughput of the connection – in order to ensure that every customer has a decent service rate. 

This seems to be an effective plan to dealing with bandwidth shortage on overcrowded networks. 

But… hmm, who else has been complaining about overcrowded, oversubscribed networks lately?  Who was it that was talking about ‘bandwidth hogs’ and tried a different solution?  I just can’t seem to put my finger on it…

Data capping is seems to be ISPs stock solution for dealing with what they claim to be oversubscribed networks.  Of course, we’ve showed that time and time again, data limits don’t solve bandwidth problems.  QoS policies and per-user bandwidth limits – like the one AT&T is rolling out for the iPhone 3G – do. 

Which leads me to suspect that the big “bandwidth shortage” for broadband providers is something that’s almost entirely overblown – perhaps even fictional - to derive a new revenue stream from consumers while hindering the competition to services that compete with other subsidiaries of ISPs parent companies.  Because if there was actually a bandwidth shortage, wouldn’t ISPs be doing what AT&T is doing with the 3G network?  Wouldn’t they choose the more effective solution to the problem? 

Something’s not quite right here.