Picture this: you just completed hours of internal Web services preparations with your system administrative team prior to the holidays. You discovered possible points of failure and made appropriate modifications with the expectation of a perfect load test. You take a few minutes to relax, refill the coffee mug sitting in front of you, and connect to the conference bridge where real-time discussion about the load test will occur. Things go well for the first 20 minutes of the test, no errors have presented themselves and your system administrator reports back that all systems are good and within normal operating thresholds.

Then it happens: one of the simulated users logs an error stating that it has timed out. Your eyes look at the error with a quizzical glance. The system administrator mentions that everything still looks good on the servers, however another error is logged with a similar message.

Your puzzled glance turns into a paralyzed stare as tens and hundreds of additional errors compound into the logs, abrogating your prior hopes that were facilitated from hours of now-futile preparations. Just as you slump into your chair, the full service load testing engineer announces, “Ah, this is good! It probably doesn’t seem as such right now, but we may have discovered a significant configuration problem with the database management system. Five minutes might be all that’s needed to fix it.”

Are Errors Bad? No!

Thomas Edison once said, “I have not failed. I’ve just found 10,000 ways that won’t work.” I doubt that Edison was referring to Web performance load testing, but it applies as much here as it did when he was referring to his inventions.

Errors are necessary to find weak spots or breaking points of your Web system and infrastructure, and are ultimately why load testing is such an important process. Discovering errors helps to reveal problems with embedded media, sourced dependency files, bandwidth allocations, database configuration as well as many other factors that could be degrading your Web system’s continual and immediate availability to prospective customers. If errors are not discovered, then one of two situations must have occurred—either you just finished going through a series of load tests and resolved the issues that were found, or you have not performed any load testing at all. Just as failure is necessary for our personal success and resilience in life, load testing errors are necessary for the success and dependability of Web services.

Load Test Errors 101: Diagnosing

Diagnosing load test errors can be a daunting task. To do so accurately requires expertise and experience with not only the front-facing Web technologies that display your intellectual data (HTML, JavaScript, Cascading Style Sheets (CSS), and more), but also with the back-end infrastructure, including the Web server, database management system and network components. (It is because of these highly technical demands that many customers turn to the Full Service Load Testing Engineers at Neustar, but if you’re a do-it-yourself person who already possesses knowledge in these areas, then you may find diagnosing load test errors to be both challenging and rewarding.)

It is important to group together errors that may relate to each other, such as one error stating that an expected element was not available on the page and another stating that half of the page failed to load. Another commonly seen association between errors is data traffic timeouts and service unavailable responses from the Web server, indicating that a fully saturated point may have been reached. It is because of these relationships among errors that it is important to follow a routine for diagnosis that makes the most sense for a given situation.

When I diagnose any number of errors that occur during a load test, I start with the highest level data that is available at that time, which is usually the general error message being logged. I then glance through the frequency of occurrence as well as the general time frame for each error, make any applicable relationships for possible root causes of errors and then dig into the object-level data to uncover underlying influences that may be instigating or affecting the errors.

These influences may include a high time-to-first-byte, poor response and transfer time on a 3rd party resource, incorrect methodologies for integrating sourced files or inline code such as CSS and JavaScript, referencing and displaying objects that are large in size (such as images) as well as many other areas of investigation. The root cause of an error may not be immediately obvious based on the error itself, but through proper attention to this object-level data as well as an in-depth understanding of Web technologies, you can make an informed decision of what to analyze next as you work to find a solution.

Converting Errors into Real Value

Many business executives and analysts equate value to not only the information learned from a situation or process, but also (and often more importantly) from the financial impact that the situation has on the organization. Proper execution of a Web performance load test and its associated error diagnosis has the potential to add significant real value to a company. The value is due, not only to discovering weaknesses and strengths in your Web services—and therefore a greater chance of success going forward—but in the beneficial cost-to-reward financial value created by such a process.