Chapter 2: Testing Tools – Jump Start Web Performance

Chapter 2: Testing Tools

Admitting your site has a performance problem is the first step on the road to recovery! This chapter provides a list of testing tools to help you understand issues using real data, showing how:

  1. the largest assets can be discovered
  2. the slowest network responses can be identified
  3. the reasons for poor browser performance can be diagnosed

It may be necessary to run tests a few times to establish a measurable performance baseline. The same tests can then be rerun to evaluate performance improvements—or deteriorations—after code has been updated.

Create a Test Plan

You should test your websites and applications for defects, ease of use, accessibility, quality assurance, and other factors. Evaluating performance is no different, and it’s best to follow a plan that lays out:

  1. who’s responsible for running tests
  2. what tools and settings will be used for each test
  3. how results will be recorded and fed back into the development process

Use performance analysis tools manually at first to understand the reports and determine optimization priorities.

As your processes evolve, it may become possible to automate these tests so developers are warned about potential problems and perhaps blocked from committing poorly optimized code. Chapter 5 describes several options for improving your workflow with build processes and performance budgets.

Automated vs Manual Testing

Automated tests are never a substitute for manual user testing! Tests are good at repeating operations to report faults, but they’re unlikely to discover issues you weren’t expecting.

It’s possible to build a wholly unusable site that fully passes automated testing. For example, a button could trigger a fast change to the page that shows skeleton content, while the real results take an hour to appear!

Identify Performance Bottlenecks

Knowing you have a performance problem is one thing. Finding and fixing the causes is another matter.

The first step is to identify whether the fault occurs server-side or client-side. A slow network response, either on the initial page load or during an Ajax request, will normally indicate a server issue. Database queries are often the culprit, but you’ll need to prove that! Existing tools and logs can help, but it may be necessary to output diagnostic information to a file in a similar way to other debugging activities.

Client-side issues can be diagnosed using browser developer tools, as described below. Performance is affected when the browser has considerable work to do—such as a long-running JavaScript function, a DOM update that causes the page to re-layout, or CSS changes that affect many elements. The tips in Chapters 3, 4, and 5 provide solutions to typical problems.

Performance Tool Concepts

Most of the tools described in this chapter diagnose a particular “page” in your site within the context of a web browser. They primarily analyze front-end performance, although a back-end server or database could be to blame for a large or slow response. (Chapter 4 provides further information about potential back-end issues and database tools.)

The Browser Rendering Process

When a site or app is first accessed by a user, the following steps occur:

  1. The browser makes an HTTP request for a specific URL. Under the hood, several network processes are taking place to resolve the domain name to an IP address and route the request to a server.
  2. The server receives and parses the request. It will reference a specific URL and may have data appended as a query string, in the HTTP header, or message body. It returns a response which, in this case, we’ll presume is HTML content.
  3. The browser starts to receive HTML data, which it parses. The document may reference further assets, such as images, fonts, style sheets, and JavaScript, which trigger additional HTTP requests to the same or another server.
  4. Eventually, the browser has enough information to start the rendering process. Behind the scenes, it has started to build the HTML DOM (document object model) which defines the page in a hierarchical tree structure. Style calculations also determine which CSS rules apply to each DOM node, and a CSSOM (CSS object model) is created for JavaScript interaction.
  5. The browser initiates the layout (or reflow) phase. This calculates the dimensions of each element and how it affects the size or positioning of elements around it.
  6. The layout is followed by a paint phase. This draws the visual parts of each element onto separate layers—that is, text, colors, images, borders, shadows, and so on.
  7. Finally, a composite phase draws each layer to the screen in the correct order.

The page is now in an initial viewable state. During or after the render, JavaScript can run to make further HTTP requests (Ajax or WebSocket calls), perform calculations, update the DOM, or apply CSS rules. This could trigger further layout, paint, and/or composite phases.

Most tools make reference to these phases, while also introducing their own metrics that often combine two or more stages. In essence, the fewer steps you require, the better the performance. Taking steps to minimize HTTP requests and reduce browser processing will result in a snappier user experience.

Google Lighthouse/Chrome Audits

Lighthouse is an open-source tool that helps evaluate the performance and quality of your page or app. You can access it in the following ways:

  1. from within Chrome’s DevTools
  2. as the online web.dev or PageSpeed Insights tools
  3. as a Node.js module, through which command-line and automated tests can be executed

It’s easiest to start with Chrome’s DevTools. Navigate to any page in Chrome, press Ctrl|Cmd + Shift + I or F12 to open the DevTools panel, and click the Audits tab:

Select the device, audit types, network speed, and check Clear storage to ensure there’s no influence from browser caching. The results screen appears shortly after clicking Run audits:

Browser Extensions

Browser extensions can affect results, but Chrome will warn you about potential issues. It may be necessary to run tests in an Incognito window since it disables extensions by default.

The Performance, Accessibility, Best Practices, and SEO scores provide a quick-view percentage result, which can be clicked for more information.

Performance information includes:

Performance Metric Description
First Contentful Paint the time when the first text or image is painted
First Meaningful Paint when the primary content is visible
Speed Index how quickly the contents of a page are visibly populated
First CPU Idle the time when the main thread is able to handle input
Time to Interactive the time taken for the page to become fully interactive
Max Potential First Input Delay the time when the browser is able to respond to interaction

The lower the figures, the better the page performance. This is followed by an Opportunity section, which suggests potential improvements and estimated savings.

Progressive Web Apps

The PWA section in Chapter 4 describes the benefits of progressive web app technologies, which allow a web application to be installed and cached, and to work offline.

DevTools’ Network Panel

The developer tools in most browsers provide a Network panel that shows a log of all network activity during page load and any subsequent file, Ajax, or WebSocket data flows.

The status bar at the bottom summarizes the number of requests, total data transfer (possibly compressed), the total size of all uncompressed resources, the total download time, and the time when the document DOMContentLoaded and window load events were triggered.

Further options are provided at the top:

  • Preserve log: don’t clear the log between page loads
  • Disable cache: load all files from the network to make a better assessment of first-time page access
  • Throttle network speed: select or define download speed profiles

Assets can be displayed, hidden, or reordered by clicking a table heading. Ordering by size or download time will help find the largest or most costly resources.

The Filter box allows you to search for specific assets or enter criteria such as:

  • is:running: show any incomplete or unresponsive requests
  • larger-than:S: limit to files larger than S, which can be expressed as bytes (10000) Kilobytes (1000k), or megabytes (1M)
  • -larger-than:S: limit to files smaller than S
  • -domain:*.yourdomain.com: show third-party requests that aren’t from your primary domain

Assets from Other Domains

Most sites request assets from other domains, such as CDNs (content delivery networks), fonts repositories, analytics trackers, advertising networks, social media share buttons, and so on. While useful, those resources can have a negative impact on performance, privacy, and security. Refer to Chapter 4 for further information.

Chrome’s Performance Monitor

Chrome’s new Performance Monitor can be accessed from the DevTools’ More tools sub-menu (although this may vary across Chrome versions).

It appears in the lower Console drawer panel, and charts are updated in real time as you use a page.

Monitors can be displayed and hidden by clicking the heading in the left:

Performance Monitor Description
CPU usage processor utilization from 0% to 100%
JS heap size memory required for JavaScript objects
DOM Nodes the number of elements in the HTML document
JS event listeners the number of registered JavaScript event listeners
Documents the number of document resources including the page, CSS, JS, etc.
Document Frames the number of frames, iframes, and worker scripts
Layouts / sec the rate at which the browser has to re-layout the DOM
Style recalcs / sec the rate at which the browser has to recalculate styles

The Performance Monitor could be used to discover unusual spikes in activity—such as rising memory use or layout recalculations when an element has been added to the page. Further investigation can then be carried out in the Performance Panel.

Developer Tools’ Performance Panel

The developer tools provided in Chrome, Firefox, Safari, and Edge provide a Performance tab that allows you to record a snapshot of browser activity when particular actions are made. Unlike the Performance Monitor, you must record a profile before it can be analyzed.

To use Chrome’s version, open DevTools and navigate to the Performance pane. The Settings cog allows you to select network and CPU throttling options before clicking the Record icon.

Load or use your site as required, then hit Stop to generate the performance report.

The report can be daunting, but it can be saved and reloaded later for further analysis.

The top timeline chart shows frames per second, CPU usage, network usage, screenshots, and the heap memory size. An area can be selected with the mouse to focus on a specific point.

Result panes can be expanded and collapsed. Versions of Chrome differ, but panes may include:

  • Network: loading times for individual files
  • Frames: screenshots at points on the timeline
  • Interactions: input and animation timings
  • Timings: events such as DOMContentLoaded and the First Meaningful Paint
  • Main: thread activities such as function calls and event handlers

These are followed by a chart showing the JavaScript memory heap, number of documents, number of nodes, event listeners, and GPU memory usage.

The final Summary panel changes as you click items in the upper panes. The breakdown may include function call and event details as well as timings where appropriate:

  • Loading: time to load assets from the network
  • Scripting: JavaScript execution resulting in visual changes (which can also include CSS animations and transitions)
  • Rendering: the browser process of calculating which CSS rules apply and how layout is affected
  • Painting: the browser process of filling in pixels and drawing layers in the correct order
  • System: other browser activities
  • Idle: no activity

The panel can typically be used to discover inefficient activities, including:

  • expensive event handlers, such as those attached to scroll or mouseover actions
  • long-running JavaScript functions
  • slow or badly throttled network requests
  • a continually rising JavaScript Heap, which could result from memory leaks or poor garbage collection
  • style changes that affect many DOM elements
  • animations that incur frequent layout changes

Chapters 4 and 5 provide common solutions to these issues.

DevTools’ Console Logs

Performance monitoring can help discover problems when specific actions are performed in a site or app. However, it may become necessary to profile JavaScript execution by logging messages to the console when events occur. Modern browsers support various Performance Timing APIs that can help to analyze code.

performance.now()

performance.now() returns the elapsed time in milliseconds since the page was loaded. Unlike Date.now(), it returns a floating-point number representing fractions of a millisecond:

let t0 = performance.now();
doSomething();
let t1 = performance.now();
console.log(`doSomething() executed in ${ t1 - t0 }ms`);

Performance Marks and Measures

performance.now() can become arduous to manage as an application grows. The API also allows you to mark when an event occurs and measure the time elapsed between two marks. A mark is defined by passing a name string to performance.mark():

performance.mark('script:start');

performance.mark('doSomething1:start');
doSomething1();
performance.mark('doSomething1:end');

performance.mark('doSomething2:start');
doSomething2();
performance.mark('doSomething2:end');

performance.mark('script:end');

Each mark() call adds a PerformanceMark object to an array, which defines the name and startTime.

Navigation and Resource Entries

The array is likely to contain other automatically generated browser entries for navigation and resource timings.

A mark can be cleared with performance.clearMarks(markName). All marks are cleared when no name is passed.

The elapsed time between two marks can be calculated by creating a performance.measure() by passing the measure name, start mark, and end mark:

performance.measure('doSomething1', 'doSomething1:start', 'doSomething1:end');
performance.measure('script', 'script:start', 'doSomething1:end');

Omitting the start mark measures from the moment the page loaded. Omitting the end mark measures to the current time. Each measure() call adds a PerformanceEntry object to the same array, which defines the name, startTime, and duration.

A measure can be cleared with performance.clearMeasures(measureName). All measures are cleared when no name is passed.

Marks and measures can be accessed with:

  1. performance.getEntriesByType(type), where type is either "mark" or "measure"
  2. performance.getEntriesByName(name), where name is a mark or measure name
  3. performance.getEntries(), to access all items in the array.

The name and duration of every measure can therefore be output:

performance.getEntriesByType('measure')
  .forEach(m => console.log(`${m.name}: ${m.duration}ms`));

WebPageTest.org

Despite its retro look, WebPageTest.org reports performance information from global locations using emulated devices with a range of settings.

DevTool-like reports take a few minutes to generate.

An A (good) to F (bad) report is shown at the top:

  • First byte time: the time taken for the first byte to be received
  • Keep-alive enabled: are persistent HTTP connections used?
  • Compress transfer: are assets gzip compressed?
  • Compress images: can images be compressed further?
  • Cache static content: does the site leverage browser caching?
  • Effective use of a CDN: are content delivery networks used?

Content Delivery Networks

The benefits of using a CDN are described in Chapter 3.

The Summary and Details panels provide a table and waterfall chart showing network metrics, where lower figures indicate better performance:

Performance Metric Description
Load Time the time taken from the initial request to the browser load event
First Byte the time taken for the first byte to be received
Start Render the time taken for the browser to start rendering content on the page
Speed Index the average time at which visible parts of the page are displayed (more information)
DOM Elements the number of DOM elements
Document Complete a set of metrics relating to the DOMContentLoaded event, when the HTML has fully loaded but other assets such as images and fonts may still be in progress
Fully Loaded a set of metrics relating to the window load event when all page assets have been downloaded and rendered

The Performance Review panel shows information about effective use of file compression, browser caching, and CDN usage.

The Content Breakdown panel provides a list of assets by type (HTML, CSS, JavaScript, images, and others) with requests, size, network response times, and rendering events.

The Domains panel shows the number and size of requests made to the page and third-party domains such as CDNs, font repositories, analytics, advertisers, social media widgets, and so on.

The Processing Breakdown panel shows the time taken for browser processes including loading, scripting, layout, and painting.

The Screenshot panel shows frames and/or a video at specific points, such as the time the first content appeared, when a hero image was shown, and the fully loaded view.

Other panels provide links to additional services such as image optimizers and domain request maps.

For more information, refer to:

WebPageTest API

WebPageTest offers a REST API to programmatically automate tests or obtain results during a build process.

Tests are limited to 200 page loads per day. Repeated views to evaluate cached loading factors also count as a page load. Developers must apply for an API key to get started.

Tests are initiated by requesting a URL. In this example, mysite.com tests are run twice:

http://www.webpagetest.org/runtest.php?k=my-api-key&url=mysite.com&runs=2&f=json

A successful request returns a JSON response with a test ID and a set of result URLs:

{
  "data": {
    "testId": "1234567890",
    "ownerKey": "ceb6128dbf05e09a1969ad",
    "jsonUrl": "https://www.webpagetest.org/jsonResult.php?test=1234567890",
    "xmlUrl": "https://www.webpagetest.org/xmlResult/1234567890/",
    "userUrl": "https://www.webpagetest.org/result/1234567890/",
    "summaryCSV": "https://www.webpagetest.org/result/1234567890/page_data.csv",
    "detailCSV": "https://www.webpagetest.org/result/1234567890/requests.csv"
  }
}

The status of a test can be checked by passing the testId to the testStatus.php URL. An HTTP 200 is returned when the result URLs are ready:

http://www.webpagetest.org/testStatus.php?k=my-api-key&f=json&test=1234567890

The completed results are stored for up to 30 days and can be accessed from the URLs shown in the initial runtest.php request.

A Node.js webpagetest module is available to help with API processing.

More Performance Assessment Tools

A variety of other free tools, browser extensions, and commercial services (often with free tiers or trials) are available, which can provide performance monitoring, usage tracking, and improvement advice:

Finally, don’t forget to test your system using a combination of real devices, operating systems, input types, and browsers on a range of network speeds. Consult your analytics reports to help determine what systems and connections your visitors typically use.