Skip to main content
SpringerLink
Log in

Web Performance Pitfalls

  • Conference paper
  • First Online:
Passive and Active Measurement (PAM 2019)

Part of the book series: Lecture Notes in Computer Science ((LNCCN,volume 11419))

Included in the following conference series:

Abstract

Web performance is widely studied in terms of load times, numbers of objects, object sizes, and total page sizes. However, for all these metrics, there are various definitions, data sources, and measurement tools. These often lead to different results and almost all studies do not provide sufficient details about the definition of metrics and the data sources they use. This hinders reproducibility as well as comparability of the results. This paper revisits the various definitions and quantifies their impact on performance results. To do so we assess Web metrics across a large variety of Web pages.

Amazingly, even for such “obvious” metrics as load times, differences can be huge. For example, for more than 50% of the pages, the load times vary by more than 19.1% and for 10% by more than 47% depending on the exact definition of load time. Among the main culprits for such difference are the in-/exclusion of initial redirects and the choice of data source, e.g., Resource Timings API or HTTP Archive (HAR) files. Even “simpler” metrics such as the number of objects per page have a huge variance. For the Alexa 1000, we observed a difference of more than 67 objects for 10% of the pages with a median of 7 objects. This highlights the importance of precisely specifying all metrics including how and from which data source they are computed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    See Fig. 2 for an overview and Appendix A for more explanation.

  2. 2.

    For details regarding the methodology and the corresponding dataset see Sect. 4.

  3. 3.

    Our tools are publicly available at https://github.com/theri/web-measurement-tools.

  4. 4.

    See http://gs.statcounter.com/browser-market-share/desktop/worldwide.

  5. 5.

    Chrome and Opera support it, Firefox is still validating their implementation.

  6. 6.

    https://chromedevtools.github.io/devtools-protocol/.

  7. 7.

    https://firefox-source-docs.mozilla.org/testing/marionette/marionette.

  8. 8.

    Note that for HTTP/2, logged header sizes do not correspond to bytes on the wire anymore due to HTTP/2 header compression.

  9. 9.

    Our scripts instrument browser automation frameworks directly to give us more control and avoid the overhead of an integrated framework such as WebPagetest.

  10. 10.

    For realistic browser behavior, which includes the rendering engine, we open Web browsers including the graphical user interface rather than using them in headless mode.

  11. 11.

    Close in terms of network distance.

  12. 12.

    18. September 2018 for Alexa 1000 and 30. September 2018 for Alexa 10001–11000.

  13. 13.

    We instruct the browser automation tool to wait for the onLoad event.

  14. 14.

    See, e.g., https://support.mozilla.org/en-US/kb/search-web-address-bar.

  15. 15.

    Server-side redirects use HTTP status 301 or 302. Client-side redirects use status 200 and contain the redirection URL in the response content, which we do not log.

  16. 16.

    For Navigation Timings, redirects are the time between navigationStart and fetchStart. For HAR files, we use the time before the first HTTP 200 response.

  17. 17.

    In September and early October 2018, TLS 1.3 was still not deployed.

  18. 18.

    See analysis script eval/validate_object_sizes.py in our repository.

  19. 19.

    Unless the ‘Timing-Allow-Origin’ header is set, see [5].

  20. 20.

    In Firefox, only HTTP/2 Server Push objects lack body size and timings.

  21. 21.

    See the examples in Sect. 4.2 of [5].

  22. 22.

    Using the Page.frameStoppedLoading event instead did not resolve this problem.

  23. 23.

    Examples include microsoftonline.com and googleusercontent.com.

  24. 24.

    After a redirect, the browser overwrites the old fetchStart value before it fetches the new URL using a GET request. This implies that once the page load is finished, fetchStart is the start time of the loading of the final base page, as all previous values related to redirects are overwritten.

References

  1. Bocchi, E., De Cicco, L., Rossi, D.: Measuring the quality of experience of web users. In: ACM SIGCOMM Computer Communication Review, vol. 46, no. 4, pp. 8–13 (2016)

    Google Scholar 

  2. W3C Recommendation: Navigation Timing. Version 17 December 2012. https://www.w3.org/TR/navigation-timing/. Accessed 29 Aug 2018

  3. W3C Working Draft: Navigation Timing Level 2. Version 30 November 2018. https://www.w3.org/TR/2018/WD-navigation-timing-2-20181130/. Accessed 17 Dec 2018

  4. W3C Candidate Recommendation: Resource Timing Level 1. Version 30 March 2017. https://www.w3.org/TR/resource-timing-1/. Accessed 29 Aug 2018

  5. W3C Working Draft: Resource Timing Level 2. Version 11 October 2018. https://www.w3.org/TR/resource-timing-2/. Accessed 13 Oct 2018

  6. W3C First Public Working Draft: Paint Timing 1. Version 07 September 2017. https://www.w3.org/TR/paint-timing/. Accessed 10 Oct 2018

  7. W3C Editor’s Draft: HTTP Archive (HAR) format. Version 14 August 2012. https://w3c.github.io/web-performance/specs/HAR/Overview.html. Accessed 29 Aug 2018

  8. Bruns, A., Kornstadt, A., Wichmann, D.: Web application tests with selenium. IEEE Softw. 26(5), 88–91 (2009)

    Article  Google Scholar 

  9. Selenium Documentation: Worst Practices. https://seleniumhq.github.io/docs/worst.html. Accessed 29 Aug 2018

  10. Meenan, P.: WebPageTest. https://www.webpagetest.org. Accessed 17 Dec 2018

  11. da Hora, D.N., Asrese, A.S., Christophides, V., Teixeira, R., Rossi, D.: Narrowing the gap between QoS metrics and web QoE using above-the-fold metrics. In: Beverly, R., Smaragdakis, G., Feldmann, A. (eds.) PAM 2018. LNCS, vol. 10771, pp. 31–43. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-76481-8_3

    Chapter  Google Scholar 

  12. Goel, U., Steiner, M., Wittie, M.P., Flack, M., Ludin, S.: Measuring what is not ours: a tale of \(3^{\text{rd}}\) party performance. In: Kaafar, M.A., Uhlig, S., Amann, J. (eds.) PAM 2017. LNCS, vol. 10176, pp. 142–155. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54328-4_11

  13. Erman, J., Gopalakrishnan, V., Jana, R., Ramakrishnan, K.K.: Towards a SPDY’ier mobile web? IEEE/ACM Trans. Netw. 23(6), 2010–2023 (2015)

    Article  Google Scholar 

  14. Qian, F., Gopalakrishnan, V., Halepovic, E., Sen, S., Spatscheck, O.: TM 3: flexible transport-layer multi-pipe multiplexing middlebox without head-of-line blocking. In: Proceedings of the 11th ACM Conference on Emerging Networking Experiments and Technologies, p. 3. ACM, New York (2015)

    Google Scholar 

  15. Wang, X.S., Krishnamurthy, A., Wetherall, D.: Speeding up web page loads with Shandian. In: Proceedings of the 13th USENIX Symposium on Networked Systems Design and Implementation (NSDI 2016), pp. 109–122. USENIX Association (2016)

    Google Scholar 

  16. Wang, X.S., Balasubramanian, A., Krishnamurthy, A., Wetherall, D.: Demystifying page load performance with WProf. In: NSDI 2013, pp. 473–485 (2013)

    Google Scholar 

  17. Butkiewicz, M., Madhyastha, H.V., Sekar, V.: Understanding website complexity: measurements, metrics, and implications. In: Proceedings of the 2011 ACM SIGCOMM Conference on Internet Measurement Conference, pp. 313–328. ACM, New York (2011)

    Google Scholar 

  18. Kelton, C., Ryoo, J., Balasubramanian, A., Das, S.R.: Improving user perceived page load times using gaze. In: Proceedings of the 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 2017), pp. 545–559. USENIX Association (2017)

    Google Scholar 

  19. Varvello, M., Schomp, K., Naylor, D., Blackburn, J., Finamore, A., Papagiannaki, K.: Is the web HTTP/2 yet? In: Karagiannis, T., Dimitropoulos, X. (eds.) PAM 2016. LNCS, vol. 9631, pp. 218–232. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-30505-9_17

    Chapter  Google Scholar 

  20. Netravali, R., Mickens, J.: Prophecy: accelerating mobile page loads using final-state write logs. In: Proceedings of the 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 2018). USENIX Association (2018)

    Google Scholar 

  21. Netravali, R., Nathan, V., Mickens, J., Balakrishnan, H.: Vesper: measuring time-to-interactivity for web pages. In: Proceedings of the 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 2018). USENIX Association (2018)

    Google Scholar 

  22. Netravali, R., Goyal, A., Mickens, J., and Balakrishnan, H.: Polaris: faster page loads using fine-grained dependency tracking. In: Proceedings of the 13th USENIX Symposium on Networked Systems Design and Implementation (NSDI 2016). USENIX Association (2016)

    Google Scholar 

  23. Zaki, Y., Chen, J., Pötsch, T., Ahmad, T., Subramanian, L.: Dissecting web latency in Ghana. In: Proceedings of the 2014 Conference on Internet Measurement Conference, pp. 241–248. ACM, New York (2014)

    Google Scholar 

  24. Han, B., Qian, F., Hao, S., Ji, L.: An anatomy of mobile web performance over multipath TCP. In: Proceedings of the 11th ACM Conference on Emerging Networking Experiments and Technologies, p. 5. ACM, New York (2015)

    Google Scholar 

  25. Naylor, D., et al.: The cost of the S in HTTPS. In: Proceedings of the 10th ACM International on Conference on Emerging Networking Experiments and Technologies, pp. 133–140. ACM, New York (2014)

    Google Scholar 

  26. Scheitle, Q., et al.: A long way to the top: significance, structure, and stability of internet top lists. In: Internet Measurement Conference 2018. ACM, New York (2018)

    Google Scholar 

  27. Let’s Encrypt: Percentage of Web Pages Loaded by Firefox Using HTTPS. https://letsencrypt.org/stats/#percent-pageloads. Accessed 30 Sept 2018

  28. Egger, S., Hossfeld, T., Schatz, R., Fiedler, M.: Waiting times in quality of experience for web based services. In: 2012 Fourth International Workshop on Quality of Multimedia Experience (QoMEX), pp. 86–96. IEEE (2012)

    Google Scholar 

  29. Barth, A.: The web origin concept. RFC 6454 (2011)

    Google Scholar 

Download references

Acknowledgements

Thanks to Dominik Strohmeier for the discussion and the pointers to resources, to our shepherd Jelena Mirkovic, as well as our anonymous reviewers.

Author information

Authors and Affiliations

  1. TU Berlin, Berlin, Germany

    Theresa Enghardt & Thomas Zinner

  2. Max-Planck Institute for Informatics, Saarbrücken, Germany

    Anja Feldmann

Authors
  1. Theresa Enghardt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Zinner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anja Feldmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Theresa Enghardt .

Editor information

Editors and Affiliations

  1. Northeastern University, Boston, MA, USA

    David Choffnes

  2. Federal University of Rio Grande do Sul, Porto Alegre, Rio Grande do Sul, Brazil

    Marinho Barcellos

Appendices

A Web Page Load Explained

In this section, we explain a Web page load in more detail. See also Fig. 2 and the processing models in the Navigation Timings specifications [2, 3].

The starting point for a new page load, also called navigation, of a particular URL, is called navigationStart in [2]. Initially, fetchStart is set to the same value, but if a redirect occurs, fetchStart is overwritten before the new URL is loaded.

If another page has been previously loaded by the browser, e.g., in the same browser tab, this document has to be first unloaded. Then, the browser checks its cache to see whether the page is already there. If the page is not in the cache, the browser usually resolves the hostname (resulting in a DNS query and usually answer), establishes a TCP connection, and performs a TLS handshake if the scheme of the URL is https. Then, the browser issues an HTTP GET request for the URL. As soon as it receives an HTTP reply, which always contains a status line, headers, and body, the browser processes the reply.

If the reply contains an HTTP status code of 3xx, such as “301 Moved Permanently” or “302 Found”, this means that the server redirects the browser to a different URL, which is given in the “Location” header in the HTTP response. This redirect may be a same-origin redirect, which roughly means that both the old and the new URL have the same scheme (http or https), hostname, and port (see RFC 6454 [29] for details), or it may be a cross-origin redirect. For same-origin redirects, the start and end time of the redirect are recorded as Navigation Timings redirectStart and redirectEnd [2], while for cross-origin redirects they are not. Unfortunately, nearly all redirects we observed are cross-origin, as the purpose of the redirect is to use a different scheme (HTTPS instead of HTTP) or hostname (www.example.com instead of example.com). The same-origin policy is an important security and privacy feature in the Web, so information access is often restricted to, e.g., the same hostname.

Given the new URL to be fetched, the browser records the current time as fetchStart, potentially overwriting the old valueFootnote 24. It then checks its application cache again, resolves the host name if needed, establishes a new TCP connection, performs a new TLS handshake, and sends an HTTP request for the new URL. If it gets an HTTP reply, this may be another redirect, an error code such as “404 Not Found” or“503 Internal Server Error”, or the request may succeed with a “200 OK”. In the latter case, the body of the HTTP response usually contains the base document of the Web page in HyperText Markup Language (HTML). As soon as the browser starts receiving this document, it parses it and starts constructing the Document Object Model (DOM) of the page. For example, the document may reference additional resources, such as JavaScript, Cascading Stylesheets (CSS), or images. Typically, for each of these additional resources, the browser has to issue a new HTTP request, unless the resource is proactively sent by the server using HTTP/2 Server Push. Each new HTTP request may involve an additional name resolution, TCP handshake, and TLS handshake, because resources are often hosted on different servers than the base page. The browser now simultaneously fetches new resources, continues to parse the HTML base page, and processes the CSS and Javascripts, even though these processes may block each other. See Wang et al. [16] for a detailed explanation of this complex process.

At some point, the browser flushes the current state of the DOM to the rendering engine. The time at which this happens corresponds to Time To First Paint (TTFP). The point at which all resources in the DOM have been loaded is called DOMContentLoaded and recorded in the Navigation Timings and HAR file. However, processing of the page usually continues, until, eventually, the browser fires the onLoad event for the page which is recorded in the Navigation Timings and HAR file. The onLoad Time is usually taken as Page Load Time (PLT). At this point, the page load is considered finished. However, onLoad usually triggers the execution of one or more javascripts, which may result in loading more resources, sending data, e.g., to third parties, or other network traffic. In fact, most modern Web pages load resources continuously long after the onLoad event. Thus, Related Work usually stops counting objects after onLoad.

B Details of Lessons Learned

Next, we outline additional details regarding our lessons learned, which led to our guidelines for Web performance measurement, recall Sect. 6.

Software Versions: The Debian Linux distribution includes a version of the Firefox browser which is usually quite dated. This can have a major impact on load times. For instance, in Firefox version 61 (“Firefox Quantum”), parts of the code have been rewritten and optimized, which makes the browser much faster than previous versions. Consequently, carrying out Web page loads using an older version results in unrealistically long load times. However, updating Firefox frequently to the newest version can result in incompatibilities with measurement tools. For instance, not every version of the HAR Export Trigger extension works with every version of Firefox, so it has to be updated along with the browser. However, the upside is that in newer versions of Firefox, HAR Export Trigger is supposed to work without having the developer panel open.

Browser Traffic Unrelated to Page Loads: Modern browsers usually issue a significant number of requests that are not directly related to the page load that a user has requested. For instance, Firefox by default loads blocklists for “safe browsing”, to protect users from malware or phishing. It also automatically checks for updates and may even automatically download and install these updates for the entire browser or for individual browser extensions. These queries can involve substantial data transfers: For example, we observed the automatic download of a binary related to an H264 media component which we never activated or requested: 500 KB were downloaded in the background. Worse yet, the state of such updates is often stored in the browser profile. This may cause such downloads to be triggered for every fresh browser profile, i.e., each of our page loads. Additionally, the Chrome browser by default issues queries to various Google servers, e.g., it tries to connect each browsing session to a Google account. We provide configurations for Firefox and Chrome to turn off most features that generate such traffic, see our repository https://github.com/theri/web-measurement-tools.

Logging a Trace and Client-Side SSL Keys: To be able to better debug and validate measurement setups and tools, we recommend capturing packet traces that include at least ports 53 (DNS), 80 (HTTP), and 443 (HTTPS). Encrypted traffic can be decrypted after logging the SSL session keys within the browser: Firefox and Chrome log keys into a specified SSLKEYLOGFILE. Note that this option must be compiled into Firefox. It, e.g., does not work with the Firefox binary in the Debian repositories.

C Artifacts Related to This Paper

The following artifacts are available:

Our Tools, Such as Measurement and Evaluation Scripts: See https://github.com/theri/web-measurement-tools. This repository includes the scripts to automatically load Web pages using Firefox with Selenium and Marionette, and using Chrome with DevTools. Furthermore, it includes the analysis scripts we used to generate our plots.

Data Set of Web Page Loads: See http://dx.doi.org/10.14279/depositonce-8100. This dataset includes data from all of our experiment runs, see Sect. 4. It can be used along with our evaluation scripts to reproduce the plots in this paper, see https://github.com/theri/web-measurement-tools for details.

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Enghardt, T., Zinner, T., Feldmann, A. (2019). Web Performance Pitfalls. In: Choffnes, D., Barcellos, M. (eds) Passive and Active Measurement. PAM 2019. Lecture Notes in Computer Science(), vol 11419. Springer, Cham. https://doi.org/10.1007/978-3-030-15986-3_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-15986-3_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-15985-6

  • Online ISBN: 978-3-030-15986-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation