So how do I measure the actual performance of a website in 2021?
Sorry, the answer isn't what you expected.
First a very brief background of me
Just to put some kind of trust into the content of this article.
- I started using computers before the internet arrived, and I boarded the public internet (yep, there has been a private, locked internet that existed before us deadly could use it) in the early 90s, making me a relatively experienced internet user
- I have for the most been on the “backside” of the internet, building countless websites, hosted and managed servers, infrastructure-stuff and what not, for others to be able to just enjoy it
- I have been in charge of building several hundred e-commerce sites over the last 15 years, where performance has been a very significant element
- I have alongside a fantastic A-team of developers and technicians tried to optimize one of the world's largest e-commerce platforms, which at the same time has been maybe the most difficult to optimize (yep, I am obviously talking about Magento), and which has enforced us to learn almost too much about web performance optimizations
- I am currently leading several large web projects, where performance is still a key factor we focus on every single day
- I am currently an advisor, mentor and what not to lots of companies needing technical guidance to platforms, technology — and performance optimizations
And, maybe most importantly, I have a true passion for blazingly fast websites, and celebrate every performance win with insane parties.
Cool, seems like you know your stuff! So, how do I measure and audit the actual performance of a given website myself?
Short answer: You don't.
Dude, what?!
No. Honestly. You don't.
Uh, okey, you actually mean it? How come?
Web performance is some hardcore technical sh*t, and so unless you are an experienced web developer you should definitely let a professional do a website performance audit. Case pretty much closed.
(scroll to the bottom of this article to find your non-technical alternative to 'real' performance audits)
Just how complex is it?
It's complex.
At least I think it's complex, so I assume most others do too.
Measuring and auditing a website's performance immediately involves things like First Contentful Paint, Largest Contentful Paint, Time to Interactive, Total Blocking Time. And not to forget Cumulative Layout Shift.
If any of those elements are unknown to you, then sorry, you don't classify to do a true web performance audit. Honestly.
Seriously? But there are so many tools where I can just paste in the URL and hit “Audit” and get the results.
Surely you can simply paste in any given URL into performance auditing tools like Pagespeed Insights, WebPageTest, Pingdom, and get a result back in a short minute.
The issue with this is two-fold: first, you need to interpret and truly understand these results, and secondly, the results given from most of these tools are not always in sync with how the actual visitors experience the website's performance.
Let's take an example
www.getinspired.no, an e-commerce site serving customers in Norway.
Disclaimer / for the record: I have been central in the creation of this website as an advisor, from the UX to many other things.
For Norwegian customers this site is fast. Like, very fast.
Surveying many visitors of the site stated that the website felt 'instant' all the way through, from the frontpage to the category- and product page, and all the way to the order success page.
So, how does the world's most used web auditing tools rank this website?
Bad. Like, really bad.
- Pagespeed Insights: 24/100 (red alert!)
- WebPageTest: D-D-A-A-A-B for the various elements they audit, where A is the best in each
- GTmetrix: E (48%), where A (100%) is the best
- Pingdom: D (64%), where A (100%) is the best
As we can see the overall score is.. well, at best average, and far off to what the actual users of the website feel when using it.
But, that Get Inspired-side seems like a “local” niche site. What about… Amazon.com?
Get Inspired's annual revenue online is around 500 million NOK (USD $60M), so it's not a small site.
Anyway, Amazon.com you said? Running an audit with Pagespeed Insights gives a score of round 56/100 on www.amazon.com. 56 of 100! On Amazon!
And the Amazon-folks really knows their shit on this topic.
Doh! You are right! How come the tools rank the website so bad?
Here comes the core of the issue and difficulty of doing performance auditing: more or less all the online performance auditing tools tries to be smart and displays a summarized score with some clear colors:
- I see red colors! (a.k.a my site's performance is crappy — we have to fix it!)
- I see orange colors! (a.k.a my site is pretty OK I guess, but we need to improve it right?!)
- Halleluja! I see green colors! (a.k.a my site's performance gotta be great!)
Let's pick one of dozens of performance factors, and try to deep-dive a bit into the technical bits and pieces
Ready? Now things start to get just slightly complicated.
If we look at the details on why this website in particular ranks relatively poorly compared to the actual performance, we only need to look at one of the dozens of factors the test has taken into consideration.
Let's meet… CLS, or Cumulative Layout Shift
Here is Google's definition of CLS:
CLS is a measure of the largest burst of layout shift scores for every unexpected layout shift that occurs during the entire lifespan of a page.
Have you ever been visiting a website like a product page, when something suddenly changes on the page? Without warning, the text moves (e.g bumps), and you’ve lost your place. Or worse: you’re about to tap a link or a button, but in the instant before your finger lands — BOOM — the link moves, and you end up clicking something else!
Most of the time these kinds of experiences are just annoying, but in some cases, they can cause real damage, like you hitting “Confirm Order” when you actually were supposed to un-tap the “Sign me up for the Newsletter”-checkbox above.
As you can see the performance audit actually checks elements that are not strictly speaking part of the performance.
Oh, so, I can't trust these tools?
Yes and no.
The tools use very concrete technical pre-specified functions to do the audit, like pre-defined browser versions which might be way behind the ones used by the actual visitors. In Norway most people have pretty modern and up-to-date mobile phones, way ahead of the virtual device specifications used for the audit tests.
Some of the tools also throttle down the bandwidth to 3G. In Norway “nobody” uses 3G any more. 3G has a maximum speed of up to around 20Mbps whereas a 4G-based phone can get theoretically up to 300Mbps, and we are now even starting to see 5G networks popping up.
So, the results are correct, but as you can see they often have very central factors that are simply wrong compared to the actual users' behaviors on the website.
Okey I get it. But, if I'd want to use one of the tools anyway, which one should I use?
If you absolutely need to run a performance audit yourself, then WebPageTest is the one to use.
Not only has it been in the works and optimized over many years, but it also doesn't simply give just one red/orange/green score, but instead a more comprehensive overview of the most important elements when it comes to website performance.
What’s even more important is that WebPageTest relatively clearly shows the Real User Data from the Chrome User Experience Report.
The.. what?
The Chrome User Experience Report.
Remember when you ticked 'Yes' on all selections when installing the Chrome browser years ago? No? Well, now you can thank yourself for ticking it, as you have now been sending anonymous visitor-data to Google for all the websites you have visited. These data has been sorted by Google and given back (still anonymously) as part of the open User Experience Report.
In the below example you can see that the website being audited has 96% of its visitors within the 'Good' rank of the FCP. The LCP element scores a bit poorer, with only 62% of the visitors hitting the 'Good' rank, while 25% hits the 'Needs improvement', while 13% of the visitors to the website scores poor. CLS is a bit worse, while the FID is great and most visitors (97%) gets a full score on that metric.
Darn! I am still a bit confused. Can you sum this up in a human readable language?
First of all, just that you have reached the end of this article means you actually want to learn how to do actual website performance audits, which is a very good start!
Secondly, given you are a bit confused still means you belong to the category of people who really shouldn't do real performance audits, which is somewhat the purpose of this article to put out straight.
So, if I am indeed a non-technical person, how could I audit the performance of an e-commerce website anyway?
Alternative 1: Easy
Duration: 1 week
- Find a URL to a semi-random category page and product page of the website you are auditing
- Run the WebPageTest on the frontpage + each of the category- and product-pages. Save the Speed Index number for each of the three runs
- Sum up the average Speed Index, weighting the frontpage 20% and the two other pages 40% each (prepare to spin up your spreadsheet math skills to the limits!)
- Do this three times during one single week, with one of the tests being run during the website's assumed peak (which should be Sunday between 7pm-11pm if you have no clue on when its expected peak is)
- Calculate the average for the three runs of all the three days
The number you then have in front of you could be treated as the official score of the website's performance.
If the number is +/- 1000 (1 second) to 4493 (4,49 seconds) then you can categorize the website as “OK”. If its below (3493 or lower) it can be ranked “Fast”, and if its above 5593 it can be ranked as “Slow”.
Alternative 2: Very easy
Duration: 5 minutes
Visit the website in question. Click around for a few minutes.
Do you feel any friction? Do you have to wait for elements to pop up? Are you twisting your thumbs while waiting for the product to be added to the cart?
No? Well, then you can safely define the site as 'Fast' (a.k.a 'Green').
Btw, I have read that performance and SEO comes hand in hand? How do I deal with that?
Having worked with technical SEO for decades I can for sure confirm that performance is an important ranking signal for Googlebot, which highly likely uses the mentioned User Experience Report as one of its most important data-sources for that particular rank out of the total rank of the website- and URL.
So, the only way to have the performance parts covered in terms of SEO is to make sure all those factors are green (e.g within the top 75% of the visitors of the website), for just to satisfy Googlebot.
Remember the Friends episode with Phoebe and Rachel and the issue with the left “phalange”? No? At least that's a good example of something that doesn't matter, but suddenly start mattering, a lot.
Happy performance auditing!
