How to Analyze Core Web Vital metrics of 1000+ URLs?

Bulk Analyze Core Web Vital Metrics

So I was doing a domain migration of a medium-sized website the other day. And I wanted to make sure all the pages have the same page speed metrics on old and new domains.

Analyzing 1000+ pages one by one will be obviously a boring and tedious task.

But thanks to Google, they have made PageSpeed API available.

And thanks to Screaming Frog, they let us connect the PageSpeed API key with their tool.

This solves a very big problem. We can use PageSpeed API Key and Screaming frog to analyze PageSpeed data of 1000+ URLs.

Let’s get started.

The first step is to get the PageSpeed API Key.

Click on this URL.

Then click on this button which says “Get a key”

Then it will ask you to select a project, if you do not have any project created then you will have to create one.

Choose the project and click Next.

Once you click next, it will share the API Key with you.

It looks something like this - AIzaSyAj3NwrzcQuM8YZWAxPXTASjzXv-sMbzro

(That’s a fake key btw)

This is your secret API key, so do not share it with anyone.

Now that you have an API key, open up Screaming Frog.

Go to Configuration > API Access > PageSpeed Insights

Enter your PageSpeed API Key and click on “Connect”

If your API Key is correct it will get connected.

The next step is to select the metrics you want to fetch.

You can open up these metric groups and select the specific metrics you want to analyze.

I usually go for Lighthouse and CrUX metrics.

Next, we need to make sure that Screaming Frog does not waste time in crawling the URLs. Because we just need PageSpeed metrics data.

There is no ideal way to do that, but there is a workaround.

Navigate to Configuration > Spider > Extraction

Uncheck everything. This way crawler will not waste time extracting the data from the pages.

Now you are all set to start the crawl. So hit that “Start” button.

And go to the “PageSpeed” tab once the crawl is finished.

You will have all your data there.

It will look something like this -

That’s all!

Thanks for reading.

Sharing is caring!

Tweet @stanabk if any queries.