Let’s Get Started #2 „Dance browser. Dance!“ – an Intro to Puppeteer

I’m gonna walk you through an early version of a script that I built for my npm package wps. I’m not gonna go though the whole thing and all its current features, but rather extract two of them and build them as a stand-alone file to show you Puppeteer’s potential. The script is a simple one, whose goal is to accomplish two things:

I’ll divide this into steps and at the end of each I’ll add a link so you can see each change I’ve made on each file. It’ll look like this:

current state: [master] initial commit – js file created · Duclearc/puppeteer-demo-lemundo@533cbeb

First, we should make a folder to keep things tidy when we install dependencies. Naming doesn’t matter here. Then we move into it and then we must create a JavaScript file to house all this. I’m calling mine puppeteer.js, but you can call yours whatever you want.

Next: to use Puppeteer, we going to download it, and to do that we need npm. So navigate to your Repo and run npm i puppeteer. This will install Puppeteer on your project.

All steps above are summarised here:

You might also add the npm start script to your package.json. You can either do that manually or run this from your Repo’s terminal:

Either way, the result should be:

To test that this entire setup is working, add a console.log to your puppeteer.js file and run npm start to see if it worked.

current state: [master] step 1 complete · Duclearc/puppeteer-demo-lemundo@ff97734

Next, we have to figure out how we are going to get Puppeteer to run. It only works on the browser, so we must launch one. Thankfully, Puppeteer takes care of that for us.

Let’s start by importing Puppeteer. Personally, I don’t feel like writing that all the time, so I’ll just name it ppt. Then we can create an anonymous async function that runs as soon as the file is called. It can be populated with a simple test:

• launch browser
• wait 3 seconds
• close browser

Activate that with npm start and you should see your browser opening and closing… by itself.

This is all thanks to the headless: false attribute. Take it out and run the code again. See what happens.

Nothing, right?

Wrong.

Your browser is still being opened and closed, just like before. But with headless set to true. That’s its default value. And it means that the browser opens without a graphical user interface (GUI). So you don’t see anything. For some tasks that might be ok, but we want to see the magic happening, so we’re gonna keep headless as false.

But since all of this is working, we can proceed with goal 1:load an URL in a given viewport (approximately the sizes of a mobile, tablet and desktop devices).

For that we should collect the viewport (read: “screen-sizes”) of a mobile, a tablet and a desktop. Since there’s about a million ones we could use, I’m going with a modern iPhone, iPad and MacBook. I’ll assemble them as an array, so it’s easier to use later. The format must be the Viewport type from Puppeteer (so an object with width and height). I got those from here: Viewport Size for Devices | Screen Sizes, Phone Dimensions and Device Resolution | YesViz.com

Having that in place, we have to pass it on to the browser we’ve just launched. There’s a few ways of doing this like setting a defaultViewport or navigating to each page in parallel… but what I’m gonna do is navigate to a page first, then set the viewport and navigate to the website we’d like. It makes the code a bit tidier in my opinion.

But wait… what’s that page thing? I thought we had the browser open already?

You were right. And I recommend you play around with this for a few minutes to get it clear in your head. But to sum it up: think of a browser as the program itself. And a page is a tab that you can use to access a url. When you open a browser it does not already have a page open. So you have to open one anyway.

Let’s start with that.

As usual, Puppeteer makes this very easy: create a constant to store your page and initiate it with browser.newPage(). To make sure everything works well, it is recommended that you use await on most Puppeteer operations. Next, we set the viewport we want with page.setViewport(viewports[0]) and last we just have to go to a url page.goto(‚https://github.com/Duclearc‘). Now run your script!

You should see a webpage on a viewport about the size of an iPhone’s.

Brilliant. Trouble is, we want to load several viewports, not just one. So let’s tweak this a bit. We’ll nest this into a loop and let it run again. This should be happening faster than we can see, so it’s wise to add a console.log() here somewhere so we can get a feedback on what’s happening.

As before, the browser should open, flick it a bit as it goes through the viewports and reloads the page, plus this is what you should see in your console.

If you got this, perfect. Now we going to get the screenshots working.

current state: [master] step 2 complete · Duclearc/puppeteer-demo-lemundo@20cf082

Alright. So if you were thinking that taking screenshots was gonna be easy, you’re absolutely right. Puppeteer takes care of that for us too. All we going to do is specify “where” we want it to be saved, a unique “name” of the file and the “format” of the image. You pass these into a single string to the page’s screenshot() method as a path. First things first though: creating a place to put them. I’ll keep it simple and create a screenshots/folder at the root of the project. You can either do this manually or via the terminal with the command bellow:

Having that covered, we simply add one line to our loop:

That’s it. To break that down: Puppeteer will take a screenshot (.screenshot()) and save it on our folder ({path: `./screenshots/…) with the name of our index – i –, since it has to be unique, (…${i}…) and in a jpg format (… .jpg`});).

That is literally it. Now when you run npm start, you should end up with 3 files on your folder, one for each of our viewports.

This basically it. All the functionality we wanted is done. Personally, I’d like to do some cleanup and a bit of refactoring to make this nicer. Feel free to expand on this yourself, if you like. But if you’re curious, stick with me a little longer.

current state: [master] step 3 complete · Duclearc/puppeteer-demo-lemundo@f1384ad

I’ll make this last step quick and just show you what’s bugging me without going into too much detail. Again, if you want you can go to GitHub and check this whole thing out yourself if you want to get more out of it.

current state: [master] step 4 complete · Duclearc/puppeteer-demo-lemundo@efd1605

That was it! You can find the whole code here: GitHub – Duclearc/puppeteer-demo-lemundo: companion code to my ‚Intro to Puppeteer‘ article for Lemundo

There’s loads more that could be done with this script. A few ideas are:

Why I’m not doing them here now? Well, I didn’t want to rob you of all the fun of discovering the wonders of Puppeteer for yourself. Also I’m hungry. I don’t want to sit here and write anymore. 🤪
Still, leave a comment if you want a tutorial on any of those. If there’s enough of you out there, I might revisit this.

For more tech content visit my first Let’s Get started article on LHCI or see what our digital technologies team is up to on our website. We love to exchange ideas and talk all things internet.

And before I forget: we are hiring

So why don’t you check the job offers below.

Happy coding! – Dan

Want to read this article in German? Then click here.