Methods to scrape Google Maps Critiques?

On this submit, we are going to be taught to scrape Google Maps Critiques.

Necessities:

Earlier than we start, we’ve to put in the whole lot we might have on this tutorial to maneuver ahead.

1. Node JS
2. Unirest JS
3. Cheerio JS

So earlier than beginning, we’ve to make sure that we’ve arrange our Node JS undertaking and put in each the packages – Unirest JS and Cheerio JS. You possibly can set up each packages from the above hyperlink.

We’ll use Unirest JS for extracting our uncooked HTML knowledge and Cheerio JS for parsing our extracted HTML knowledge.

Goal:

Eiffel Tower Google Maps Outcomes

We’ll goal to scrape the consumer critiques on Eiffel Tower.

Course of:

Now, we’ve arrange all of the issues to be wanted for making ready our scraper. We’ll use an npm library Unirest JS to make a get request to our goal URL, so we are able to get our uncooked HTML knowledge. Then we are going to use Cheerio JS for parsing the extracted uncooked HTML knowledge.

We’ll goal the sort of URL:

`https://www.google.com/async/reviewDialog?hl=en_us&async=feature_id:${data_ID},next_page_token:${next_page_token},sort_by:qualityScore,start_index:,associated_topic:,_fmt:computer`

The place,
data_ID – Information ID is used to uniquely determine a spot in Google Maps.
next_page_token – The next_page_token is used to get the following web page outcomes.
sort_by – It’s used for sorting and filtering outcomes.

The assorted values of sort_by are:

1. qualityScore – probably the most related critiques.
2. newestFirst – the newest critiques.
3. ratingHigh – the very best score critiques.
4. ratingLow – the bottom score critiques.

Now, the query arises how will we get the Information ID of anyplace?

https://www.google.com/maps/place/Eiffel+Tower/@48.8583701,2.2922926,17z/knowledge=!4m7!3m6!1s0x47e66e2964e34e2d:0x8ddca9ee380ef7e0!8m2!3d48.8583701!4d2.2944813!9m1!1b1

You possibly can see, within the URL the half after our !4m7!3m6!1s and earlier than !8m2! is our Information ID.
So, our knowledge ID on this case is 0x47e66e2964e34e2d:0x8ddca9ee380ef7e0
Our goal URL ought to appear like this:

https://www.google.com/async/reviewDialog?hl=en_us&async=feature_id:0x47e66e2964e34e2d:0x8ddca9ee380ef7e0,next_page_token:,sort_by:qualityScore,start_index:,associated_topic:,_fmt:computer

Copy this URL in your browser and press enter. You will note, {that a} textual content file will likely be downloaded, by coming into this URL in your browser. Open this file in your respective code editor. Convert it right into a .html file. After opening the HTML file, we are going to seek for the HTML tags of the weather we wish in our response.

We’ll first parse the placement info of the place, which accommodates – location identify, handle, common score, and complete critiques.

From the above picture, the tag for our location identify is .P5Bobd, the tag for our handle .T6pBCe, tag for our common score is span.Aq14fc and tag for our complete variety of critiques is span.z5jxId.

All performed for the placement info half, we are going to now transfer in the direction of parsing Information ID and next_page_token.

Seek for the tag .lcorif. Within the above picture you’ll find the .lcorif tag within the second line. Beneath this tag, we’ve our tag for Information ID as .loris and of next_page_token as .gws-localreviews__general-reviews-block.

Now, we are going to seek for the tags which comprise knowledge concerning the consumer and his assessment.
Seek for the tag .gws-localreviews__google-review.

This tag accommodates all details about the consumer and his critiques.
We’ll parse the extracted HTML for the consumer’s identify, hyperlink, thumbnail, variety of critiques, score, assessment, and the pictures posted by the consumer.
This makes our complete code appear like this:

You possibly can copy this code from the next hyperlink: https://github.com/Darshan972/GoogleScrapingBlogs/blob/primary/GoogleMapsReviewsScraper.js

End result:

Our end result ought to appear like this 👆🏻.
These are the outcomes of the primary ten critiques. If you wish to get one other 10 put the token, which we’ve present in our code within the beneath URL:

https://www.google.com/async/reviewDialog?hl=en_us&async=feature_id:0x47e66e2964e34e2d:0x8ddca9ee380ef7e0,next_page_token:tokenFromResponse,sort_by:qualityScore,start_index:,associated_topic:,_fmt:computer

On this case, we’ve our token as CAESBkVnSUlDZw== .
You’ll find the critiques for each subsequent web page utilizing the token from their earlier pages.

With Google Maps Critiques API:

Serpdog | Google Search API presents you 100 free requests on sign-up.
Scraping can take lots of time generally, however the already made structured JSON knowledge can prevent lots of time.

const axios = require('axios');

axios.get('https://api.serpdog.com/critiques?api_key=APIKEY&data_id=0x89c25090129c363d:0x40c6a5770d25022b')
  .then(response => {
    console.log(response.knowledge);
  })
  .catch(error => {
    console.log(error);
  });

End result:

Conclusion:

On this tutorial, we realized the best way to scrape Google Maps Critiques. Be at liberty to ask me something on my e mail. Observe me on Twitter Thanks for studying!

Further Assets:

1. Methods to scrape Google Natural Search Outcomes utilizing Node JS?
2. Scrape Google Information Outcomes?
3. Scrape Google Pictures Outcomes
4. Scrape Google Autocomplete Recommendations Outcomes

Creator:

My identify is Darshan and I’m the founding father of serpdog.io.