JavaScript: Retrieve YouTube Search Data for Data Science

eyecatch_youtube.jpg

In this article, we will learn how to extract the search data out of the YouTube Search Results using JavaScript. The question arises why we need such a data. In Data Science, we analyse the data and extract the information out of it. Other fields like Machine Learning or Deep Learning also rely on some kind of data. For example, what if we want to remove the outliers from the YouTube search result; what if we want to train a deep model to learn the patterns in the search results using that YouTube data. In a nutshell, data in this era plays a vital role in an array of fields. So, without further ado, let us get started!

Mechanics

We will divide this tutorial-based article into two sections, which are as follows:

  • Enabling YouTube API (for data collection)
  • Getting Search Results (the data itself)

Tools

  • Google Chrome or Safari (Any browser you like; I am going to use Safari in the article.)
  • Code Editor (You can use simple Text Editor; however, I am using Visual Studio Code.)
  • XAMPP (Required; Google’s YouTube Data API does not run locally)

Languages

The following languages will be used to retrieve the YouTube Search Results:

  • HTML5 (for scaffolding the webpage)
  • JavaScript (ES6+) (for data extraction)

Knowledge to have

You are supposed to have some know-how of HTML and JavaScript programming for this article. If you do not know anything about these languages, then I am afraid, this article will be hard to follow. Although I would try to explain most of the things.

Enabling YouTube API (for JavaScript)

First, we need to enable the YouTube API to get the search results. In order to do so, follow the following steps:

Step 1

Go to the following link and sign-in:

https://console.developers.google.com/

Step 2

Once you log into the console, we need to make a new project. Click on the New Project button from the projects panel and fill out the following fields (as shown below): 1.png

Once you fill out the above field, click on the Create button.

  • After creating the project, go to the Dashboard and click on ENABLE APIS AND SERVICES button. Now select the “YouTube Data API v3” from the APIs and services section and Enable it.

  • Now go to the Credentials tab and click on the Create Credentials button; there you would be given the following options:

    i. API Key ii. OAuth Client ID iii. Service Account Key

  • Choose the API Key option, and you now have the API key.

  • Copy the API key and paste it in a text document, as we would need that API key later.

If you follow the above two steps, you would be able to enable YouTube Data API for your application.

Getting Search Results

In order to get the search result, we will first create the HTML file for running our JavaScript code. It could be made without the HTML file by using node.js, but in this article, the simplest approach will be used.

The code of the HTML document looks like this:

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <title></title>
    <link rel="stylesheet" href="">
</head>
<body>
    

    <script src="main.js"></script>
    <script src="https://apis.google.com/js/client.js?onload=init"></script>
</body>
</html>

Following are the salient features in the above code:

  • src="main.js": This tag is used to link our own JavaScript file with this HTML document.
  • src="https://apis.google.com/js/client.js?onload=init": This is the required script that would let us use the Google YouTube Data API in the JavaScript file that we linked above.

Once we create the HTML file, next step is to use the following code to retrieve the search results from YouTube Data API:

function init() {

  gapi.client.setApiKey("AIzaSyC8PNB56WbaaY715lTHnOiOmJtYbsXDgUE");
  gapi.client.load("youtube", "v3", () => {
  let request = requestData();
  request.execute(receivedResponse);
  });
}

function receivedResponse(response) {
  response.items.map((item) => {console.log(item.snippet.title);});
  if(response.nextPageToken !== "") {
    let r = requestData(response.nextPageToken);
    r.execute(receivedResponse);
  }
}

function requestData(pageTokenNext = "") {
    return gapi.client.youtube.search.list({
    q: 'Education',
    type: 'video',
    maxResults: 50,
    pageToken: pageTokenNext,
    part: 'snippet'
  });
}

Following are the important features of the above code:

  • The API Key that we saved in a text document before is used in the function gapi.client.setApiKey. This function call will let us use the Google’s YouTube Data API. Pass the API Key to this function.
  • Now we need to load the Youtube v3 client. In order to do so, we would use the following function call provided by Google’s YouTube API: gapi.client.load. This function takes three parameters. The first parameter is used to specify the Google service, which in this case is youtube; the second parameter is used to specify the version of the API (we are using the v3 version). The third parameter is a callback, which is called when the API is loaded.
  • Once the API is loaded, we would request the search data by using the following function call: gapi.client.youtube.search.list. In this function call we have to specify the various parts that we need.
  • q represents the query we want to search. In this case, Education is hardcoded in the query; you can pass the custom string. type specifies that it is a video. maxResults is the number of results we get in return. By default, the maximum value is 50. In order to get more than 50 results, we would need the pageToken. That is why the nextPageToken is passed to the function in order to get more than 50 results.
  • Those results are actually in JSON form. In that JSON, there is an item called snippet. In that snippet, the search results of videos are present.
  • Next, we need to execute this query. For that, the following function call is used: request.execute. The request is the object that we returned by using the search query (mentioned in above features).
  • Then we loop through each of the item and get the titles of the videos. The nextPageToken is passed over and over again to get all the results available. This, however, may remain into the loop unless you get an empty pageToken. You can add exceptions to the code.

If you run the script above, you will get the search results in the console.

Summary

In this article, we have discussed YouTube’s Data API to get the search result for Data Science. We have also added the feature of getting more than fifty (default) results by using the recursive call. That’s it for this article! I hope you like it!

AUTHOR

READ NEXT

Boostlog is an online community for developers
who want to share ideas and grow each other.

Delete an article

Deleted articles are gone forever. Are you sure?