HTTP GET Requests with Node

This series of posts catalogs basic functions in Node via the built-in APIs. There are many libraries that provide syntactic sugar over these functions. However, it's a good exercise to understand how to do them manual. As serverless architectures increase in popularity, for performance reasons it's important to reduce dependencies and size of the deployment packages.

I'm starting this series with a post on making HTTP GET requests since it was one of the recent changes I made to a serverless application to improve performance. This functionality is found in https://nodejs.org/api/http.html

GET Request

The simplest request to make is a GET request. This example converts the request into a Promise based function using the standard Promise class.

function get({ host, path }) {  
  return new Promise((resolve, reject) => {

    // construct the http options
    let opts = {
      host,
      path,
    };

    // create a callback function to handle the response
    function callback(res) {
      let buffers = [];
      res.on('data', (buffer) => buffers.push(buffer));
      res.on('error', reject);
      res.on('end', () => res.statusCode === 200 
          ? resolve(Buffer.concat(buffers))
          : reject(Buffer.concat(buffers)));
    }

    // initiate the http request
    let req = http.request(opts, callback);
    req.on('error', reject);
    req.end();

  });
}

Breaking this down... The first section creates the request options that are defined in the http request API. In this instance, I'm only defining the host and path properties.

// construct the http options
let opts = {  
  host,
  path,
};

The next section defines the callback. The callback from a request has a single argument, the response. The callback function will perform three functions using this response.

  • listen to the data event and collect all buffers that are reported
  • listen to the error event and immediately reject if there is an error
  • listen to the end event and depending on the status code will either resolve or reject the response by combining all of the buffers that were compiled during the data events.
// create a callback function to handle the response
function callback(res) {  
  let buffers = [];
  res.on('data', (buffer) => buffers.push(buffer));
  res.on('error', reject);
  res.on('end', () => res.statusCode === 200 
      ? resolve(Buffer.concat(buffers))
      : reject(Buffer.concat(buffers)));
}

This pattern is suitable for small requests (such as web page fetches) due to how it keeps all buffer parts in memory. This pattern would not be suitable for large request where you would want to return the result object directly and stream it to disk or some other output location.

The final piece is actually initiating the request. The request function takes two arguments options and callback and returns an instance of ClientRequest.

With this instance, it will listen for an error event and reject the promise if encountered. Otherwise it will finalize the request by calling end.

// initiate the http request
let req = http.request(opts, callback);  
req.on('error', reject);  
req.end();  

Finally, to use this new helper function, you can simply call it as you would any other promise:

get({ host: 'www.southsidecomicspgh.com', path: '/' })  
  .then(buffer  => console.log(buffer.toString()))
  .catch(buffer => console.error(buffer.toString()));

The next article will discuss making POST requests with the HTTP APIs.

comments powered by Disqus