{"id":869,"date":"2021-09-10T06:18:35","date_gmt":"2021-09-10T11:18:35","guid":{"rendered":"https:\/\/www.becomebetterprogrammer.com\/?p=869"},"modified":"2022-04-25T09:41:16","modified_gmt":"2022-04-25T14:41:16","slug":"what-does-pipe-mean-in-node-js-how-to-use-it-practical-guide","status":"publish","type":"post","link":"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/what-does-pipe-mean-in-node-js-how-to-use-it-practical-guide\/","title":{"rendered":"Practical Guide: Learn How to Use .pipe() in Node.js"},"content":{"rendered":"\n<p>Have you been using Node.js for a while and only until now have you heard about the word pipe? You tried checking the documentation but still can&#8217;t figure out what it means or does? In this article, I will clarify those doubts by explaining what .pipe or piping is and how to use it in Node.js. To ensure you understand the article, previous knowledge about streams is strongly recommended.<\/p>\n\n\n\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_82_2 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\"><p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<\/div><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/what-does-pipe-mean-in-node-js-how-to-use-it-practical-guide\/#What_Does_pipe_Method_Do\" >What Does .pipe() Method Do?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/what-does-pipe-mean-in-node-js-how-to-use-it-practical-guide\/#How_to_Use_The_pipe_Method\" >How to Use The .pipe() Method?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/what-does-pipe-mean-in-node-js-how-to-use-it-practical-guide\/#Only_Works_With_Readable_Streams\" >Only Works With Readable Streams<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/what-does-pipe-mean-in-node-js-how-to-use-it-practical-guide\/#Pushing_Data_No_Matter_Readable_Streams_Flowing_Mode\" >Pushing Data No Matter Readable Stream&#8217;s Flowing Mode<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/what-does-pipe-mean-in-node-js-how-to-use-it-practical-guide\/#Dont_Confuse_The_pipe_Method_With_The_Event_pipe\" >Don&#8217;t Confuse The pipe Method With The Event pipe<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/what-does-pipe-mean-in-node-js-how-to-use-it-practical-guide\/#Why_You_Should_Use_pipe_Method\" >Why You Should Use .pipe Method?<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"What_Does_pipe_Method_Do\"><\/span>What Does .pipe() Method Do?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p><strong>The method .pipe was added in v0.9.4 of Node.js and its purpose is to attach a writeable stream to a readable stream allowing to pass the readable stream data to the writeable stream.<\/strong> One good way to understand this concept is by thinking about PVC pipes and connecting two pipes. <\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-full is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/www.becomebetterprogrammer.com\/wp-content\/uploads\/2021\/09\/example-piping-a-readable-to-writeable-stream.png\" alt=\"\" class=\"wp-image-871\" width=\"802\" height=\"451\" title=\"Analogy of of the pipe method when using readable and writeable streams\" srcset=\"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/wp-content\/uploads\/2021\/09\/example-piping-a-readable-to-writeable-stream.png 800w, https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/wp-content\/uploads\/2021\/09\/example-piping-a-readable-to-writeable-stream-300x169.png 300w, https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/wp-content\/uploads\/2021\/09\/example-piping-a-readable-to-writeable-stream-768x432.png 768w\" sizes=\"auto, (max-width: 802px) 100vw, 802px\" \/><figcaption>Analogy of of the pipe method when using readable and writeable streams<\/figcaption><\/figure><\/div>\n\n\n\n<p>For the sake of explanation, let&#8217;s assume the first PVC pipe is a readable stream and the second pipe is a writeable stream. The method .pipe will be the orange pipe fitting which will connect both pipes allow for the water, or the data, to flow from one pipe to another.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"How_to_Use_The_pipe_Method\"><\/span>How to Use The .pipe() Method?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>In this pipe implementation, we are going to create a simple HTTP server that will read data from a file and send the response to the client.<\/p>\n\n\n\n<p><strong>1.<\/strong> Let&#8217;s start by creating the HTTP server using the <code>http<\/code> package that returns some data.<\/p>\n\n\n\n<pre><code class=\"language-javascript\">\nconst http = require('http');\n\nhttp.createServer(function(req, res) {\n  res.write('hello!');\n  res.end();\n}).listen(8080);\n\n<\/code>\n<\/pre>\n\n\n\n<p>Let&#8217;s make sure it works, by making a request to our server using curl.<\/p>\n\n\n\n<pre><code class=\"language-bash\">\ncurl localhost:8080\n<\/code>\n<\/pre>\n\n\n\n<p>Or another option is to open a new tab <a href=\"http:\/\/localhost:8080\/\" target=\"_blank\" rel=\"noreferrer noopener\">http:\/\/localhost:8080\/<\/a>. Once you make the request, you should receive &#8220;hello!&#8221;.<\/p>\n\n\n\n<p>We are going to pause for a second. Let&#8217;s recall the anatomy of an HTTP transaction. An HTTP transaction is made of a server, created by the method <code>createServer<\/code> which in itself is an EventEmitter. When an HTTP request hits the server, node calls the request handler using the req and res objects, which are request and response respectively, for dealing with the transaction.<\/p>\n\n\n\n<p>The req or request object is an instance of the <a href=\"https:\/\/nodejs.org\/api\/http.html#http_class_http_incomingmessage\" target=\"_blank\" rel=\"noreferrer noopener\">IncomingMessage<\/a> object. The IncomingMessage object is a child object of a <a href=\"https:\/\/nodejs.org\/api\/stream.html#stream_class_stream_readable\" target=\"_blank\" rel=\"noopener\">ReadableStream<\/a>.<\/p>\n\n\n\n<p>The res or response object is an instance of the <a href=\"https:\/\/nodejs.org\/api\/http.html#http_class_http_serverresponse\" target=\"_blank\" rel=\"noopener\">ServerResponse<\/a> object. The ServerResponse object is a child object of a <a href=\"https:\/\/nodejs.org\/api\/stream.html#stream_class_stream_writable\" target=\"_blank\" rel=\"noreferrer noopener\">WriteableStream<\/a>.<\/p>\n\n\n\n<p>Therefore, we know we have a writeable and a readable stream. <\/p>\n\n\n\n<p><strong>2<\/strong>. We are going to create a data.txt file in the same directory folder, and save some information. For the sake of making things clear, I will save the following text: &#8220;This is data from the data.txt file&#8221;.<\/p>\n\n\n\n<p><strong>3.<\/strong> Remove the existing logic from the event handler.<\/p>\n\n\n\n<p><strong>4<\/strong>. We are going to read the content of the data.txt file using the <code>fs<\/code> package using <a href=\"https:\/\/nodejs.org\/api\/fs.html#fs_fs_createreadstream_path_options\" target=\"_blank\" rel=\"noreferrer noopener\">fs.createReadStream<\/a>. The fs.createReadStream will return a ReadableStream. We are going to use that ReadableStream to pipe or pass the data from data.txt file to the response object, which is a WriteableStream.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code class=\"language-javascript\">const http = require('http');\nconst fs = require('fs');\n\nhttp.createServer(function(req, res) {\n  \/\/ generete readable stream to read content of data.txt\n  const readStream = fs.createReadStream(__dirname + '\/data.txt');\n  \n  \/\/ pass readable stream data, which are the content of data.txt, to the \n  \/\/ response object, which is a writeable stream\n  readStream.pipe(res);\n}).listen(8080);<\/code><\/pre>\n\n\n\n<p>Once updated the event handler&#8217;s logic, make a request to <a href=\"http:\/\/localhost:8080\/\" target=\"_blank\" rel=\"noreferrer noopener\">http:\/\/localhost:8080\/<\/a> and you should see data.txt data.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Only_Works_With_Readable_Streams\"><\/span>Only Works With Readable Streams<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Remember, the pipe method can only be used in readable streams. Don&#8217;t let yourself fool by your IDE in case it suggests the pipe method in a writeable stream.<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"556\" height=\"231\" src=\"https:\/\/www.becomebetterprogrammer.com\/wp-content\/uploads\/2021\/09\/Pipe-does-not-work-with-writeable-streams-1.png\" alt=\"\" class=\"wp-image-875\" srcset=\"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/wp-content\/uploads\/2021\/09\/Pipe-does-not-work-with-writeable-streams-1.png 556w, https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/wp-content\/uploads\/2021\/09\/Pipe-does-not-work-with-writeable-streams-1-300x125.png 300w\" sizes=\"auto, (max-width: 556px) 100vw, 556px\" \/><\/figure><\/div>\n\n\n\n<p>In case you attempt using the .pipe method using a writeable stream, like in the example below:<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"566\" height=\"477\" src=\"https:\/\/www.becomebetterprogrammer.com\/wp-content\/uploads\/2021\/09\/Pipe-does-not-work-with-writeable-streams-another-example.png\" alt=\"\" class=\"wp-image-876\" srcset=\"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/wp-content\/uploads\/2021\/09\/Pipe-does-not-work-with-writeable-streams-another-example.png 566w, https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/wp-content\/uploads\/2021\/09\/Pipe-does-not-work-with-writeable-streams-another-example-300x253.png 300w\" sizes=\"auto, (max-width: 566px) 100vw, 566px\" \/><\/figure><\/div>\n\n\n\n<p>At the moment of executing this code, it will throw the following error.<\/p>\n\n\n\n<p><code>Error [ERR_STREAM_CANNOT_PIPE]: Cannot pipe, not readable<\/code><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Pushing_Data_No_Matter_Readable_Streams_Flowing_Mode\"><\/span>Pushing Data No Matter Readable Stream&#8217;s Flowing Mode<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>If you are familiar with readable streams, you will know there are two modes in which data flows, flowing and paused mode. You can use the pause() or resume() method to update the flowing mode.<\/p>\n\n\n\n<pre><code class=\"language-javascript\">\nconst http = require('http');\nconst fs = require('fs');\n\nhttp.createServer(function(req, res) {\n  const readStream = fs.createReadStream(__dirname + '\/data.txt');\n  \n  readStream.on('data', function(chunk) {\n    console.log('this is the data from file', chunk);\n  });\n\n  readStream.pause();\n  console.log('on pause: readable flowing', readStream.readableFlowing);\n\n  readStream.resume();\n  console.log('on resume: readable flowing', readStream.readableFlowing);\n\n  res.write('Hello!')\n  res.end();\n}).listen(8080);\n\n<\/code>\n<\/pre>\n\n\n\n<p>If you run the example above, you will only read data from the data.txt file whenever the readable stream flowing mode is set to true which is enabled by using the resume() method. If the flowing mode is set to false, it will never read the content of the data.txt file.<\/p>\n\n\n\n<p>However, when using the pipe method the flowing mode will automatically be set to true ensuring the data is passed from one stream to another. We can confirm this if we try to pause the flowing mode prior to piping both streams.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code class=\"language-javascript\">const http = require('http');\nconst fs = require('fs');\n\nhttp.createServer(function(req, res) {\n  \/\/ generete readable stream to read content of data.txt\n  const readStream = fs.createReadStream(__dirname + '\/data.txt');\n  \n  readStream.on('data', function(chunk) {\n    console.log('this is the data from file', chunk);\n  });\n\n  readStream.pause();\n  console.log('on pause: readable flowing', readStream.readableFlowing);\n\n  readStream.pipe(res);\n\n}).listen(8080);<\/code><\/pre>\n\n\n\n<p>After making a request to the server, we will still receive the content from the data.txt file.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Dont_Confuse_The_pipe_Method_With_The_Event_pipe\"><\/span>Don&#8217;t Confuse The pipe Method With The Event pipe<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>If you have never heard of the word &#8220;pipe&#8221; or &#8220;piping&#8221; when working with streams, there is a slight chance you could find the wrong information if you go to Node.js documentation and start finding for the word &#8220;pipe&#8221;. When you do a quick search, you will find two options.<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li><a href=\"https:\/\/nodejs.org\/api\/stream.html#stream_event_pipe\" target=\"_blank\" rel=\"noopener\">Event &#8220;pipe&#8221;<\/a><\/li><li><a href=\"https:\/\/nodejs.org\/api\/stream.html#stream_readable_pipe_destination_options\" target=\"_blank\" rel=\"noreferrer noopener\">Readable.pipe<\/a><\/li><\/ol>\n\n\n\n<p>If you find the first option, you will notice it is an event listener that writeable streams can set when a readable stream uses the pipe method to pass the data from one stream to another. The event pipe is only available on writeable streams. We are going to use our simple server API to demonstrate the event pipes.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code class=\"language-javascript\">const http = require('http');\nconst fs = require('fs');\n\nhttp.createServer(function(req, res) {\n  const readStream = fs.createReadStream(__dirname + '\/data.txt');\n\n  \/\/ setting pipe event listener before triggering the pipe method in the readable stream\n  \/\/ otherwise, the pipe event listener won't be triggered if set after triggering the pipe method\n  res.on('pipe', function(src) {\n    console.log('Triggered the pipe event listener whenever a source readable stream pipes the writeable stream');\n  });\n\n  readStream.pipe(res);\n\n}).listen(8080);<\/code><\/pre>\n\n\n\n<p>In other words, calling the pipe method on the readable stream causes the pipe event listener to be triggered on the writeable stream.<\/p>\n\n\n\n<p>It is important to mention to define the pipe event listener prior to calling the pipe method from the readable stream. Attempting to call the pipe method prior to setting the event listener in the writeable stream won&#8217;t work.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Why_You_Should_Use_pipe_Method\"><\/span>Why You Should Use .pipe Method?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p>Streams are one of the most powerful and fundamental concepts of Node.js applications. They allow us to handle data in a more efficient way as pieces of data can be transported in smaller chunks preventing you from running out of memory and maintaining good performance in your applications. <\/p>\n\n\n\n<p>Therefore, using the pipe method is an effective and easy solution to push data between streams. In that way, we avoid storing too much data that needs to be manipulated or modified all at the same time. Also, the code will be shorter, elegant, and easy to follow.<\/p>\n\n\n\n<p><strong>Was this article helpful?<\/strong><\/p>\n\n\n\n<p>Share your thoughts by replying on <a href=\"https:\/\/twitter.com\/bbprogrammer\" target=\"_blank\" rel=\"noreferrer noopener\">Twitter of Become A Better Programmer<\/a> or to<a href=\"https:\/\/twitter.com\/arealesramirez\" target=\"_blank\" rel=\"noopener\"> personal my Twitter account<\/a>.<\/p>\n\n\n\n<figure class=\"wp-block-embed aligncenter is-type-rich is-provider-twitter wp-block-embed-twitter\"><div class=\"wp-block-embed__wrapper\">\n<blockquote class=\"twitter-tweet\" data-width=\"550\" data-dnt=\"true\"><p lang=\"en\" dir=\"ltr\">How do you use the pipe method in <a href=\"https:\/\/twitter.com\/hashtag\/NodeJS?src=hash&amp;ref_src=twsrc%5Etfw\" target=\"_blank\" rel=\"noopener\">#NodeJS<\/a>? <a href=\"https:\/\/t.co\/LFh4btK0Tw\">https:\/\/t.co\/LFh4btK0Tw<\/a><a href=\"https:\/\/twitter.com\/hashtag\/softwareengineertips?src=hash&amp;ref_src=twsrc%5Etfw\" target=\"_blank\" rel=\"noopener\">#softwareengineertips<\/a>  <a href=\"https:\/\/twitter.com\/hashtag\/programmer?src=hash&amp;ref_src=twsrc%5Etfw\" target=\"_blank\" rel=\"noopener\">#programmer<\/a> <a href=\"https:\/\/twitter.com\/hashtag\/javascript?src=hash&amp;ref_src=twsrc%5Etfw\" target=\"_blank\" rel=\"noopener\">#javascript<\/a>  <a href=\"https:\/\/twitter.com\/hashtag\/becomeabetterprogrammer?src=hash&amp;ref_src=twsrc%5Etfw\" target=\"_blank\" rel=\"noopener\">#becomeabetterprogrammer<\/a><\/p>&mdash; Become A Better Programmer (@bbprogrammer) <a href=\"https:\/\/twitter.com\/bbprogrammer\/status\/1436295758937038848?ref_src=twsrc%5Etfw\" target=\"_blank\" rel=\"noopener\">September 10, 2021<\/a><\/blockquote><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script>\n<\/div><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>Learning how to use the pipe method in   Node.js is an advance topic, but it doesn&#8217;t have to be. This guide provides an easy-to-follow explanation.<\/p>\n","protected":false},"author":1,"featured_media":883,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"nf_dc_page":"","_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[23],"tags":[],"class_list":["post-869","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-node-js","generate-columns","tablet-grid-50","mobile-grid-100","grid-parent","grid-50"],"_links":{"self":[{"href":"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/wp-json\/wp\/v2\/posts\/869","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/wp-json\/wp\/v2\/comments?post=869"}],"version-history":[{"count":5,"href":"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/wp-json\/wp\/v2\/posts\/869\/revisions"}],"predecessor-version":[{"id":2497,"href":"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/wp-json\/wp\/v2\/posts\/869\/revisions\/2497"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/wp-json\/wp\/v2\/media\/883"}],"wp:attachment":[{"href":"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/wp-json\/wp\/v2\/media?parent=869"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/wp-json\/wp\/v2\/categories?post=869"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.becomebetterprogrammer.com\/staging\/4563\/wp-json\/wp\/v2\/tags?post=869"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}