One of the most powerful shell operators in Linux is the pipe. In this article we will see how regular and named pipes work, how to use them and how they differ from each other.
The vertical bar symbol
| denotes a pipe. Because of the pipe, you can take the output from one command and feed it to another command as input. In other words, a pipe is a form of redirection that is used in Linux to send the output of one program to another program for further processing.
Of course, you’re not limited to a single piped command. You can stack them as many times as you like.
The syntax for the pipe or unnamed pipe command is the
| character between any two commands:
command1 | command2 | ... | commandN
How does a pipe work in Linux
To see how pipe work, let’s take a look at examples bellow.
Let’s say we have a directory full of many different types of files. We want to know how many files of a certain type are in this directory. We can get a list of files easily using
conf license.txt wp-activate.php wp-comments-post.php wp-content wp-links-opml.php wp-mail.php wp-trackback.php favicon.ico readme.html wp-admin wp-config-sample.php wp-cron.php wp-load.php wp-settings.php xmlrpc.php index.php robots.txt wp-blog-header.php wp-config.php wp-includes wp-login.php wp-signup.php
To separate the type of file we are interested in, we will use
grep. We want to find files that have the word “txt” in their name or file extension.
We will use the special shell character
| to direct the output of
ls | grep txt
The output of
ls was not sent to the terminal window. Therefore the result is not displayed on the screen, and is instead redirected to the input of the
grep command. The output we see above comes from
grep, which is the last command in this chain.
Now, let’s start extending our chain. We can count files “txt” by adding the
wc command . We will use the
-l option (number of lines) with
ls | grep txt | wc -l
grep is no longer the last command in the chain, so we do not see its output. The output of
grep is fed into the
wc command. The output that we see in the terminal window comes from
wc reports that there are 2 files “txt” in the directory.
What is a named pipe in Linux
As the name itself suggests, these are pipes with names. One of the key differences between regular pipes and named pipes is that named pipes have a presence in the file system. That is, they show up as files.
The named pipe is a method for passing information from one computer process to another using a pipe which is given a specific name. Named pipes are also known as FIFO, which stands for First In, First Out.
You can create a named pipe using the
mkfifo command. For example:
You can tell if a file is a named pipe by the
p bit in the file permissions section.
ls -l mypipe
prw-r--r-- 1 root root 0 Mar 20 12:58 mypipe
The named pipes are actually files on the file system itself. Unlike a standard pipe, a named pipe is accessed as part of the filesystem, just like any other type of file.
The named pipe content resides in memory rather than being written to disk. It is passed only when both ends of the pipe have been opened. And you can write to a pipe multiple times before it is opened at the other end and read. By using named pipes, you can establish a process in which one process writes to a pipe and another reads from a pipe without much concern about trying to time or carefully orchestrate their interaction.
To see how named pipe work, let’s take a look at examples bellow. Let’s first create our named pipe:
Now let’s consume messages with this pipe.
tail -f mypipe
Open another terminal window, write a message to this pipe:
echo "hi" >> mypipe
Now in the first window you can see the “hi” printed out:
tail -f pipe1 hi
Because it is a pipe and message has been consumed, if we check the file size, you can see it is still 0:
ls -l mypipe
prw-r--r-- 1 root root 0 Mar 20 14:11 mypipe
Since a named pipe is just a file, to remove one we can use the
rm command. Therefore, to remove the pipe we created in the previous examples, we would run:
When to use regular or named pipes
Using a regular pipe instead of a named pipe depends on the characteristics we’re looking for. Some of them can be persistence, two-way communication, having a filename, creating a filter, and restricting access permissions among others.
For example, if we want to filter the output of a command multiple times, using an anonymous pipe seems the most appropriate option. On the other hand, if we need a filename and we don’t want to store data on disk, what we’re looking for is a named pipe.
In conclusion, the next time you’re working with commands at the Linux terminal and find yourself moving data between commands, hopefully a pipe will make the process quick and easy.