Log in
Registration
- Welcome, visitor !
- |
- Log In | Register
- 05/20 GParted 0.12.1-5
- 05/19 LuninuX 12.00-beta2
- Chris Lawrence: cnlmisc 0.2 for R
- 05/19 Dream Studio 12.04
- Matthew Garrett: Fixing Asus UX21e unexpected power off
- Thorsten Glaser: MirBSD goes webservice – manpages and WTF‽
- Jonathan McDowell: 6 months of GNOME Shell
- Ubuntu developers: Leo Iannacone: Forum ubuntu-it gets a new theme and a new engine
- Richard Hartmann: Mongolia
- Maemo developers: Do Not Track support in Epiphany
- Pietro Abate: how to create VMs with ganeti / xen and dnsmaq
- Disk encryption in ROSA Marathon 2012
- Ubuntu developers: Francisco Molinero: Zentyal on Galapinux
- 05/19 Extix 10
- 05/19 Clonezilla 1.2.12-58
Introduction to Named Pipes and FIFOs
One of the fundamental features that are used in Linux are pipes. By utilizing pipes you are able to process huge amounts of data without the requirement to waste hard disk space. Generally speaking pipes are concatenating the output data stream of process A with the standard input channel of process B.
Contents
Introduction to Named Pipes
What are Pipes?
A Pipe is a FIFO. FIFO is an acronym for First In, First Out. A pipe (pipeline) consists of a chain of processing elements (e.g. processes, threads), arranged so that the output of an element is the input of the next one.
When you are using pipelines within the context of Unix shells (e.g. bash, sh, ksh or others), the handling of pipe objects is managed automatically. But take care: the syntax how to handle pipes and data streams may vary between different shells!
For further details and exmples how to use pipes take a Look at the article Introduction to Pipes and FIFOs.
What are Named Pipes?
When using pipes in the context of Unix shells, the creation and concatenation of processes and even the clean-up of unused pipes is handled by the surrounding shell.
Named pipes have to be managed manually. They are created with OS tools and they are represented by file system objects.
To create a named pipe use one of the following commands: mkfifo or mknod. To remove a named pipe delete if with with any tool you would use to delete files, e.g. rm.
1 2 3 4 5 6 7 8 9 10 11 12 | # create a named pipe $ mkfifo myfifo # ...or use this command $ mknod myfifo p # list the named pipe in file system $ ls -l myfifo prw-r--r-- 1 mario users 0 13. Nov 22:37 myfifo # remove a named pipe $ rm myfifo |
Please note the ‘p’ at the beginning of line 9. This indicates that the file myfifo is a pipe.
Why to Use Named Pipes?
Why to use named pipes when Unix Shells are managing the handling of pipes?
Some tools are using files only to read and write data. In this case you cannot use data stream redirection to connect different processes. Named pipes behave like ordinary files. So it is possible to configure third party tools to use named pipes to write and read data.
Named pipes are connecting different processes via files, that are available in file systems. Authorization is granted or denied on basis of access permissions within file systems.
How to Use Named Pipes?
Named pipes are used in the same way you would use pipes. You just have to care about creation of pipes and you have to redirect data streams to connect different processes.
Simple Example
The following example describes how to utilize a named pipe to print the contents of a directory.
1 2 3 4 5 6 7 8 9 10 11 12 | # create a named pipe $ mkfifo myfifo $ mkfifo myfifo2 # start a process and redirect standard output stream to the pipe $ ls -l > myfifo & [1] 9823 # start a process that reads contents from the pipe $ cat myfifo insgesamt 0 prw-r--r-- 1 mario users 0 13. Nov 22:37 myfifo prw-r--r-- 1 mario users 0 13. Nov 22:37 myfifo2 |
Advanced Example
The last example was a quite simple one. If you are asking yourself, what could be a use case to utilize named pipes to link processes, then take a look at the following example.
The following example creates a dump of an Oracle database schema and imports the dump into another remote Oracle database. The size of the database dump does not matter, because the data stream is not stored to hard disks. The data stream will be redirected to the Oracle import tool.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | # create a pipe $ mkfifo pipe # create the par file for exporting data $ cat > exp_tagesfp_archive.dat FILE=pipe LOG=exp_tagesfp_archiv.log TABLES=(tagesfp_archiv.bahnhofname_archiv ,tagesfp_archiv.betreiber_archiv) INDEXES=N TRIGGERS=N CONSTRAINTS=N GRANTS=N STATISTICS=NONE FEEDBACK=10000 ^D # create the par file for importing data $ cat > imp_tagesfp_archive.dat FILE=pipe LOG=imp_tagesfp_archiv.log FROMUSER=tagesfp_archiv TOUSER=tagesfp_archiv GRANTS=N INDEXES=N CONSTRAINTS=N IGNORE=Y FEEDBACK=1000 ^D # copy tnsnames.ora to the local directory $ copy /path/to/tnsnames.ora . # create the export script $ cat > doexport.sh #!/bin/sh set -x ORACLE_HOME=/path/to/oracle/1020/product/10.2.0/ ORACLE_BIN=$ORACLE_HOME/bin #TNS_ADMIN=$ORACLE_HOME/network/admin TNS_ADMIN=. export TNS_ADMIN export ORACLE_HOME dba=dbaaccount dbapasswd=dbapasswd inst_src=mysid.tmp.oracle.db.de $ORACLE_BIN/exp $dba/$dbapasswd@$inst_src parfile=exp_tagesfp_archiv.dat ^D # create the import script $ cat > doimport.sh #!/bin/sh set -x ORACLE_HOME=/path/to/oracle/1020/product/10.2.0/ ORACLE_BIN=$ORACLE_HOME/bin #TNS_ADMIN=$ORACLE_HOME/network/admin TNS_ADMIN=. export TNS_ADMIN export ORACLE_HOME dba=dbaaccount dbapasswd=dbapasswd inst_tgt=mysid.dev.oracle.db.de $ORACLE_BIN/imp $dba/$dbapasswd@$inst_tgt parfile=imp_tagesfp_archiv.dat ^D # start the import $ sh doimport.sh & # start the export $ sh doexport.sh |
When the data transfer has been finished, the import/export tools will report that the job has been completed (hopefully without any errors and warnings).
Related Articles:
- Introduction to Pipes and FIFOs
- http://en.wikipedia.org/wiki/FIFO
- http://en.wikipedia.org/wiki/Unix_Pipe