Linux pipes, redirection and command chaining let you combine terminal tools without copying output between programs by hand. A pipe passes data to another command. Redirection changes where a command reads or writes. Command chaining controls whether the next command runs, usually based on whether an earlier step succeeded.
Those jobs overlap, but they are not interchangeable. Sending a report into a filter is different from saving it to a file, and neither operation guarantees that the report was generated successfully.
This guide uses Bash behavior unless stated otherwise. If you need a refresher on the individual tools, start with essential Linux terminal commands. Here, the focus is how to connect them, preserve useful error messages and avoid turning a small typing mistake into an empty file.
Standard Input, Standard Output and Standard Error
Most terminal programs work with three standard streams. Standard input, or stdin, supplies incoming data. Standard output, or stdout, carries normal results. Standard error, or stderr, carries diagnostics, warnings and error messages.
The shell identifies these streams with file descriptor numbers: 0 for standard input, 1 for standard output and 2 for standard error. A file descriptor is simply a numbered handle through which a process reads or writes.
During an ordinary interactive session, input often comes from your keyboard, while both output streams appear in the terminal. Sharing the same display does not make stdout and stderr the same stream. You can send one to a file while leaving the other visible.
That separation matters when processing data. A program might produce valid records on stdout and report an unreadable directory on stderr. Keeping those streams separate prevents an error message from becoming an accidental record in your report.
What a Linux Pipe Actually Does
The pipe operator, |, connects the standard output of one command to the standard input of the next. A useful conceptual flow is:
Log records → matching records → line count
In that workflow, a log reader produces text, grep selects matching lines and wc counts the lines that remain. Each stage has one job. The pipe supplies the connection between them.
A pipeline does not normally wait for the first program to finish before starting the second. Its stages can run concurrently, with data moving between them as it becomes available. Some programs still need to collect their input before producing useful results: sort, for example, generally needs the complete input before it can deliver the final sorted sequence.
A pipe carries data, not command arguments
Standard input and command arguments are different interfaces. A program may accept a filename as an argument without accepting a list of filenames through stdin.
This explains why piping a list of paths into an arbitrary command does not necessarily make that command operate on those paths. Tools such as xargs can convert input into arguments, but filenames containing spaces or newlines require careful handling. For file operations, prefer tools with explicit support for safe filename processing rather than treating every line of text as a trustworthy pathname.
Check what each stage produces
Suppose you are examining disk usage. A producer reports sizes, a filter removes unwanted entries and a final stage selects the largest results. The data format must remain suitable at every connection. Human readable sizes, headings and decorative output can make a report pleasant to read but awkward to sort numerically.
The same distinction applies to Linux resource monitoring: choose output intended for processing where available, rather than assuming an interactive display will behave like a clean data feed.
Also check what you are counting. Matching lines are not necessarily matching events. One event might span several lines, while another line might contain several matches.
Redirection: Read From Files and Save Output
Redirection connects a stream to a different destination or input source. Unlike a pipe, it does not need another program on the receiving end.
| Operator | Purpose | Practical caution |
|---|---|---|
| |
Connect stdout to another command’s stdin | Does not include stderr by default |
> |
Write stdout to a file, replacing its contents | An existing file can be emptied before the program runs |
>> |
Append stdout to a file | Repeated runs can add duplicate results |
< |
Read stdin from a file | The receiving program must use standard input |
2> |
Write stderr to a file | Normal results still go to stdout |
2>> |
Append stderr to a file | Error logs can grow without a retention policy |
&& |
Run the next step after success | Tests exit status, not the quality of the result |
|| |
Run the next step after a nonzero exit status | A nonzero status does not always mean a program malfunctioned |
; |
Run commands sequentially without a success condition | Later steps can run after an earlier failure |
Overwrite versus append
Use replacement output for a report that should contain only the latest results. Use append output when retaining earlier entries is intentional, such as a running diagnostic record. Both forms can create a file if it does not exist, provided the destination is writable.
Replacement deserves more caution. The shell normally opens and truncates the destination before starting the command. A later failure does not restore the previous contents. Even an unsuccessful attempt can therefore leave an empty output file.
Never use the same file as both the input and replacement output of a transformation. The shell may empty it before the program has a chance to read it. Write to a separate temporary destination, validate the result, then replace the original through a deliberate step.
For valuable files, keep a recoverable copy through a proper Linux backup and recovery process. An overwrite operator is not an undo system.
Where the output file goes
A relative destination is resolved from the shell’s current working directory. An absolute path names its location from the filesystem root. Redirection does not create missing parent directories.
If an expected report seems to have vanished, check the working directory before assuming the command failed. The Linux filesystem and directory structure guide covers the path conventions behind this behavior.
Quote destination paths containing spaces. An unquoted path can be split or expanded by the shell before the redirection is applied.
Redirecting Errors and Why Order Matters
A normal pipe handles stdout only. Error messages can therefore remain visible in the terminal even when a command’s results are being filtered or saved. That is usually helpful: an empty report and a permission error tell a different story from an empty report with no diagnostics.
The notation 2>&1 makes descriptor 2 refer to descriptor 1’s current destination. It does not mean “put errors wherever stdout ends up later.” Bash processes redirections from left to right.
Consider two arrangements. Redirect stdout to a report file, then duplicate stdout’s destination onto stderr: both streams reach that file. Reverse those steps, and stderr keeps stdout’s original destination while stdout alone moves to the file. The Bash reference on redirection order and descriptor duplication documents this distinction.
Bash also provides &> for redirecting both output streams to one file, and |& for piping both streams to the next command. Do not assume Bash shorthand works unchanged in every shell.
Merging streams is useful for a diagnostic transcript. It is often a poor choice for structured data, where warnings can corrupt the input expected by the next stage.
Discarding errors into /dev/null can reduce known, harmless noise. During troubleshooting, though, it removes the evidence you need. Capture errors separately until you know why they occur.
Use tee to Save Output Without Losing Sight of It
Ordinary output redirection sends stdout away from the terminal. The tee utility provides a split: it reads stdin, copies that data to one or more files and also writes it to stdout. That continuing stdout can reach your display or another pipeline stage.
A practical arrangement is:
Diagnostic output → saved copy and continuing output → selected messages
The position of tee determines what gets saved. Place it before a filter to preserve the original incoming data. Place it after the filter to save only the selected results.
By default, tee replaces existing destination contents. Its append option preserves earlier content, as documented in the Debian manual for GNU tee. Treat its output files with the same care as other replacement or append operations.
Remember that tee receives only what reaches its stdin. Errors from an earlier command are not included unless you deliberately merge them into that stream.
Command Chaining: Control Which Step Runs Next
Pipes connect data. Chaining operators connect decisions.
A command returns an exit status when it finishes. The shell treats zero as success and nonzero as a condition that fails a success test. This status is separate from anything the command prints: a silent command can succeed, and a command that prints plausible output can still fail.
Use && for dependent steps
The && operator runs the next command only when the preceding command or pipeline returns zero.
A typical dependency is creating a working directory before entering it. If directory creation fails, the dependent step should not proceed under the assumption that preparation succeeded.
Package maintenance has similar dependencies, but avoid copying a universal update chain across distributions. The operations and their meanings differ between APT, DNF and Pacman. Check each package manager’s behavior before tying several administrative steps together.
Use || for a fallback or failure action
The || operator runs the next command when the preceding result is nonzero. This suits a fallback destination, a warning or another action that should happen only when the earlier step does not pass its success test.
Interpret that status in context. A search can return nonzero because it found no matching lines, not because it could not perform the search. “No results” and “broken command” may require different responses.
Use a semicolon for independent steps
A semicolon waits for the preceding command to finish, then starts the next without testing for success. It suits independent status checks. It does not suit a destructive operation that depends on successful preparation.
Mixed chains need extra care. The operators && and || have equal precedence and are evaluated with left associativity. In a preparation, success action and fallback arrangement, the fallback can run because the success action failed, even if preparation succeeded. These rules are defined in the POSIX shell command language provisions for command lists.
Once a chain contains several conditions, use a readable conditional structure rather than compressing the logic into punctuation.
Pipeline Success Is Not Every Command’s Success
In Bash, with pipefail disabled, a pipeline normally reports the exit status of its final command. That can hide an earlier failure.
Consider a report generator feeding a filter, followed by a file writer. The generator fails, but the writer successfully saves the small amount of data it received. A successful final stage does not prove that the original report is complete.
Bash’s pipefail option changes the pipeline result to the status of the rightmost command that returned nonzero, or zero when all stages returned zero. It does not stop every other stage immediately or reverse files already written. The Bash pipeline documentation covers both the default status rule and pipefail behavior.
This affects chaining too. A success action attached to a pipeline tests that pipeline’s reported status, not an independent assessment of every stage.
Use pipefail deliberately in automation, especially where incomplete output would be dangerous. Also account for expected nonzero results and consumers that intentionally stop reading early. Status handling should match the task, not just satisfy a familiar shell habit.
Grouping Commands and Running Work in the Background
Grouping lets several commands share a redirection. Instead of assigning a separate output destination to each diagnostic check, you can collect a group’s output into one report.
In Bash, braces group commands in the current shell environment. Parentheses run a group in a subshell environment. That difference matters for changes to variables or the working directory: changes made inside a subshell do not carry back into the parent shell.
Neither form makes the group transactional. If an early command changes a file and a later command fails, grouping does not undo the earlier change.
A single ampersand, &, serves another purpose: it starts work asynchronously so the shell does not wait before continuing. It is not a shorter spelling of &&, and receiving a prompt again does not prove the background task succeeded.
For persistent services, use proper systemd service management rather than treating an interactive background job as a service supervisor.
Build a Log Processing Workflow in Stages
Suppose you need a saved report of errors from one service during a defined period. Start by choosing the service and time range at the producer, before adding general text processing. With the system journal, journalctl filtering and troubleshooting can reduce the amount of unrelated output entering the pipeline.
Next, inspect the format. Decide whether you need readable messages, complete records or structured fields. Then add a filter and check a sample of its results before introducing a count or summary.
Decide where to save evidence. Keeping the original selected records allows you to revisit a summary that looks suspicious. Saving only the final number leaves little room for investigation.
Finally, test the failure path. What happens if the service has no entries, access is denied or the destination cannot be written? A reliable workflow distinguishes those cases instead of presenting every empty result as “nothing happened.”
Troubleshooting Missing, Delayed or Unexpected Output
Permission denied during redirection
The shell performs redirection before launching the command. Giving a program elevated privileges does not automatically give the calling shell permission to open a protected output file.
Check whether the error came from the shell opening the destination or from the program accessing its own resources. The fix depends on Linux file permissions and ownership, not on repeatedly adding administrative privileges to the same failed arrangement.
Output arrives in bursts
A program may buffer output differently when writing into a pipe rather than directly to a terminal. A live stream can therefore appear delayed even while its producer is working.
Check the program’s own buffering controls first. GNU stdbuf can adjust standard stream buffering for compatible programs, but it does not affect every input and output implementation. The Debian manual for stdbuf describes those restrictions.
Also check whether a stage needs the complete input. Sorting a stream that never ends cannot produce a final complete ordering.
Operators appear as ordinary text
Quoted or escaped operators lose their usual shell role. A pipe character inside a quoted search pattern belongs to that pattern; it does not connect two commands. This matters when searching for punctuation or using regular expressions.
Build workflows one connection at a time: verify the producer, inspect the consumer’s input, choose the output destination, then add success and failure handling. Short, explicit chains are easier to trust than a long line that happens to work once.