0

LFCS command line part 2

Welcome to post 18 of my 100 day challenge. Checkout my introduction for some background.

This is post three of my LFCS series. In this post I will continue the discussion of the intricacies of the Linux command line interface. You can go back to the overview post for a brief introduction or take a look at post one for instructions on setting up the exam practice system which I will be using throughout this series.

This is the part two of a three part article. For part one click here.

LFCS command line part 2

Manipulating text files from the command line

Cat and Tac

The cat command short for catenate. Can be used to output 1 or more files to standard output.

$ cat poem.txt
Strive to discover the mystery
before life is taken from you.
If while living you fail
to find yourself,
to know yourself,
how will you be able to understand
the secret of your existence?

Tac is the opposite of cat. Prints out one or more files in reverse.

$ tac poem.txt
the secret of your existence?
how will you be able to understand
to know yourself,
to find yourself,
If while living you fail
before life is taken from you.
Strive to discover the mystery

Sed

Sed stands for stream editor. It is used for parsing and transforming text using a succinct language.

In the example below using poem.txt the first occurrence of your will be replaced with my on every line. The transformed text will be send to standard output which therefore leaves the original file intact. You can redirect the output to another file with the operator.

$ sed 's/your/my/' poem.txt
Strive to discover the mystery
before life is taken from you.
If while living you fail
to find myself,
to know myself,
how will you be able to understand
the secret of my existence?

$ cat poem.txt

Strive to discover the mystery
before life is taken from you.
If while living you fail
to find yourself,
to know yourself,
how will you be able to understand
the secret of your existence?

Redirection to a new file:

$ sed 's/your/my/' poem.txt > poem2.txt
$ cat poem2.txt
Strive to discover the mystery
before life is taken from you.
If while living you fail
to find myself,
to know myself,
how will you be able to understand
the secret of my existence?

You saw above how to replace the first occurrence of a string on each line below is how to transform all occurrences:

$ sed 's/e/E/g' poem.txt
StrivE to discovEr thE mystEry
bEforE lifE is takEn from you.
If whilE living you fail
to find yoursElf,
to know yoursElf,
how will you bE ablE to undErstand
the sEcrEt of your ExistEncE?

Special characters need to be escaped with a back slash to stop them being interpreted by the shell. Put a backslash before $.*/[]^ if you actually want to search for them as part of the string you are transforming.

You can pipe the output of a log to sed and use it to search for particular occurrences of a string.

[root@centospractice ~]# cat /var/log/messages | sed -n '/initialized/p'
Apr 22 22:20:35 centospractice kernel: Security Framework initialized
Apr 22 22:20:35 centospractice kernel: devtmpfs: initialized
Apr 22 22:20:35 centospractice kernel: SCSI subsystem initialized
Apr 22 22:20:35 centospractice kernel: type=2000 audit(1429737629.274:1): initialized

In the above command we are piping in /var/log/messages to sed and using -n to hide all lines which don’t match our expression. The p at the end of the command is telling sed to print the results to standard ouput.

You can use sed to output just the commend lines of a configuration file. This is useful if you want to see the comments of a utility on the fly.

[root@centospractice ~]# sed -n '/^#/p' /etc/sudo.conf
#
# Default /etc/sudo.conf file
#
# Format:
# Plugin plugin_name plugin_path plugin_options ...
# Path askpass /path/to/askpass
# Path noexec /path/to/sudo_noexec.so
# Debug sudo /var/log/sudo_debug all@warn
# Set disable_coredump true
#
# Sudo plugins:

grep

Grep is a powerful tool for search for patterns on Linux systems. Below is a table of the most common commands you will require when using grep.

Flag What it does
-i Ignore case.
-l List only the file names containing matches.
-n Output the line number of the match.
-r Recursively search.
-v Invert, show all lines which don’t match.
-h Don’t output the file names.
-H Print the file name containing the match.
-c Count the number of results.
-E Use an extended regular expression.
  • grep the user list for your user
[root@centospractice ~]# grep root /etc/passwd
root:x:0:0:root:/root:/bin/bash
operator:x:11:0:operator:/root:/sbin/nologin
  • grep for a user and show the line number the result occurs on
[root@centospractice ~]# grep -n ftp /etc/passwd
14:ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
  • As with the other commands you can pipe to grep and output the results.
[root@centospractice ~]# ll /var | grep lib
drwxr-xr-x. 15 root root 4096 Apr 22 22:17 lib
  • Recursively search for files using grep
[root@centospractice ~]# grep -r root /etc/
/etc/ssh/sshd_config:#ChrootDirectory none
/etc/makedev.d/00macros:=ALLREAD 644 root root
/etc/makedev.d/00macros:=ALLWRITE 666 root root
/etc/makedev.d/00macros:=FLOPPY 660 root floppy
/etc/makedev.d/00macros:=KMEM 640 root kmem
/etc/makedev.d/00macros:=PRINTER 660 root lp
/etc/makedev.d/00macros:=PTY 666 root tty
/etc/makedev.d/00macros:=ROOT 600 root root
/etc/makedev.d/00macros:=SERIAL 660 root uucp
  • Recursively search only output files containing matches
[root@centospractice ~]# grep -rl root /etc
/etc/gshadow-
/etc/postfix/master.cf
/etc/postfix/main.cf
/etc/ssh/sshd_config
/etc/makedev.d/00macros
/etc/makedev.d/01linux-2.6.x
/etc/shadow

Tune in tomorrow for part 3 of LFCS command line where we will discuss further on how to manipulate text files from the command line. Including cut, sort and uniq.

Subscribe to my feed either by E-mail or by RSS to receive updates as they happen.

Can you improve on any of the tips I’ve discussed here? If you can let me know in the comments.

Jason Edwards