Welcome to post 18 of my 100 day challenge. Checkout my introduction for some background.
This is post three of my LFCS series. In this post I will continue the discussion of the intricacies of the Linux command line interface. You can go back to the overview post for a brief introduction or take a look at post one for instructions on setting up the exam practice system which I will be using throughout this series.
This is the part two of a three part article. For part one click here.
LFCS command line part 2
Manipulating text files from the command line
Cat and Tac
The cat command short for catenate. Can be used to output 1 or more files to standard output.
$ cat poem.txt Strive to discover the mystery before life is taken from you. If while living you fail to find yourself, to know yourself, how will you be able to understand the secret of your existence?
Tac is the opposite of cat. Prints out one or more files in reverse.
$ tac poem.txt the secret of your existence? how will you be able to understand to know yourself, to find yourself, If while living you fail before life is taken from you. Strive to discover the mystery
Sed stands for stream editor. It is used for parsing and transforming text using a succinct language.
In the example below using poem.txt the first occurrence of your will be replaced with my on every line. The transformed text will be send to standard output which therefore leaves the original file intact. You can redirect the output to another file with the > operator.
$ sed 's/your/my/' poem.txt Strive to discover the mystery before life is taken from you. If while living you fail to find myself, to know myself, how will you be able to understand the secret of my existence? $ cat poem.txt Strive to discover the mystery before life is taken from you. If while living you fail to find yourself, to know yourself, how will you be able to understand the secret of your existence?
Redirection to a new file:
$ sed 's/your/my/' poem.txt > poem2.txt $ cat poem2.txt Strive to discover the mystery before life is taken from you. If while living you fail to find myself, to know myself, how will you be able to understand the secret of my existence?
You saw above how to replace the first occurrence of a string on each line below is how to transform all occurrences:
$ sed 's/e/E/g' poem.txt StrivE to discovEr thE mystEry bEforE lifE is takEn from you. If whilE living you fail to find yoursElf, to know yoursElf, how will you bE ablE to undErstand the sEcrEt of your ExistEncE?
Special characters need to be escaped with a back slash \ to stop them being interpreted by the shell. Put a backslash before $.*/^ if you actually want to search for them as part of the string you are transforming.
You can pipe the output of a log to sed and use it to search for particular occurrences of a string.
[root@centospractice ~]# cat /var/log/messages | sed -n '/initialized/p' Apr 22 22:20:35 centospractice kernel: Security Framework initialized Apr 22 22:20:35 centospractice kernel: devtmpfs: initialized Apr 22 22:20:35 centospractice kernel: SCSI subsystem initialized Apr 22 22:20:35 centospractice kernel: type=2000 audit(1429737629.274:1): initialized
In the above command we are piping in /var/log/messages to sed and using -n to hide all lines which don’t match our expression. The p at the end of the command is telling sed to print the results to standard ouput.
You can use sed to output just the commend lines of a configuration file. This is useful if you want to see the comments of a utility on the fly.
[root@centospractice ~]# sed -n '/^#/p' /etc/sudo.conf # # Default /etc/sudo.conf file # # Format: # Plugin plugin_name plugin_path plugin_options ... # Path askpass /path/to/askpass # Path noexec /path/to/sudo_noexec.so # Debug sudo /var/log/sudo_debug all@warn # Set disable_coredump true # # Sudo plugins:
Grep is a powerful tool for search for patterns on Linux systems. Below is a table of the most common commands you will require when using grep.
|Flag||What it does|
|-l||List only the file names containing matches.|
|-n||Output the line number of the match.|
|-v||Invert, show all lines which don’t match.|
|-h||Don’t output the file names.|
|-H||Print the file name containing the match.|
|-c||Count the number of results.|
|-E||Use an extended regular expression.|
- grep the user list for your user
[root@centospractice ~]# grep root /etc/passwd root:x:0:0:root:/root:/bin/bash operator:x:11:0:operator:/root:/sbin/nologin
- grep for a user and show the line number the result occurs on
[root@centospractice ~]# grep -n ftp /etc/passwd 14:ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
- As with the other commands you can pipe to grep and output the results.
[root@centospractice ~]# ll /var | grep lib drwxr-xr-x. 15 root root 4096 Apr 22 22:17 lib
- Recursively search for files using grep
[root@centospractice ~]# grep -r root /etc/ /etc/ssh/sshd_config:#ChrootDirectory none /etc/makedev.d/00macros:=ALLREAD 644 root root /etc/makedev.d/00macros:=ALLWRITE 666 root root /etc/makedev.d/00macros:=FLOPPY 660 root floppy /etc/makedev.d/00macros:=KMEM 640 root kmem /etc/makedev.d/00macros:=PRINTER 660 root lp /etc/makedev.d/00macros:=PTY 666 root tty /etc/makedev.d/00macros:=ROOT 600 root root /etc/makedev.d/00macros:=SERIAL 660 root uucp
- Recursively search only output files containing matches
[root@centospractice ~]# grep -rl root /etc /etc/gshadow- /etc/postfix/master.cf /etc/postfix/main.cf /etc/ssh/sshd_config /etc/makedev.d/00macros /etc/makedev.d/01linux-2.6.x /etc/shadow
Tune in tomorrow for part 3 of LFCS command line where we will discuss further on how to manipulate text files from the command line. Including cut, sort and uniq.
Can you improve on any of the tips I’ve discussed here? If you can let me know in the comments.