0

LFCS command line part 3

Welcome to post 19 of my 100 day challenge. Checkout my introduction for some background.

This is post four of my LFCS series. In this post I will continue the discussion of the intricacies of the Linux command line interface. You can go back to the overview post for a brief introduction or take a look at post one for instructions on setting up the exam practice system which I will be using throughout this series.

This is the part three of a three part article. For part one click here, for part 2 click here.

LFCS command line part 3

Manipulating text files from the command line

cut

The cut command is an extraction tool. Used to extract sections from lines of input. The lines are usually from a file but can be piped to it from stdout.

The most common options of cut are

Option Explanation
-f list Cut at field number.
-c list Cut by the column position.
-d Provide the field delimiter to cut by. The default is tab.

Examples

[root@centospractice ~]# cut -d ":" -f 1 /etc/passwd
root
bin
daemon
adm
lp
sync
shutdown
halt
mail
uucp
operator
games
gopher
ftp
nobody
vcsa
saslauth
postfix
sshd

The command above cuts /etc/passwd using the colon as the delimiter and returns the first field. You can also specify the delimiter without quotes but if it is a special character (colon is) you have to escape it with a backslash. In this event the command about without the quotes would look like this:

cut -d\: -f 1 /etc/passwd

I would say this method is more succinct. But each to their own as both approaches work.

tr

The tr (translate) command translates characters in a file from one form to another.

Lets use our poem from earlier.

[root@centospractice ~]# cat poem.txt
Strive to discover the mystery
before life is taken from you.
If while living you fail
to find yourself,
to know yourself,
how will you be able to understand
the secret of your existence?

We can make all the letters uppercase like so.

[root@centospractice ~]# cat poem.txt | tr [:lower:] [:upper:]
STRIVE TO DISCOVER THE MYSTERY
BEFORE LIFE IS TAKEN FROM YOU.
IF WHILE LIVING YOU FAIL
TO FIND YOURSELF,
TO KNOW YOURSELF,
HOW WILL YOU BE ABLE TO UNDERSTAND
THE SECRET OF YOUR EXISTENCE?

You can tell from the command above that we are saying for all lower case chars [:lower:] you find, make them upper case [:upper:] therefore all characters already upper case are not transformed.

sort

The sort command is used to sort the lines in a text file. Below is a table containing the most used options:

Option Explanation
-b Ignore any leading blank characters on a line.
-r Reverse the order of the lines.
-R Randomly sort the lines.
-f Ignore case of the characters on the line.
[root@centospractice ~]# ls /etc/init.d
auditd ip6tables lvm2-lvmetad netfs rsyslog udev-post
blk-availability iptables lvm2-monitor network sandbox
crond iscsi mdmonitor postfix saslauthd
functions iscsid multipathd rdisc single
halt killall netconsole restorecond sshd

Random sort of the listing

[root@centospractice ~]# ls /etc/init.d | sort -R
network
restorecond
sshd
lvm2-monitor
iscsid
halt
netconsole
rsyslog
functions
single
iscsi
killall
udev-post
sandbox
ip6tables
lvm2-lvmetad
crond
mdmonitor
multipathd
auditd
saslauthd
iptables
rdisc
netfs
blk-availability
postfix

nl

The nl command writes out a file to standard output but also adds line numbers.

[root@centospractice ~]# cat /etc/passwd | nl
 1 root:x:0:0:root:/root:/bin/bash
 2 bin:x:1:1:bin:/bin:/sbin/nologin
 3 daemon:x:2:2:daemon:/sbin:/sbin/nologin
 4 adm:x:3:4:adm:/var/adm:/sbin/nologin
 5 lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
 6 sync:x:5:0:sync:/sbin:/bin/sync
 7 shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
 8 halt:x:7:0:halt:/sbin:/sbin/halt

join

The join command can be used to join the lines of two text files together where they share a common field.

In the example below we attach a file containing pets to a file containing colours using the line number as the common field.

[root@centospractice ~]# cat colours.txt
 1 blue
 2 black
 3 brown
 4 green
[root@centospractice ~]# cat pets.txt
 1 fish
 2 dog
 3 cat
 4 snake
[root@centospractice ~]# join pets.txt colours.txt
1 fish blue
2 dog black
3 cat brown
4 snake green

uniq

The uniq command can be used to filter out only the uniq lines or characters from a file.

Option Explanation
-u Prints only uniq occurrences.
-w Used to specify N characters in lines.
-c Prefixes the line by the number of occurrences.
[root@centospractice ~]# cat numbers | sed 's/ /\n/g' | sort | uniq -c
 2 1
 1 2
 3 3
 3 4
 2 5
 1 6
 5 7
 1 8
 2 9

awk

There are whole books dedicated to the awk command so I doubt I could do it justice in this blog post. Here is a link to a good tutorial. But I have included one example below for illustration.

[root@centospractice ~]# awk 'length($0) > 50' /etc/passwd
vcsa:x:69:69:virtual console memory owner:/dev:/sbin/nologin
saslauth:x:499:76:"Saslauthd user":/var/empty/saslauth:/sbin/nologin
sshd:x:74:74:Privilege-separated SSH:/var/empty/sshd:/sbin/nologin

The example above prints to standard output all lines in /etc/passwd that have more than 50 characters.

Tune in tomorrow for the first part of my revision article on LFCS Filesystem & storage where we will discuss how to archive and compress data.

Subscribe to my feed either by E-mail or by RSS to receive updates as they happen.

Can you improve on any of the tips I’ve discussed here? If you can let me know in the comments.

Jason Edwards