The grep command in Unix and Linux and its common usage

Monday, December 29, 2008

I have been using the grep command in UNIX virtually everyday in my work but yet sometimes some of the file manipulation tasks annoy me, especially the pattern matching tasks for which I always needed a quick solution, so I decided to do a writeup on grep command and its practical applications, hope this may make the life of UNIX users a little easier.


The grep series of commands.... For a customary introduction, the grep command is used to find/print lines in a file that matches a pattern, it comes in different flavors, there is an enhanced version of the grep command, namely the egrep which does some advanced regular expression pattern matching on text files and the fgrep command which is used to search text in a file without any regular expession interpretation of the search pattern.


Some common grep usage options....

grep pattern filenames - Displays lines in files which matches the pattern.
grep -i pattern filenames - Displays lines in files which matches the pattern, ignores case while matching the pattern.
grep -c pattern filenames - Prints the count of the number of lines which matches the pattern in files.
grep -l pattern filenames - Lists the names of files which matches the pattern along with the lines which matched the pattern.

Similarly egrep command is used with different enhanced pattern matching options used with regular expressions like

egrep (x|y) filename - Displays lines which match the pattern x or y.
egrep (x|y)+ filename - Displays lines which matches atleast one or more text x or y.

fgrep is mainly used to avoid using search patterns as regular expressions or metacharacters

fgrep '*' filename - matches the occurence of pattern * in filenames, note here * doesn't mean a regular expression or a metacharacter.

Applications of grep command:

grep
command can be used as a standalone utility to match a pattern in a file or piped to other utility commands like sed or awk to manilupate and extract text fom files, some of the common applications of grep from simple to little tricky ones are shown below.

The text file used to illustrate the below grep commands is shown below.
bash-3.00# cat textfile.txt 
This is line 1
This is line 2
This is line 3
This is line 4
This is line 5
bash-3.00#

1. Displaying or Saving a file with line numbers

There are several ways to save or display a text file with line numbers, some of them are
bash-3.00# grep -n '^' textfile.txt 
1:This is line 1
2:This is line 2
3:This is line 3
4:This is line 4
5:This is line 5
The above command matches the first line of the input file ('^' matches a pattern at the beginning of a line), or it matches all the lines in a file, one can also use grep -n '$' textfile.txt to produce the same output (the regular expression '$' means "ends with").
bash-3.00# grep -n '$' textfile.txt 
1:This is line 1
2:This is line 2
3:This is line 3
4:This is line 4
5:This is line 5
Also one can use the Regular expression ".*" to match and print all lines in the file with line numbers.
bash-3.00# grep -n '.*' textfile.txt 
1:This is line 1
2:This is line 2
3:This is line 3
4:This is line 4
5:This is line 5
The RE . (dot) is used to match a single character and the pattern * is used to match zero or more occurences, thus printing all line numbers when used with grep -n.

2. Deleting or Removing all lines in a file which matches a pattern

The grep -v command can be used display lines which DON'T match a pattern or in other words remove lines which matches a pattern in a file.

For example in the above text file, to remove or delete lines which matches the pattern "line 1", the command would be
bash-3.00# grep -v "line 1" textfile.txt 
This is line 2
This is line 3
This is line 4
This is line 5
To delete lines which matches the pattern "line 1" or "line 3", use the egrep command
bash-3.00# egrep -v '(line 1|line 3)' textfile.txt 
This is line 2
This is line 4
This is line 5
3. Deleting the first line which matches a pattern in a file

This is an example which uses the grep command piped to other UNIX commands like sed. To delete only the first line which matches a pattern in a file, use the following grep - sed pipe.

The below command deletes the first line which matches the pattern "line" in the above text file.
bash-3.00# grep "line" textfile.txt | sed 1d
This is line 2
This is line 3
This is line 4
This is line 5
The sed 1d command deletes the first line in the above grep output.

4. Deleting the last line which matches a pattern in a file

To delete the last line which matches a pattern in a file, use the grep - sed command pipe as shown below.

The below command deletes the last line which matches the pattern "line" in the above text file.
bash-3.00# grep "line" textfile.txt | sed '$d'
This is line 1
This is line 2
This is line 3
This is line 4
The sed $d command deletes the last line in the above grep output.

Likewise there are countless applications of grep command piped with other utility commands like awk, sed, etc to match patterns in a file and I will add more as and when I find something interesting.

Polymorphism in Java Arrays

Wednesday, December 24, 2008

When I was preparing for my SCJP 6.0,I was learning many interesting features of java, one such thing was how polymorphism would apply to java arrays as compared to java objects.

For a customary introduction, polymorphism is a OOPS concept which allows a class to hold a reference to an instance of any of its subclasses and invoke methods of different subclass objects to which the reference points to at runtime, in other words this is also called runtime ploymorphism in the sense the object on which the method is invoked is determined at runtime, of course the subclass should override the method which the superclass defines and those are the only methods which can be invoked through the superclass reference to the subclass instance. Ok, enough on polymorphism.

I was discussing how polymorphism would work with arrays with one of my friend (in fact I attribute this entry to him as I learnt this through his fancy questions), the scenario is like this.

1. Declare an array of type baseclass which points to the subclass array object (Polymorphism or polymorphic reference)
2. Assign the first array element of baseclass to the subclass object (Not a great deal of thinking required to tell that its allowed).
3. Assign the second array element of baseclass to the baseclass object.

Now we are interested in how the behavior would be on the assignment in line 3, lets go through the code.

Listing 1: subClass.java

1: /**
 2:  * @(#)subClass.java
 3:  *
 4:  *
 5:  * @author: Prasanna S 
 6:  * @version 1.00 2008/11/25
 7:  */
8:
9: class baseClass {
10:         public baseClass() {    
11:         }
12: }
13:
14: public class subClass extends baseClass {
15:
16:     public subClass() {
17:     }
18:
19:     public static void main (String args[]) {
20: 
21:         // Creating a baseClass Array reference to point to subClass Array object - OK
22:         baseClass baseClassArray [] = new subClass [5];
23: 
24:         // Assigning a baseClass Array element to subClass instance - OK
25:         baseClassArray[0] = new subClass();
26: 
27:         // Assigning a baseClass Array element to baseClass instance 
28:         // Runtime exception - ArrayStoreException
29:         
30:         baseClassArray[1] = new baseClass();
31:     }
32: }
Output and Explanation:

As commented out in the code, the above program will compile successfully, but will throw a runtime exception at line 30 as shown below.

Exception in thread "main" java.lang.ArrayStoreException: baseClass
at subClass.main(subClass.java:30)

There are two classes named baseClass and subClass in the above listing, and in line 22, we create a base class array reference to point to a subclass object array reference.

Line 25 assigns a subClass object to baseClassArray[0] which is acceptable according to polymorphism rules.

Line 30 assigns a baseClass object to baseClassArray[0], which results in ArrayStoreException because you can't have a reference to a baseClass object in an subClass array object NEVER, note that its very easy to get confused and accept that this code would work if one sees the reference type in the left hand side of the assignment.

The lesson: If we try to store a baseClass reference in an object of subClass type, we will get a ClassCastException for java objects as it violates polymorphism principle or IS-A relationship (a subclass IS-A baseclass through inheritance, but not otherwise), but for arrays you will get an ArrayStoreException.

Sun Certified Solaris Associate (SCSAS) - Your first Solaris Certification

Sunday, December 21, 2008

Today I took my first Solaris 10Certification exam Sun Certified Solaris Associate (SCSAS) and cleared it with 86% score, there were 52 questions in it and I got 45 of them right. Even though I have been working in Solaris environment for some years, but I wanted to rate/test myself on my Solaris skills and with little preparation, got a good result, also its not a cake walk either, as always I learn something new with every adventure, so that served the purpose of going for this certification at the first place and spending some time on it.

The Sun Certified Solaris Associate (SCSAS) is the recent Solaris 10 Certification exam introduced by Sun mainly for beginners who want to get a basic knowledge of UNIX environment through Solaris 10, if you are a UNIX newbie, I strongly RECOMMEND this to you as you will get a solid foundation in Solaris/UNIX environments by taking SCSAS and will feel much more confident in working with Solaris or other UNIX Environments.

For intermediate or experienced Solaris Programmers who are keen on pursuing advanced certifications like Sun Certified System Administrator for the Solaris Operating System (SCSA) or Sun Certified Network Administrator for the Solaris Operating System (SCNA) or Sun Certified Security Administrator for the Solaris Operating System (SCSECA), this may be a good warm up to refresh their basic Solaris skills before advancing to further levels, in any case having a certification like this would be handy as one can establish themselves in Solaris with an officially recognized certification like this.

Also for University students, this would be the first place to start with to get some recognition for your UNIX/Solaris skills before getting into the industry, OK lets look at the concepts tested in SCSAS, I have subdivided into five categories.

The Certification Objectives....

1. Understanding Solaris desktop environment, command line, searching man pages, introduction of Korn, Bourne, C Shells and simple system security concepts you need to be aware of.
2. Getting comfortable with file systems, directory manipluation, search commands, vi editor, file/directory permission concepts and Configuring Access Control lists.
3. Understanding UNIX Processes, the commands associated in creating tasks, killing processes and other aspects process handling.
4. Advanced shell functionality, shell metacharacters, shell scripting, customizing your environments on different shells.
5. Archiving techniques and commands for remote file transfer.

Getting hands on in these concepts will always be helpful for any UNIX environments and from my experience, even though I was initially complacent and took a practice test hoping to complete the formalities soon, I was really humbled as this certification requires some preparation no matter how much UNIX experience you may, it will just expose the areas you need to focus as it did it for me, prompting me to learn for a couple of days, tying up some loose ends.

Tips and Suggestions.... Finally my suggestions for those who wanna take this exam would be as follows.

1. If you are a UNIX newbie, give yourself a couple of months and most importantly work in a Solaris 10 environment, try and experiment in your system, get as comfortable as you can, master all the features and objectives of SCSAS so that you WILL never forget those anytime as that's the purpose of doing this certification.

2. If you are an intermediate level Solaris programmer, then this is for you as I mentioned earlier, it will expose the areas you need to focus and improve on, so that you can become a better UNIX programmer.

3. If you call yourself as a Solaris Guru, then it won't hurt to reaffirm that belief through this certification, I am sure some of them might require you to think.

References and Readings Though there were not any official certification material released for this, if you need some reference, any reading on Sun Solaris 10 or Solaris 10 Complete Reference may help, but those might be a little overkill for beginners, I can help with some beginners materials for this through Sun, also Sun is promoting/encouraging Solaris awareness among beginners and university students with many good initiatives, do let me know if you are serious about it.

SCJP Sun Certified Java Programmer 1.6 Tips and Tricks

Saturday, December 13, 2008

This week I took the SCJP Sun Certified Programmer for Java 6 Exam 310-065and cleared it with 73%. The SCJP 1.6 exam is little tricky in that Sun has raised the bar a little to pass the exam, you should get 65% to pass the SCJP 1.6 exam, you will have 72 questions and should answer atleast 65% of them right or atleast 47 right responses.

Ok, why should anyone go for Sun Certified Java Programmer (SCJP) 1.6 exam, the reason varies, if you want to have a career in Java but need a strong foundation then its for you, I took it because I wanted to refresh the latest java features, to master some key OO concepts, assertion mechanisms, object oriented concepts like polymorphism, overloading, inheritance, object serialization, inner classes/nested classes, threading/synchronization, get hands on with new features of java 1.5 and above like Generics, etc, I should say I am quite satisfied with the way it went.

A word of caution though, SCJP 1.6 is not only for beginners of Java programming language but for seasoned Java programmers too, although I knew Java before, but I really learnt a lot from the SCJP 1.6 preparation and you will too realize that once you go through the SCJP 1.6 objectives, it's most likely that it will clear some of the misconceptions may have had in java and will set a firm foundation on Java programming language and its important concepts with its potential use in real world applications.

The following may be some useful tips and tricks for those who are planning to take SCJP 1.6.

1. Reference for SCJP 1.6

The only reading I went through till date for SCJP 1.6 preparation was SCJP Sun Certified Programmer for Java 6 Exam 310-065by Katherine Sierra, Bert Bates, if you grasp the contents in this reading and take the practice tests which includes tricky questions and lucid explanation of answers, you WILL sail through for sure, but make sure that you go through all the chapters even if you are a good java programmer because its likely that you will learn something new in some section of this book, NEVER ignore any sections on reading.

2. Time for preparation

Ideally it will take a month and a half to prepare for the exam including the two practice tests, but if you want to get the best of of the SCJP 1.6, give a couple of months, its not that getting through SCJP which should matter, but its where you take your Java skills through SCJP preparation which really matters, in that case to get the best worth out of it, it may take a couple of months as was the case for me.

3. Key Concepts

The objectives of SCJP 1.6 is to test your skills on Java fundamentals, Object Oriented concepts like polymorphism, overloading, overriding, Generics concepts, Inner classes, Threads and Synchronization, differences between abstract classes & interfaces, packaging, rudimentary concepts like flow control, declarations, etc. However the most important concepts which is likely to stump you would be on sections in

1. Thread Programming and Syncronization
2.
Generics
3. Inner Classes, use of static classes and methods
4. OOPS Concepts like polymorphism
5. Garbage Collection features

One needs to be vary sharp in answering questions in these sections.

Therefore to start with learn simple, fundamental concepts of java like declaration, looping constructs, flow control, etc, then look keenly on the core features of java mentioned above, its likely that every rule you learn on generics, OOPS concepts and other important features of java will be crucial to your success on SCJP 1.6.

4. Practice, Practice, Practice....

Give a months time for going through the contents of the book, learn all chapters page by page, my advice is never give the tests at the end of each chapter till you finish your reading although you may go through the two minute drills in the above reading at the end of each chapter which summarizes the key points for that chapter.

Reserve the second part of the preparation completely on practice tests, go through the individual practice tests at the end of each chapter, learn the key concepts which require additional focus, then take the the sample exams, again the best way to go through the sample exam is to attempt individual questions and then look the answers for them even if they are right or wrong, learning in this way would ensure that you will never forget the key concepts you will be tested on in SCJP 1.6.

Last but not the least repeat your practice tests at the end of the chapter and sample tests, its most likely that you may have some minor mistakes even if you take them twice or more, but the more you learn from mistakes, the stronger you will be on the SCJP 1.6, best of luck!

I will also be blogging on some useful java code and key java concepts for my reference and also for your reading, you will find themselves useful for sure to clear your doubts and learn new concepts, keep visiting :).


Copyright © 2016 Prasanna Seshadri, www.prasannatech.net, All Rights Reserved.
No part of the content or this site may be reproduced without prior written permission of the author.