Shell Programming: Adding lines to a file, before and after the file contents

Sunday, July 27, 2008

While appending lines after the contents of the file is a relatively easy task using cat command, I came across several applications where I need to add lines before the contents of the file in bash shell.

One can use cat >> command to append contents to the end of file, but if you need to add/print lines before or after the file's content, the following combination of echo and cat commands may also be useful.

For example, assume that the file newfile.txt has the following content.
bash-3.00$ cat newfile.txt
This is the first line of the file
This is the second line of the file
bash-3.00$
The below commands add/print single/multiple lines before and after the file's contents respectively.

Add lines before the contents of the file:
bash-3.00$ echo -e "This is the start of file\nNow printing the contents of the file" | cat - newfile.txt
This is the start of file
Now printing the contents of the file
This is the first line of the file
This is the second line of the file
bash-3.00$
Add lines after the contents of the file:
bash-3.00$ echo -e "This is the end of the file\nThe file contents are printed" | cat newfile.txt -
This is the first line of the file
This is the second line of the file
This is the end of the file
The file contents are printed
bash-3.00$
One more thing to note in the above command is the usage echo -e to output newline characters, for example there is a variable with /n character, then using echo -e would print a new line instead of /n, to see the difference, look at the commands below.
bash-3.00$ var="This is the first line\nThis is the second line"

bash-3.00$ echo $var
This is the first line\nThis is the second line
bash-3.00$

bash-3.00$ echo -e $var
This is the first line
This is the second line
bash-3.00$

Asynchronous Socket Programming

Many a times I got this question from my friends, how to transfer data from one client to another through a server, the solution is very easy using the select call supported in sockets.

To understand the benefit of select call in sockets, one should understand asynchronous socket programming. Assume a case where many clients connect to a server and send data for processing concurrently, then the server has to handle the the clients asynchronously or process the data as and when they are available from any of them.

In synchronous socket programming, the server processes each client sequentially, in this case when it waits for a response/data from a client using the recv call, it blocks or in other words the recv call cannot return until there is some data received from the socket, in a real time scenario, this way of handling clients is inefficient in the sense that all other connected clients need to wait till the server completes processing the current one.

One way handle such a scenario is for the server to accept a connection and fork a child process to handle that client's communication. But this way of handling multiple clients sometimes consumes system resources.

Therefore one needs a more elegant way to asynchronously handle client requests or the ability to read, write from multiple sockets whenever they are ready to be read or written, which is where the select call comes handy.

To explain the use of select system call, I will illustrate a TCP/IP chat Client server program, where the functionalities of the server and the client program were mentioned below.

The chat client and server program were implemented in python for the ease of understanding, note that you need not necessarily use the python client program below to connect to the server, even a telnet appllication can do the same, try to see how the chat client/server works by launching multiple clients in different windows connected to the chat server.

TCP/IP Chat Server:

1. Accepts connection from multiple clients
2. Use select call to get the list of available sockets which are ready to be read.
3. Whenever a client connects, the server notifies all other connected clients of this new connection, in the same way the server notifies all when a client quits or that client connection is lost.
4. The server broadcasts data sent by a client to all other connected clients.

TCP/IP Chat Client:

1. Connects to the server and starts two threads, one to process received data and one for getting data input to be sent to other connected clients through the server.
2. When the client quits (using q or Q) or the server is suddenly down (handle the worst case scenario), the socket is closed and the process exits.
3. Whenever any thread closes the socket connection, it interrupts the main program using the thread.interrupt_main() call, then the main exits.

The syntax used for the select call is as follows

read_sockets,write_sockets,error_sockets = select.select(CONNECTION_LIST,[],[])

The select call returns three lists, the list of sockets which are ready to be read, written and those which caused an error, since we are interested only in the list of sockets which are ready to be read, we use only a single select input parameter.

TCP/IP Chat Server: tcpchatserver.py
#
# tcpchatserver.py
# TCP/IP Chat server
# Author: S.Prasanna
# The server accepts connection from multiple clients and
# broadcasts data sent by a client to all other clients
# which are online (connection active with server)
#

import socket
import select
import string

def broadcast_data (sock, message):
"""Send broadcast message to all clients other than the
server socket and the client socket from which the data is received."""

for socket in CONNECTION_LIST:
if socket != server_socket and socket != sock:
socket.send(message)

if __name__ == "__main__":

# List to keep track of socket descriptors
CONNECTION_LIST=[]
RECV_BUFFER=4096 # Advisable to keep it as an exponent of 2

# Do basic steps for server like create, bind and listening on the socket

server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(("127.0.0.1", 5000))
server_socket.listen(10)

# Add server socket to the list of readable connections
CONNECTION_LIST.append(server_socket)

print "TCP/IP Chat server process started."

while 1:
# Get the list sockets which are ready to be read through select
read_sockets,write_sockets,error_sockets = select.select(CONNECTION_LIST,[],[])

for sock in read_sockets:

if sock == server_socket:
# Handle the case in which there is a new connection recieved
# through server_socket
sockfd, addr = server_socket.accept()
CONNECTION_LIST.append(sockfd)
print "Client (%s, %s) connected" % addr
broadcast_data(sockfd, "Client (%s, %s) connected" % addr)

else:
# Data recieved from client, process it
try:
#In Windows, sometimes when a TCP program closes abruptly,
# a "Connection reset by peer" exception will be thrown
data = sock.recv(RECV_BUFFER)
except:
broadcast_data(sock, "Client (%s, %s) is offline" % addr)
print "Client (%s, %s) is offline" % addr
sock.close()
CONNECTION_LIST.remove(sock)
continue

if data:
# The client sends some valid data, process it
if data == "q" or data == "Q":
broadcast_data(sock, "Client (%s, %s) quits" % addr)
print "Client (%s, %s) quits" % addr
sock.close()
CONNECTION_LIST.remove(sock)
else:
broadcast_data(sock, data)

server_socket.close()

TCP/IP Chat Client: tcpchatclient.py
#
# tcpchatclient.py
# TCP/IP Chat client
# Author: S.Prasanna
# The client program connects to server and sends data to other connected
# clients through the server
#

import socket
import thread
import sys

HOST = '127.0.0.1' # The remote host
PORT = 5000 # The same port as used by the server
RECV_BUFFER=4096

def recv_data():
"Receive data from other clients connected to server"
while 1:
try:
recv_data = client_socket.recv(RECV_BUFFER)
except:
#Handle the case when server process terminates
print "Server closed connection, thread exiting."
thread.interrupt_main()
break
if not recv_data:
# Recv with no data, server closed connection
print "Server closed connection, thread exiting."
thread.interrupt_main()
break
else:
print "Received data: ", recv_data

def send_data():
"Send data from other clients connected to server"
while 1:
send_data = str(raw_input("Enter data to send (q or Q to quit):"))
if send_data == "q" or send_data == "Q":
client_socket.send(send_data)
thread.interrupt_main()
break
else:
client_socket.send(send_data)

if __name__ == "__main__":

print "*******TCP/IP Chat client program********"
print "Connecting to server at 127.0.0.1:5000"

client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.connect((HOST, PORT))

print "Connected to server at 127.0.0.1:5000"

thread.start_new_thread(recv_data,())
thread.start_new_thread(send_data,())

try:
while 1:
continue
except:
print "Client program quits...."
client_socket.close()
The next time we will see how to handle asynchronous I/O using non-blocking sockets.

NFS Troubleshooting: RPC Error Program not registered possible solution

Sunday, July 20, 2008

Today when I was trying to mount an NFS shared folder from a solaris box on a linux host, I got the below error message (assume that the linux and solaris hosts are linux-host and solaris-host rspectively)


mount to NFS server 'hostname' failed: RPC Error: Program not registered


The directory /opt/web-apps from the solaris-host is to be mounted on /mount-app in linux-host.

bash-3.00# mount solaris-host:/opt/web-apps /mount-app
mount: mount to NFS server 'solaris-host' failed: RPC Error: Program not registered.

The Solaris host seems to be missing some entries in /etc/dfs/dfstab as shown below (In Solaris, the /etc/dfs/dfstab has NFS mount related details)


bash-3.00# cat /etc/dfs/dfstab

# Place share(1M) commands here for automatic execution
# on entering init state 3.
#
# Issue the command 'svcadm enable network/nfs/server' to
# run the NFS daemon processes and the share commands, after adding
# the very first entry to this file.
#
# share [-F fstype] [ -o options] [-d ""] [resource]
# .e.g,
# share -F nfs -o rw=engineering -d "home dirs" /export/home2

I inferred that if dfstab has no entries, you may get an RPC Error as mentioned above, that being said, now I NFS share the above resource by adding the below line in

/etc/dfs/dfstab on solaris-host

share -F nfs -o rw -d "Webapp dir" /opt/web-apps

Where

-F is the filesystem type,
-o is to specify access control, which is rw (read-write) for the above command,
-d specify description of the shared resource followed by the resource to be shared through NFS.

For more help on share command, visit the man page here.

One can also use the share command as shown below

bash-3.00# share -F nfs -o rw -d "Webapp dir" /opt/web-apps

But then the above command won't persist reboots, therefore its a better option to edit the /etc/dfs/dfstab and add the resource to be shared if it needs to persist

reboots.

Then, the nfs service needs to be restarted.

bash-3.00# /etc/init.d/nfs.server stop
bash-3.00# /etc/init.d/nfs.server start

Now the mount will succeed

bash-3.00# mount solaris-host:/opt/web-apps /mount-app
bash-3.00#

To put it in a nutshell, I observed the following behaviour wrt NFS, hope these will be useful for troubleshooting NFS.

1. If /etc/dfs/dfstab has no entries and you try to mount a directory from the remote system through NFS, an RPC Error: Program not registered error will be thrown.

2. If an entry is added in /etc/dfs/dfstab to share a resource and you try mounting another directory in the remote system which is not shared, you will get a "Permission denied error" as shown below.

For example if /opt/web-apps is shared through NFS in the solaris-host using

share -F nfs -o rw -d "Webapp dir" /opt/web-apps

trying to mount /var in the target system would result in the following.

bash-3.00# mount solaris-host:/var /mount-app
mount: solaris-host:/var failed, reason given by server: Permission denied

Therefore RPC Error will be thrown ONLY when /etc/dfs/dfstab has no entries.

4. If mount needs to persist reboots on client side (from linux-host), add the following entry in /etc/fstab on the linux host.

solaris-host:/opt/web-apps/ /mount-app nfs nouser,auto,rw 0 0

where

solaris-host:/opt/web-apps - The mounted directory
/mount-app - Mount point
nfs - Type of file system
nouser - Can be mounted only by root and not any user
auto - Mounted at boot time
rw - Mounted directory has read-write permissions
The number in the fifth column if for file system backups, 0 means ignore backups
The number in the sixth coulmn is for fsck option to check the file system, again 0 means such checks are ignored.

After adding the above entries, make sure you restart NFS using

bash-3.00# service nfs restart

Now the NFS mount between linux client and the solaris will persist reboots.

5. For the sake of completeness, from a solaris client you need to add the following in /etc/vfstab for persistent mounts.

The contents of the initial lines in /etc/vfstab on a Solaris 10 would be like the following

bash-3.00# cat /etc/vfstab

#device device mount FS fsck mount mount
#to mount to fsck point type pass at boot options

The comment says it all about vfstab entries, so for a mount to persist a solaris client, the entry would be

solaris-host:/opt/web-apps/ - /mount-app nfs - yes rw

Customizing blogger post URL

Saturday, July 12, 2008

Customizing blogger post URL is another important Search Engine Optimization (SEO) technique I learned recently. URLs play an equally important role as blog titles in bringing you relevant ads for the content. In blogger though the URL length is fixed, one can still customize the post URL so as to get relevant ads for the content. The more specific the URL, the more relevant the ads will be.

Its common for someone using blogger to focus on an appropriate title for the blog, post contents and let URL take care of itself. Though blogger appends the fields of your title with '-' in deriving the URL and somehow removes filler words and articles to make your URL relevant, I noted some minor pitfalls due to blogger URL length restrictions which would ignore some important keywords in the post title.

For example if the title of your blog is "Search Engine Optimization".

the blogger post URL would be blogname.blogspot.com/search-engine-optimization.

Well and good, the URL is relevant to the title.

But if you have a title say "Advanced server side programming in java", can you guess what the URL would be, is it

blogname.blogspot.com/advanced-server-side-programming-in-java.html

WRONG.

Due to URL size limits, the title would be different. The following were the URLs I got when I had recreated the same post twice, note that the important keyword "java" is missed.

blogname.blogspot.com/advanced-server-side-programming-in.html
blogname.blogspot.com/advanced-server-side-programming-in_10.html

Also its a common practice to have attractive titles like "Top ten ways to improve your search engine rankings" to attract visitors, again as you guess the blogger URL for the above URL would be something like this.

http://blogname.blogspot.com/top-ten-ways-to-improve-your-search_12.html

missing the important keywords. If the length of the title exceeds the URL size restrictions, blogger would ignore the later part of your title and sometimes substitute random numbers to fill the URL part till its maximum limit is reached

Though this may not drastically affect the relevance of your ads, but why miss an important keyword in the URL.

Here are some steps I would recommend you to follow before posting your content in order to have a relevant URL which won't be missing your keywords.

1. As obvious, once you post an entry and modify its title any number of times, the blogger URL for that post would be the same as that derived from your first title. Therefore before you name your title for the blog, make sure you have a title which is URL friendly, post the blog, edit it, replace the URL customized title with your original title, as length restriction applies only for URL and not for title.

2. Always have important keywords in the beginning of the URL, chances are that longer the URL with keywords at the end, it may be ignored, also avoid using articles and filler words in the URL.

For example for the above title "Advanced Server side programming in java", before posting the content, I would do the following.

2.1 Make sure that all important keywords are preserved, the word "Advanced" in the above title doesn't have much relevance compared to other words, therefore the title for optimizing URL would be

"Server side programming in java"

then post the blog, make sure I get all keywords in the URL or else the post need to be deleted and the title needs to be reworked so that the important keywords won't be missed in the URL.

Similarly for the title "Top ten ways to improve your search engine rankings", the keywords which should have more weightage would be "Search engine rankings" than the first three words, which anyway the title takes care of.

2.2. Edit the blog again, replace the URL customized title with the original one, note that the URL will not change this time.

3. Another way to ensure that your URL remains intact (the way you want to be) is to create it by yourself in the title field the way blogger would do it for you otherwise, like "server-side-programming-in-java", "improving-search-engine-rankings" respectively for the above titles, in this way some articles and filler words may not be removed if you desire, but keep in mind the URL length restrictions.

Integrating Apache with Tomcat Weblogic and Websphere Application Server

Friday, July 4, 2008

Integrating Apache with Application Servers

S.Prasanna,
sprasanna199@gmail.com

This article discusses the procedure for Integrating Apache (2 and above) with three Application servers,namely Apache Tomcat (Tomcat 5.x), BEA Weblogic server (8.1) and IBM Websphere Application server 5.x.

1. Web Servers and Application Servers

Before proceeding, one needs to understand the definition and the difference between a Web Server and an Application Server and why they are integrated.

A Web Server is a HTTP server program, which accepts browser requests from the client and sends the response back to the client. In a three-tier architecture, the Web server receives the HTTP requests and passes it to the application server program,which services the request. A Web server understands only the HTTP protocol.Examples of commonly used Web servers include Apache, Zeus, IIS, Sun ONE, etc.

Figure 1: Three tier architecture

The application server is the one, which services the client requests. Typically the Application server provides a platform for deploying business applications across the web. The application server contains the code that implements the functionality of the application (Business logic). Most of the application servers are Java based, example Tomcat, Bea Web logic, IBM Websphere, etc. The application server serves dynamic contents like servlets, JSPs, etc. In short, the application server provides access to business logic for the client.

Advantages of Web Server Application Server Integration:

Improved Performance:

Since the Web server and the application server are logically separated (sometimes they will be in a different box too), the Web Server can serve all static pages (HTML, etc) while delegating the application server to handle other dynamic content which improves performance.

Load Balancing:

It is possible for a Web server to load balance requests to one or more application servers.

Security:

Since the Web server is the one, which is exposed to the real world, and since its service levels are less compared to an application server, the web server can be hardened to prevent attacks. The Web server can also act as a reverse proxy in some cases. A reverse proxy is one, which stands between an application server and many clients. A Web server in this way can act as an application level gateway, monitoring all traffic from the clients, thus preventing HTTP level attacks.(See www.modsecurity.org for details).


2. Procedure for Integrating Apache with Application servers

Note:

From here, APACHE_HOME will be used as a reference for the root directory of Apache HTTP server installation.

This section discusses integrating apache with three Application servers namely Tomcat, BEA Weblogic and IBM Web sphere Application server.

2.1 Integrating Apache 2.x with Tomcat 5.x

Steps for integrating Apache with Tomcat using mod_jk2:

1.Download the mod_jk2 module from http://jakarta.apache.org

From Apache side:

1. Copy the module file mod_jk2.so file in the APACHE_HOME/modules directory

2. Add these two lines in the httpd.conf file in the APACHE_HOME/conf directory

LoadModule jk2_module modules/mod_jk2.so
JkSet config.file conf/workers2.properties

3. Create a file workers2.properties in the APACHE_HOME/conf directory and add these lines.

# Example socket channel, override port and host.

[channel.socket:localhost:8009]
port=8009
host=127.0.0.1

# define the worker
[ajp13:localhost:8009]
channel=channel.socket:localhost:8009

# Map the default Tomcat directory to the Web server uri space
[uri:/*]
group=ajp13:localhost:8009

From Tomcat side:

1. Goto TOMCAT_HOME/conf directory and add these lines in the jk2.properties file

handler.list=apr,request,channelJni
channelSocket.port=8009

channelJni.disabled = 0
apr.jniModeSo=inprocess

(Just Uncomment these lines in the jk2.properties file,make sure that port number is 8009)

Now if you type

http://127.0.0.1/, you will get the default Tomcat Homepage (http://127.0.0.1:8080).

Steps for Apache with Tomcat using mod_proxy:

From Apache side:

Edit the httpd.conf file in the <APACHE_HOME>/conf directory

Add the following lines:

LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_ftp_module modules/mod_proxy_ftp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so

#Here Apache and Tomcat are running in the same machine.
#Change the IP address to x.y.com if Tomcat is running in a
#different box

NameVirtualHost 127.0.0.1:80

#Here Apache is configured as a proxy server

<VirtualHost 127.0.0.1:80>
ProxyPass / http://127.0.0.1:8082/
ProxyPassReverse / http://127.0.0.1:8082/
</VirtualHost>

#Note: Port 8082 in Tomcat 5 recieves the proxied requests from Apache

From Tomcat side:

Uncomment these lines in the <TOMCAT_HOME>/conf/server.xml if commented (where TOMCAT_HOME is the TOMCAT installation directory.)

<!-- Define a Proxied HTTP/1.1 Connector on port 8082 -->
<!-- See proxy documentation for more information about using this. -->

<Connector port="8082"
maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
enableLookups="false"
acceptCount="100" debug="0" connectionTimeout="20000"
proxyPort="80" disableUploadTimeout="true" />


2.2 Integrating Apache 2.x with BEA Weblogic 8.1

1. Goto WL_HOME\Server\bin and locate the file (where WL_HOME is the Weblogic installation directory)

mod_wl_20.so (Regular Strength encryption)
mod_wl28_20.so (128 bit encryption)

2. Goto APACHE_HOME\modules directory and copy the above file(s)

3. The Apache HTTP Server Plug-In will be installed in your Apache HTTP Server Installation as an Apache Dynamic Shared Object (DSO) which is based on mod_so.c. Make sure it is enabled (By default it should be enabled) by executing the command below.

APACHE_HOME\bin\apache -l (You should see mod_so.c in the output).

4.Edit the Apache httpd.conf file in APACHE_HOME/conf directory and add these lines

#Weblogic configuration details

LoadModule weblogic_module modules\mod_wl_20.so

#Location tag is added only for proxy support by path
#It will take precedence when proxying by MIME type
#is also enabled

<Location /weblogic>
SetHandler weblogic-handler
PathTrim /weblogic
#ErrorPage http://myerrorpage.mydomain.com
</Location>

#Proxying requests by MIME type

<IfModule mod_weblogic.c>
WebLogicHost 127.0.0.1
WebLogicPort 7001
MatchExpression *
</IfModule>

5. Test the syntax of the httpd.conf file using the command

APACHE_HOME\bin\apache -t (You should see SYNTAX OK as output).

6. Restart Weblogic and Apache.

Now if you type

http://127.0.0.1, it should lead to the Weblogic home page (provided the <Location> definition was not there, else it will take precedence and instead of http://127.0.0.1 you have to type http://127.0.0.1/weblogic/ to see the Weblogic Home Page).

Steps for Apache with Web logic using mod_proxy:

From Apache side:

Edit the httpd.conf file in the <APACHE_HOME>/conf directory

Add the following lines:

LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_ftp_module modules/mod_proxy_ftp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so

#Here Apache and Web logic are running in the same machine.
#Change the IP address to x.y.com if Weblogic is running in a
#different box

NameVirtualHost 127.0.0.1:80

#Here Apache is configured as a proxy server

<VirtualHost 127.0.0.1:80>
ProxyPass / http://127.0.0.1:7001/
ProxyPassReverse / http://127.0.0.1:7001/
</VirtualHost>

#The default listening port for BEA Web logic application server is 7001

Now if you type

http://127.0.0.1 it should lead to the Weblogic home page.


2.3 Integrating Apache 2.x with IBM Web sphere 5.1 x and above

1. Download the Web server plugins for Websphere 5.x from

http://www-1.ibm.com/support/docview.wss?uid=swg24007227 for (5.1x)

http://www-1.ibm.com/support/docview.wss?uid=swg24007265 for (5.0x)

2. Get the file mod_was_ap20_http.dll from the plugin and copy it in

Apache modules directory i.e <APACHE_HOME>/modules directory.

3. Get the plugin-cfg.xml (from Websphere) file

This can be got by

1. Running the command GenPluginCfg in the

<WAS_ROOT>\bin directory (GenPluginCfg.bat)
(where WAS_ROOT is the Websphere Application Server installation directory)

The Plugin-cfg.xml file will be created in the location

<WAS_ROOT>\config\cells\plugin-cfg.xml

2. Running Websphere admin console and exporting the plugin.

Login to the admin console (at port 9090 i.e 127.0.0.1:9090) (This doesn’t require a user ID or a password)

Goto Environment -> Update Web Server Plugin

Click OK to update the plugin configuration file.

The Plugin-cfg.xml file will be created in the location below

<WAS_ROOT>\config\cells\plugin-cfg.xml

4. Copy the plugin-cfg.xml file in Apache modules directory
i.e <APACHE_HOME>/modules directory.

5. Add these lines in the httpd.conf file in the <APACHE_HOME>/conf directory.

LoadModule was_ap20_module modules/mod_was_ap20_http.dll
WebSpherePluginConfig modules/plugin-cfg.xml

6. You can change the location of the log file generated by the plugin by editing the plugin-cfg.xml file. For example, if the Plugin
error log is to be generated in the apache logs directory, the plugin-cfg.xml file can be edited as follows.

<Log LogLevel="Error" Name= "<APACHE_HOME>\logs\http_plugin.log"/>

A simplified form of the plugin-cfg.xml file is shown below.

<Config>
<Log LogLevel="Error" Name="<APACHE_HOME>\logs\http_plugin.log"/>

<VirtualHostGroup Name="default_host">
<VirtualHost Name="*:80"/>
</VirtualHostGroup>

<ServerCluster Name="MyCluster">
<Server Name="server1">
<Transport Hostname="Server-host" Port="9080" Protocol="http"/>
</Server>
</ServerCluster>

<UriGroup Name="MyURIs">
<Uri Name="/snoop/*"/>
<Uri Name="/*"/>
</UriGroup>

<Route ServerCluster="MyCluster" UriGroup="MyURIs" VirtualHostGroup="default_host"/>
</Config>

7.Now restart Websphere and Apache server and type

http://127.0.0.1/snoop and you can see the Websphere's snoop servlet in the browser.

Steps for Apache with IBM Web sphere using mod_proxy:

From Apache side:

Edit the httpd.conf file in the <APACHE_HOME>/conf directory

Add the following lines:

LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_ftp_module modules/mod_proxy_ftp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so

#Here Apache and Web logic are running in the same machine.
#Change the IP address to x.y.com if Websphere is running in a
#different box

NameVirtualHost 127.0.0.1:80

#Here Apache is configured as a proxy server

<VirtualHost 127.0.0.1:80>
ProxyPass / http://127.0.0.1:9080/
ProxyPassReverse / http://127.0.0.1:9080/
</VirtualHost>

#The default listening port for IBM Websphere application server is 9080
#If you want to proxy requests for admin console, use 9090 instead.

Now restart the Apache server and type

http://127.0.0.1/snoop

and you can see the Websphere's snoop servlet in the browser.

Integrating IIS with Tomcat

Steps for Integrating IIS (5 and above) with Tomcat (5 and above)

S.Prasanna,
sprasanna199@gmail.com

Introduction:

This article explains the way to integrate IIS (Version 5 and above) with Tomcat (Version 5 and above). Since IIS cannot serve dynamic java contents like servlets and JSPs, the IIS Tomcat connector delegates Tomcat servlet engine to handle servlet and JSP calls. I still couldn't find a good document which clearly explains how to use the install4iis.js (A javascript installer used to configure the IIS Tomcat connector) to make the installation process much simpler. So here's mine.


1. Download the Tomcat IIS Connector (Jk2 binaries) from one of the mirrors

(http://archive.apache.org/dist/tomcat/tomcat-connectors/jk2/binaries/win32/jakarta-tomcat-connectors-jk2.0.4-win32-IIS.zip)

2. Unzip the connector. There will be three directories namely bin, conf and doc. The bin folder has the isapi_redirector2.dll and an installation javascript file install4iis.js. I call the directory where you unzipped the IIS connector zip file as <IIS_CONNECTOR_HOME>.

3. From IIS perspective I assume that the default web site is running and you need to redirect all URIs related to Servlets and jsps for Tomcat to execute them i.e URIs of the form

http://127.0.0.1/servlets-examples and http://127.0.0.1/jsp-examples to Tomcat servlet container URI
http://127.0.0.1:8080/jsp-examples and http://127.0.0.1:8080/jsp-examples respectively.

These are the steps to be executed from Tomcat and IIS.

From Tomcat:

I assume that you installed Tomcat properly and have a running version of Tomcat at port 8080 with jk2 at 8009. Since these are the default settings in Tomcat 5, there is virtually no need to do anything from Tomcat side once you have a running version of Tomcat.

From IIS:

1. Since the default web site is running, invoking the install4iis.js

(<IIS_CONNECTOR_HOME>/bin directory) will be straightforward.

2. Configuring the jakarta filter:

To do this one should create a new virtual directory from the IIS management console.

1. Goto Settings -> Control Panel -> Administrative tools -> Internet Information Services

2. Expand the Local Computer icon.

3. Expand Web sites folder and Default Website.

4. Under the Default Web site create a new virtual directory.

1. Right click Default Web site -> Click New -> Virtual Directory -> Next

2. In the (Alias) Text field enter the name as jakarta and Click next

3. Enter <IIS_CONNECTOR_HOME>/bin as the directory path and Click Next

4. Select read, run scripts and execute access permissions, Click Next and Finish.

5. Configure workers2.properties file:

This file is used to configure the connector properties, the URI pattern to be filtered, and details of the IP address and the port where the Tomcat connector is listening, etc.To map URIs of the form /Servlets-examples and /jsp-examples for Tomcat, the following settings need to be specified in the workers2.properties file.

Create a file workers2.properties in <IIS_CONNECTOR_HOME>/conf directory with the following settings.

# Example socket channel, override port and host.

[channel.socket:localhost:8009]

port=8009

host=127.0.0.1

# define the worker

[ajp13:localhost:8009]

channel=channel.socket:localhost:8009

# Map the default Tomcat directory to the Web server uri space

[uri:/jsp-examples/*]

group=ajp13:localhost:8009

[uri:/servlets-examples/*]

group=ajp13:localhost:8009

6. Now you are set to execute the IIS Tomcat connector installation script install4iis.js. Either double click install4iis.js or execute it in command line as windows script using

Wscript //E:Jscript install4iis.js

The installation process using install4iis.js will take 10 – 15 steps. Click Ok for all messages in the message box one
after another and finally check for the message

Filter [jakarta] set. Click Ok to complete the installation.

Note: By Default the install4iis.js configures the filter (jakarta, the virtual directory configured above) for the Default
Web Site. If you want to configure the filter for other sites, you need to use the appropriate command line options.
For more help,
type Wscript //E:Jscript install4iis.js –h.

7. Restart IIS (In IIS Management console, right click Local Computer -> All tasks -> Restart IIS).

8. To check the configuration of the filter, in the IIS Management console, expand Local Computer -> Web Sites and Right click Default Web Sites and Click Properties and ISAPI Filters tab. There you should see the green status of the jakarta filter indicating that the filter is loaded.

Once the above procedures are completed correctly, then typing

http://127.0.0.1/servlets-examples in browser will fetch the Tomcat page http://127.0.0.1:8080/servlets-examples and http://127.0.0.1/jsp-examples will fetch http://127.0.0.1:8080/jsp-examples page. Try executing the sample Servlets and jsp examples through IIS Connector and check the results.

Socket Programming tutorial

Generic Socket Programming tutorial

S.Prasanna
sprasanna199@gmail.com

This is a generic socket programming tutorial which mainly concentrates on how to communicate with two machines using TCP/IP and UDP/IP protocols irrespective of the programming language used to implement the server and client

I will list a simple Connection oriented (TCP) and Connectionless (UDP) Server and Client programs for all the four languages namely Python , Perl, C and Java. The TCP Server program is an iterative one which means it will process clients one by one. The communication between the TCP Server and the Client is a simple chat program. If the client quits, the current TCP connection between the server and the client gets disconnected and the server processes the next client and if the server decides to quit, that client gets disconnected and the server processes the next client.

The UDP Server program is very simple. The server recieves datagrams from many clients and prints them.

In all the programs , the Server process listens at port 5000 and I have used loopback address for the Server so that you can run the server and the client in different terminals in the same system. Besides you can use any of the four servers implemented in Python , Perl, C and Java with any of the four client programs implemented inthe four languages.In short you can use a Client program in Perl with a Server program in python , Server program in python with a Client progam in C , etc. Morever you can invoke the client program without any command line arguments since the client and the server are designed to communicate using the loopback address.

I will go from easy to difficult ( in terms of readability and lines of code ). i.e for example socket programming is simple and easy to understand in Python than in C or java.


1. Socket programming in Python

Connection oriented Server and Client program.

tcpserver.py and tcpclient.py

Connectionless Server and Client program.

udpserver.py and udpclient.py


2. Socket programming in Perl

Connection oriented Server and Client program.

tcpserver.pl and tcpclient.pl

Connectionless Server and Client program.

udpserver.pl and udpclient.pl


3. Socket programming in C

Connection oriented Server and Client program.

tcpserver.c and tcpclient.c

Connectionless Server and Client program.

udpserver.c and udpclient.c


4. Socket programming in Java

Connection oriented Server and Client program.

TCPServer.java and TCPClient.java

Connectionless Server and Client program.

UDPServer.java and UDPClient.java

I will list similar programs in as many languages in future. I could have explained about the details of sockets, how system calls like socket , bind, listen, etc works, but those should be explained from the perspectives different languages which is beyond the scope of this tutorial. In general these programs explain the fundamental concepts of socket programming irrespective of their implementation language.

Socket programming can be easily understood in interpreted languages like Perl and Python. Perl, especially is weapon for hackers. Dangerous scripts can be developed in minutes, in perl. For example spoofing source IP address can easliy be done in Perl but it requires some effort in C or in other languages. This small script shows how to spoof UDP source address.

Raw Sockets Programming (UDP Spoofing):

udpspoof.pl and udpserver.c

When you run the above simple script as client you can see the fake IP source address and the port the client used at the UDPServer. To execute the Perl script you need the Net::RawIP Module which can be downloaded here and also its documentation.

TCP/IP spoofing is a challenging task and involves a lot of calculation involving prediction of sequence numbers,etc but worth giving a try.

References:

1. Beej's guide to Network Programming

The highly rated Socket Programming tutorial in the Web. It explains all system calls used for socket programming in a clear way and also about Networking concepts in general.

2. UNIX Sockets for Newbies

Another Wonderful Socket programming tutorial.

3. Network Programming in Python

An excellent document on Python Socket programming.

4. UNIX Sockets FAQ

Answers to common questions asked by socket programmers.

5. Perl Socket Programming tutorial

Good introduction to Perl sockets.

6. http://pont.net/socket/index.html

A Collection of different Client-Server programs in C and Java.

Hacking and Computer Security related sites and links

There are perhaps hundreds and thousands (and more) of good Hacking and Security sites all over the world. The below list may not be complete, but one can get interesting articles and references from these sites.

1. www.hackersplayground.org

Go and play there.

2. www.binarii.com

Subscribe in this group and you will get the latest security related incidents, Security related books, white papers,etc.

3. www.deter.com/unix

Some useful codings (in Unix and Linux Security), research papers ,etc.

4. www.deathstar.ch/security

This is a site which couldn't be easily found in search. You can find a lot of information related to cryptography, E-mail security, etc.

5. www.linuxsecurity.org

Best site to learn about Linux Security.

6. www.securityfocus.com

Lots of Information related to Penetration testing, latest Vulneribility reports, etc.

7. www.packetstormsecurity.org

Good readings in all the fields of security ( like IDS , Cryptography, Firewalls, etc).

8. www.insecure.org

Open Source penetration testing tools like tools like nmap, various exploit code , etc.

9. www.research.att.com/~smb

Steven M. Bellovin's research papers.

10. www.astalavista.com

A Complete Security site.

Hacking tips for Beginners

This writing can be regarded as a small tip to become a hacker. Note that the term hacker is misunderstood by different people. A hacker can be a programming hacker, Internet hacker, etc and this writing can be applied to anything. The first and foremost thing in becoming a hacker is choosing the right and interesting problems to solve (This depends on one's interest). The second thing is doing things differently or trying to solve a problem in a different way. This develops the ability to research and increases our curiosity towards a particular thing.

Next is developing skill sets required to become a hacker. This needs time, hard work and dedication. Most of us (including me) are content with just a few high level languages. To gain a firm understanding of "How Computers really work", learning assembly language is a must (If I had known this in my childhood I would have chosen it as the first language!). There is an excellent book which answers the above question (Assembly Language Step by Step: Programming in DOS and Linux by Jeff Duntemann). I think it's a book which should be in every computer programmer's bookshelf. Moreover there are numerous books and documentation available for every software which makes learning new things easier. The third is publishing interesting articles or manuals which may be of interest to other hackers. I feel that doing these things consistently would make one a hacker in his profession.

The below things may be of interest to Network hackers.

If one is new to networking, then a good understanding of sockets is a must to start things.Though there a lot of Network progamming tutorials available in the Web , most of them concentrate on only a single programming language. I've written a Generic Socket programming tutorial in four programming languages namely Python , Perl, C and Java which mainly concentrates on how to communicate with two machines using TCP/IP and UDP/IP protocols irrespective of the programming language used to implement the server and client and also a very simple example of using raw sockets in Perl which will definitely be useful.

The other thing is learning about basic exploits (an introductory article can be found here). There is a wonderful tutorial on the basics of Linux Hackingavailable here. There are many useful references available in the web. Only learning and experimenting things would make one a hacker.

I have listed some Hacking and Security related sites, which may definitely be a good reference.


Copyright © 2016 Prasanna Seshadri, www.prasannatech.net, All Rights Reserved.
No part of the content or this site may be reproduced without prior written permission of the author.