Share

Thursday, December 31, 2015

Websphere Application Server heap size monitoring with wsadmin

In Middleware infrastructure, These two issues are very common.

1. Connection Pool Overflow
 2. OutOfMemory(OOM)

Both of them will lead the application to the rock bottom and application unavailability, So its always recommended to have on eye on both of these key factors, by having some kind of Run time Monitoring and alerting system tools. (Tivoli, Wily, Nagios etc..)

Sometime, We will not be having the sufficient monitoring tool to capture this OutOfMemory and Connection Pool Overflow.

That's when, we fall back into the efficient and easiest way,  which is scripting tools,  like WLST and WSADMIN etc.

Hope you have read our another post about "Connection Pool Overflow monitoring and Alerting System" if not click here

Now, here is the Jython script designed to help monitor the "HeapSize"  which can let us able to proactively measure and take action for OOM issue.

As mentioned earlier in our post, You can have some kind of scheduling tools like Autosys (or) Cron and run it in a frequent interval along with Shell/Perl script and generate an email (or) ticket.

Let me know if there is any help required in designing of Shell/perl (or) in the scheduling part.

Latest version of this code can be found here

TheCode
Running_JVMS=AdminControl.queryNames("*:type=Server,*").split(java.lang.System.getProperty("line.separator"))
ignorelist=['nodeagent','dmgr']
for JVM in Running_JVMS:
        ServerName=AdminControl.invoke(JVM ,"getName()")
        if ServerName not in ignorelist:
                JVMName=AdminControl.completeObjectName('type=JVM,process='+ServerName+',*')
                JVMObject=AdminControl.makeObjectName(JVMName)
                perf=AdminControl.completeObjectName('type=Perf,process='+ServerName+',*')
                perfObject=AdminControl.makeObjectName(perf)
                Obj=AdminControl.invoke_jmx(perfObject,"getStatsObject",[JVMObject,java.lang.Boolean('false')],['javax.management.ObjectName','java.lang.Boolean'
])
                current=Obj.getStatistic('HeapSize').getCurrent()
                used=Obj.getStatistic('UsedMemory').getCount()
                usage=float(used)/float(current)*100
                uptime=float(Obj.getStatistic('UpTime').getCount())/60/60/24
                print "--------------------------------------------"
                print "ServerName      :", ServerName
                print "uptime(in days) :", int(uptime)
                print "--------------------------------------------"
                print "CurrentUsage    :", current
                print "Usedmemory      :", used
                print "Usage in Percent:", int(usage)
                print "--------------------------------------------"



Save this with .py extention and run it with wsadmin

The Result
bash-3.00# ./wsadmin.sh -username wasadmin -password wasadmin -lang jython -f ./python_scripts/getHeapStatus.py
WASX7209I: Connected to process "dmgr" on node mwiCellManager01 using SOAP connector;  The type of process is: DeploymentManager
--------------------------------------------
ServerName      : mwi_server01
uptime(in days) : 13
--------------------------------------------
CurrentUsage    : 119936
Usedmemory      : 85675
Usage in Percent: 71
--------------------------------------------
--------------------------------------------
ServerName      : mwi_server02
uptime(in days) : 50
--------------------------------------------
CurrentUsage    : 483392
Usedmemory      : 319485
Usage in Percent: 66
--------------------------------------------



Hope it helps.

For any question related to this. Please feel free to reach us via comment.
Always you can write an email to us at admin@mwinventory.in

Cheers and Have a great and prosperous new year. Best wishes from teammwi.


Wednesday, December 30, 2015

Apache Tomcat Web Application Development with Form Based Authentication

Designing WEB Applications with Web Browser based security is very common.

There are three types of Authentication methods generally being used such as
  1. Basic Authentication
  2. Form Based Authentication
  3. Client Cert Authentication
Most of the Web Applications are using Form Based Authentication method where  the user will be
submitting their credentials through the html form. Which will be further transmitted to the server over HTTP(SSL) as an additional security measure.


Now we are going to see how to develop a web application with Form Based Authentication.


Prerequisites

  • Apache Tomcat Server 6 (or) above
  • Editors like Notepad ++




Step1:  Create one directory named "TestSecApp"

Step2:  Under "TestSecApp" Create another directory named "WEB-INF"

Step3:  Copy All these files with their respective name under "TestSecApp" Directory

error.html


<html>
 <head><title>ERROR PAGE 404</title></head>
 <body>
<H1> CUSTOMER ERROR MESSAGE: THE PAGE YOU HAVE REQUESTED IS NOT FOUND </H1>
 </body>
</html>

fail_login.jsp
<html>

  <head>
  <link rel="stylesheet" href="style1.css" type="text/css" media="all">
    <title>Login failed</title>
  </head>
  <body bgcolor=#ffffff>
  <div id="login">
  
  <center><h2>Authentication Failure.</h2>
  
  <a href="login.jsp">Return to login page</a> </center>
  </div>
  
  </body>
</html>

login.jsp
<html>
  <head>
  <link rel="stylesheet" href="style1.css" type="text/css" media="all">
    <title>Security WebApp login page</title>
  </head>
  <body bgcolor="white"> 
  <div id="login">
  <blockquote>
  
  <h2>Please enter your user name and password:</h2>
  <p>
  <form method="POST" action="j_security_check">
    <div id="box1">  
      
      <p>Username:<input id="textbox1"  type="text" name="j_username"> </p>
      <p>Password:<td><input id="textbox1" type="password" name="j_password"> </p>
   <br>
   <input id="button1" type=submit value="Submit">
    </div>
  </form>
  </blockquote>
  </div>
  </body>
</html>


logout.jsp
<html>

  <head>
  <link rel="stylesheet" href="style1.css" type="text/css" media="all">
    <title>Welcome Back</title>
 </head>
 <body bgcolor=#ffffff>
  <div id="login">
  <center>
  <br>
  <br>
<%@ page session="true"%>

User '<%=request.getRemoteUser()%>' has been logged out.

<% session.invalidate(); %>
<br>
<a href="login.jsp">Click here to login again</a>
</center>
  </div>
  
  </body>
</html>

Test.jsp
<!DOCTYPE html>
<html>
 <head>
   <title>Browser Based Authentication Example Welcome Page</title> 
   <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script>

   <script>
   $(document).ready(function(){
        
        $("#div1").animate({left:"200px", top: "200px" , right:"200px", bottom: "200px"});
});
</script>
 
 <style>
#div1
{
   position: absolute;
   left : 0px;
   background-color: #68A3B3;
   border: 2px silver solid;
   color: white;
   text-align: center;
   
  
}
</style>

 </head>  
<div id="div1">
<h1> Browser Based Authentication Example Welcome Page </h1>  

<h1> <p> Welcome <%= request.getRemoteUser() %>!  </h1>

<a href="logout.jsp"> Logout </a>

</div>

</body>

</html>

style1.css
#login
 {
   position: absolute;
   left : 400px;
   right: 400px;
   top: 200px;
   bottom: 200px;
   border: 2px silver solid;
   color: #389093;
 }
 h2
 {
 color: 389093;
 
 }
 
 #message
 {
  color: violet;
  
 }
 #box1
 {
   padding: 5px;
 }
 #textbox1
 {
   position: absolute;
   left: 140px;
  padding: 5px;
  width: 200px;
 }
 
 #textbox1:hover
 {
   position: absolute;
   left: 140px;
   padding: 5px;
   width: 200px;
   border-color: violet;
   color: black;
 }
 
#button1
 {
  background-color: green;
  color: silver;
  font-weight: bold; 
 }
 
#button1:hover
 {
  background-color: green;
  color: white;
  font-weight: bold; 
 }

Step4: Under "WEB-INF/classes" directory copy the below file

This  servlet is to print the session creation in the tomcat STDOUT. check the catalina.log once deployed to verify.

MySessionListener.java


import javax.servlet.http.HttpSessionListener;
import javax.servlet.http.HttpSessionEvent;
public class MySessionListener implements HttpSessionListener {
       private static int sessionCount = 0;

       public void sessionCreated(HttpSessionEvent se) {
               sessionCount++;
                // Write to a log or do some other processing.
        System.out.println("Session has been created");
        System.out.println("Current Session count is"+sessionCount);
        
       }
       public void sessionDestroyed(HttpSessionEvent se) {
                if(sessionCount > 0)
                         sessionCount--;
                     //Write to a log or do some other processing.
        System.out.println("Session has been destroyed");
        System.out.println("Current Session count is"+sessionCount);

           }
}


compile the code  using javac and make sure you are getting a class file getting generated  MySessionListener.class



"javac MySessionListener.java"


Step 5: create "web.xml"  (deployment descriptor) file under  "TestSecApp/WEB-INF"

<web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd" 
id="WebApp_ID" version="2.5">
<welcome-file-list>
 <welcome-file>Test.jsp</welcome-file>
</welcome-file-list>
    <error-page>
        <error-code>404</error-code>
        <location>/error.html</location>
    </error-page>

<security-constraint>

                <web-resource-collection>
                      <web-resource-name>Success</web-resource-name>
                      <url-pattern>/Test.jsp</url-pattern>
                     <http-method>GET</http-method>
                     <http-method>POST</http-method>
                </web-resource-collection>

                <auth-constraint>
 
             <description>
                 These are the roles who have access.
             </description>

                    <role-name>webuser</role-name> 

                </auth-constraint>

          
  <user-data-constraint>
            <description>
                This is how the user data must be transmitted.
            </description>
            <transport-guarantee>NONE</transport-guarantee>
        </user-data-constraint>


</security-constraint>

<login-config>
              <auth-method>FORM</auth-method>
              <form-login-config>
  <form-login-page>/login.jsp</form-login-page>
  <form-error-page>/fail_login.jsp</form-error-page>
       </form-login-config>
</login-config>

<security-role>
               <role-name>webuser</role-name>
</security-role>

<listener>
  <listener-class>MySessionListener</listener-class>
</listener>


</web-app>


Step7: Build war file using "jar -cvf" command like given below.


Step 8:  Update the "TOMCAT_HOME/tomcat-users.xml" - <tomcat-users> tag.


Here "mwiuser" is the user id that we are going to use and it has been mapped to "webuser". As per the configuration in web.xml. Any member of "webuser" group can login and access the secured resource.




Step9:  Deploy the Application to tomcat using the console  (i.e: localhost:8080/manager) (or) just copy the war file you have created to the "Tomcat_home/webapps/"

Step10: Restart the Tomcat  Server.

Step11: Access the deployed application using the url

http://localhost:8080/TestSecApp

Note*:  TestSecApp is the war file name which will be taken as a context root by default.


Tuesday, December 29, 2015

listing directories only from tar file

Recently I faced one requirement where i had to list only certain level of directories with "tar -tvf" (or) "jar -tvf" output.

Here is my sample tar file. where I need to list only two levels of files/directories.

bash-3.2# tar -tvf content.tar
drwxr-xr-x   0/0        0 Dec 29 16:58 2015 direcotry2/
drwxr-xr-x   0/0        0 Dec 29 16:58 2015 direcotry2/directory2a/
drwxr-xr-x   0/0        0 Dec 29 17:00 2015 direcotry2/directory2a/directory2a_media/
-rw-r--r--   0/0       33 Dec 29 17:01 2015 direcotry2/directory2a/directory2a_media/home.html
drwxr-xr-x   0/0        0 Dec 29 16:58 2015 direcotry2/directory2a/directory2a_business/
drwxr-xr-x   0/0        0 Dec 29 16:58 2015 direcotry2/directory2b/
drwxr-xr-x   0/0        0 Dec 29 16:58 2015 direcotry2/directory2c/
drwxr-xr-x   0/0        0 Dec 29 16:56 2015 directory1/
drwxr-xr-x   0/0        0 Dec 29 16:57 2015 directory1/directory1_a/
drwxr-xr-x   0/0        0 Dec 29 16:57 2015 directory1/directory1_a/directory1a_media/
-rw-r--r--   0/0       33 Dec 29 17:00 2015 directory1/directory1_a/directory1a_media/media.html
-rw-r--r--   0/0       33 Dec 29 17:00 2015 directory1/directory1_a/directory1a_media/home.html
drwxr-xr-x   0/0        0 Dec 29 16:57 2015 directory1/directory1_a/directory1a_press/
-rw-r--r--   0/0        0 Dec 29 16:57 2015 directory1/directory1_a/directory1a_press/press.html
-rw-r--r--   0/0        0 Dec 29 16:57 2015 directory1/directory1_a/directory1a_press/welcome.html
drwxr-xr-x   0/0        0 Dec 29 16:55 2015 directory3/


The "awk" way:

Listing the parent directories alone:


bash-3.2# tar -tvf content.tar |awk -F/ '{if(NF==3) print}'
drwxr-xr-x   0/0        0 Dec 29 16:58 2015 direcotry2/
drwxr-xr-x   0/0        0 Dec 29 16:56 2015 directory1/
drwxr-xr-x   0/0        0 Dec 29 16:55 2015 directory3/



Listing the Parent directories with its sub - directories(or) files
bash-3.2# tar -tvf content.tar |awk -F/ '{if(NF==4) print}'
drwxr-xr-x   0/0        0 Dec 29 16:58 2015 direcotry2/directory2a/
drwxr-xr-x   0/0        0 Dec 29 16:58 2015 direcotry2/directory2b/
drwxr-xr-x   0/0        0 Dec 29 16:58 2015 direcotry2/directory2c/
drwxr-xr-x   0/0        0 Dec 29 16:57 2015 directory1/directory1_a/



Another level down
bash-3.2# tar -tvf content.tar |awk -F/ '{if(NF==5) print}'
drwxr-xr-x   0/0        0 Dec 29 17:00 2015 direcotry2/directory2a/directory2a_media/
-rw-r--r--   0/0       33 Dec 29 17:01 2015 direcotry2/directory2a/directory2a_media/home.html
drwxr-xr-x   0/0        0 Dec 29 16:58 2015 direcotry2/directory2a/directory2a_business/
drwxr-xr-x   0/0        0 Dec 29 16:57 2015 directory1/directory1_a/directory1a_media/
-rw-r--r--   0/0       33 Dec 29 17:00 2015 directory1/directory1_a/directory1a_media/media.html
-rw-r--r--   0/0       33 Dec 29 17:00 2015 directory1/directory1_a/directory1a_media/home.html
drwxr-xr-x   0/0        0 Dec 29 16:57 2015 directory1/directory1_a/directory1a_press/
-rw-r--r--   0/0        0 Dec 29 16:57 2015 directory1/directory1_a/directory1a_press/press.html
-rw-r--r--   0/0        0 Dec 29 16:57 2015 directory1/directory1_a/directory1a_press/welcome.html


Just change the "NF" value according to your requirement.

you can use this in case of  "jar -tvf" as well

Hope it helps. Comments welcome.


Thursday, December 17, 2015

Websphere Connection Pool monitoring and Alerting System ( Connection Pool runtime information)


Many of the infrastructure monitoring tools  brings their own JMX monitoring now a days to monitor all the JVM components including connection pool (DataSource) JMS etc.

But, If in case you wanted to set  up your own alerting system (Email Notification etc) for Connection pool.  You can use this  wsadmin script.

Design
  1. It will find all the running servers registered with DMGR (cell) and take a list of running connection pools targeted on those servers.
  2. It will automatically set 80 percent of the connection max_limit as a threshold.
  3. It will print a alert message, in case of possible connection pool overflow. When the current connection limit reaches the threshold.
If you do not want any alerting system and want to see the connection pool usage manually in case of requirement (or) on demand basis. you can just use this jython script with wsadmin. In case if you wanted to setup alerting system refer the additional notes for some ideas of designing the alerting system.

Additional notes
  1. you can run this script in a frequent interval like 2 minutes (or) 10 minutes using the external shell script scheduled in crontab and get the alert generated. ( let me know in comment section if you need help) 
  2. If you have Autosys, you can rather schedule it with JIL and run this in a periodic interval.
  3. After scheduling it to run on certain interval you can use any log parse monitoring tools like Tivoli and get alerts/tickets upon your infrastructure set up.

Version compatibility

was6.1 and above ( for was 6 there is little bit modification required, write a comment if you need)

wsadmin script

import re
Running_JVMS=AdminControl.queryNames("*:type=Server,*").split(java.lang.System.getProperty("line.separator"))
ignorelist=['nodeagent','dmgr']
TMPFILE='/tmp/PoolContents.tmp'
global current_conn
for JVM in Running_JVMS:
        ServerName=AdminControl.invoke(JVM ,"getName()")
        if ServerName not in ignorelist:
                DS=AdminControl.queryNames('*:process='+ServerName+',type=DataSource,*').split(java.lang.System.getProperty("line.separator"))
                for entry in DS:
                        if AdminControl.invoke(entry, "getStatus()") != '99':
                                print "============================================"
                                print "ServerName    :", ServerName
                                DSN=AdminControl.invoke(entry, "getName()")
                                print "DataSourceName:", DSN
                                JNDN=AdminControl.invoke(entry, "getJndiName()")
                                print "JNDIName      :", JNDN
                                MaxConn=AdminControl.invoke(entry, "getMaxConnections()")
                                print "MaxConnections:", MaxConn
                                print "StatusCode    :", AdminControl.invoke(entry, "getStatus()")
                                ##########################
                                fout=open(TMPFILE, 'w')
                                data=AdminControl.invoke(entry, "showPoolContents")
                                fout.write(data)
                                fout.close()
                                percent=0.8
                                threshold=int(MaxConn)*float(percent)
                                print "Threshold     :", threshold
                                try:
                                        fin=open(TMPFILE)
                                        filedata=fin.readlines()
                                        for line in filedata:
                                                #match 1= re.search(r'\:\s\d', line)
                                                #matchstr=re.search(r'Total number of connection in shared pool', line)
                                                matchstr=re.search('(.....\s......\s..\s..........\s..\s......\s....):(\s\d)', line)
                                                if matchstr:
                                                        #current_conn=match.group().split(":")[1].strip(" ")
                                                        current_conn=matchstr.group(2)
                                                        if int(current_conn) >= int(threshold):
                                                                ALERT='YES'
                                                                ALERTSTRING='Connection Pool %r reached 80 percent of its max_limit on Server %r'%(DSN,ServerName)
                                                                print ALERTSTRING
                                                        else:
                                                                ALERT='NO'
                                                        break

                                                else:
                                                        current_conn=0


                                        fin.close()
                                        print "Currently used:", current_conn
                                        print "============================================"
                                except IOError:
                                        print  'Something went Wrong.'



how to run

Save the above code with ".py" extension and run it with wsadmin.

In example


wsadmin.sh -lang jython -username <username> -password <password> -lang jython -f <saved py file name>

Cheers,
Sara Velu

Follow us at  : https://www.facebook.com/middlewareinventory/