Share

Thursday, May 19, 2016

Weblogic NodeManager SSL HandShake Exception while starting

Exception:

<May 19, 2016 4:37:35 AM PDT> <Warning> <Security> <BEA-090476> <Invalid/unknown SSL header was received from peer localhost - 127.0.0.1 during SSL handshake.>
<May 19, 2016 4:37:35 AM> <WARNING> <Uncaught exception in server handlerjavax.net.ssl.SSLHandshakeException: [Security:090476]Invalid/unknown SSL header was received from peer localhost - 127.0.0.1 during SSL handshake.>
javax.net.ssl.SSLHandshakeException: [Security:090476]Invalid/unknown SSL header was received from peer localhost - 127.0.0.1 during SSL handshake.
        at com.certicom.tls.interfaceimpl.TLSConnectionImpl.fireException(Unknown Source)
        at com.certicom.tls.interfaceimpl.TLSConnectionImpl.fireAlertSent(Unknown Source)
        at com.certicom.tls.record.ReadHandler.fireAlert(Unknown Source)
        at com.certicom.tls.record.ReadHandler.getProtocolVersion(Unknown Source)
        at com.certicom.tls.record.ReadHandler.checkVersion(Unknown Source)
        at com.certicom.tls.record.ReadHandler.readRecord(Unknown Source)
        at com.certicom.tls.record.ReadHandler.readUntilHandshakeComplete(Unknown Source)
        at com.certicom.tls.interfaceimpl.TLSConnectionImpl.completeHandshake(Unknown Source)
        at com.certicom.tls.record.ReadHandler.read(Unknown Source)
        at com.certicom.io.InputSSLIOStreamWrapper.read(Unknown Source)

Resolution:

By Default Node Manager will start in a secure port and it should be disabled in "nodemanager.properties"

<May 19, 2016 4:58:36 AM> <INFO> <Secure socket listener started on port 5556>
May 19, 2016 4:58:36 AM weblogic.nodemanager.server.SSLListener run
INFO: Secure socket listener started on port 5556


[weblogic@localhost nodemanager]$ more nodemanager.properties
#Tue May 10 00:52:47 PDT 2016
DomainsFile=/opt/wls/wlserver/common/nodemanager/nodemanager.domains
LogLimit=0
PropertiesVersion=10.3
DomainsDirRemoteSharingEnabled=false
javaHome=/usr/java/jdk1.7.0_79/
AuthenticationEnabled=false
NodeManagerHome=/opt/wls/wlserver/common/nodemanager
JavaHome=/usr/java/jdk1.7.0_79/jre
LogLevel=INFO
DomainsFileEnabled=true
StartScriptName=startWebLogic.sh
ListenAddress=
NativeVersionEnabled=true
ListenPort=5556
LogToStderr=true
SecureListener=false
LogCount=1
DomainRegistrationEnabled=false
StopScriptEnabled=false
QuitEnabled=false
LogAppend=true
StateCheckInterval=500
CrashRecoveryEnabled=false
StartScriptEnabled=false


Update the StartNodeManager.sh  JAVA_OPTS section with below startup argument  ( Update the StartNodeManager.sh and put this line before weblogic.NodeManager)

 "-Dweblogic.nodemanager.sslHostNameVerificationEnabled=false"

Update the AdminServer JAVA_OPTS with the below argument


java -server -Xmx512m -Xms512m -XX:MaxPermSize=128m -DWeblogic.Name=AdminServer -Dweblogic.Stdout="/tmp/AdminOut.log" -Dweblogic.Stderr="/tmp/AdminErr.log" -Dweblogic.security.SSL.ignoreHostnameVerification=true weblogic.Server

Start the Node Manager now using "/opt/wls/wlserver/server/bin/StartNodeManager.sh" script or by commandline as follows


/usr/java/jdk1.7.0_79//bin/java -Xms32m -Xmx200m -Dcoherence.home=@COHERENCE_HOME -Dbea.home=/opt/wls -Djava.security.policy=/opt/wls/wlserver/server/lib/weblogic.policy -Dweblogic.nodemanager.javaHome=/usr/java/jdk1.7.0_79/ -Djava.library.path=/opt/wls/wlserver/server/native/linux/x86_64/ -Dweblogic.nodemanager.sslHostNameVerificationEnabled=false weblogic.NodeManager

Check the Node manager Status in the Console. Now it should be Reachable



Friday, May 13, 2016

Apache Tomcat JDBC Realm with mysql and Form Based Web Application.

In my earlier post, we have designed and tested the Web Application  in Apache Tomcat with Default Authentication Provider . this time we will use JDBC realm as the Authentication provider and we will use mysql as the Database.

Let's get started

Note*:  I presume that you have basic knowledge of Linux user setup and Tomcat installation and administration. So I am skipping few basic steps here, I believe  you should be able to understand. in case of any doubts of further support please comment.

Technology Stack

Operating System: RHEL6
MySQL Version: 5.7.12
Tomcat Version7.0.69

MYSQL SETUP


Download the product here

Install mysql community server using RPM and YUM


[root@localhost mysql]# yum install mysql-community-{server,client,common,libs}-* mysql-5.*  
Loaded plugins: product-id, refresh-packagekit, security, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
Setting up Install Process
Examining mysql-community-server-5.7.12-1.el6.x86_64.rpm: mysql-community-server-5.7.12-1.el6.x86_64
Marking mysql-community-server-5.7.12-1.el6.x86_64.rpm to be installed
Examining mysql-community-client-5.7.12-1.el6.x86_64.rpm: mysql-community-client-5.7.12-1.el6.x86_64
Marking mysql-community-client-5.7.12-1.el6.x86_64.rpm to be installed
Examining mysql-community-common-5.7.12-1.el6.x86_64.rpm: mysql-community-common-5.7.12-1.el6.x86_64
Marking mysql-community-common-5.7.12-1.el6.x86_64.rpm to be installed
Examining mysql-community-libs-5.7.12-1.el6.x86_64.rpm: mysql-community-libs-5.7.12-1.el6.x86_64
Marking mysql-community-libs-5.7.12-1.el6.x86_64.rpm to be installed
Examining mysql-community-libs-compat-5.7.12-1.el6.x86_64.rpm: mysql-community-libs-compat-5.7.12-1.el6.x86_64
Marking mysql-community-libs-compat-5.7.12-1.el6.x86_64.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package mysql-community-client.x86_64 0:5.7.12-1.el6 will be installed
---> Package mysql-community-common.x86_64 0:5.7.12-1.el6 will be installed
---> Package mysql-community-libs.x86_64 0:5.7.12-1.el6 will be obsoleting
---> Package mysql-community-libs-compat.x86_64 0:5.7.12-1.el6 will be obsoleting
---> Package mysql-community-server.x86_64 0:5.7.12-1.el6 will be installed
---> Package mysql-libs.x86_64 0:5.1.73-3.el6_5 will be obsoleted
--> Finished Dependency Resolution

Dependencies Resolved


Install       5 Package(s)

Total size: 843 M
Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
.....
.....

Installed:
  mysql-community-client.x86_64 0:5.7.12-1.el6               mysql-community-common.x86_64 0:5.7.12-1.el6          mysql-community-libs.x86_64 0:5.7.12-1.el6         
  mysql-community-libs-compat.x86_64 0:5.7.12-1.el6          mysql-community-server.x86_64 0:5.7.12-1.el6         
                     

Complete!


[root@localhost mysql]# 



Start the mysql Server

[root@localhost mysql]# /etc/init.d/mysqld start
Initializing MySQL database:                               [  OK  ]
Installing validate password plugin:                       [  OK  ]
Starting mysqld:                                           [  OK  ]


Check the version of mysql server

[root@localhost mysql]# mysqladmin --version
mysqladmin  Ver 8.42 Distrib 5.7.12, for Linux on x86_64


grep 'temporary password' /var/log/mysqld.log

[root@localhost ~]# grep 'temporary password' /var/log/mysqld.log
2016-05-13T18:33:49.054595Z 1 [Note] A temporary password is generated for root@localhost: >!ds6sfiEijp

Login with that temporary Password

[root@localhost ~]# mysql -u root -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.7.12

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> 

Update the  password for "root" account

mysql> ALTER USER 'root'@'localhost' IDENTIFIED BY 'secure';
ERROR 1819 (HY000): Your password does not satisfy the current policy requirements

mysql> ALTER USER 'root'@'localhost' IDENTIFIED BY 'mwi@4321';
Query OK, 0 rows affected (0.02 sec)


Execute the SQL queries given below one by one (or) All at once.

DROP DATABASE IF EXISTS mwidb;
CREATE DATABASE mwidb;
USE mwidb;
CREATE TABLE tomcat_users (
user_name varchar(20) NOT NULL PRIMARY KEY,
password varchar(32) NOT NULL
);
CREATE TABLE tomcat_roles (
role_name varchar(20) NOT NULL PRIMARY KEY
);
CREATE TABLE tomcat_users_roles (
user_name varchar(20) NOT NULL,
role_name varchar(20) NOT NULL,
PRIMARY KEY (user_name, role_name),
CONSTRAINT tomcat_users_roles_foreign_key_1 FOREIGN KEY (user_name) REFERENCES tomcat_users (user_name),
CONSTRAINT tomcat_users_roles_foreign_key_2 FOREIGN KEY (role_name) REFERENCES tomcat_roles (role_name)
);
INSERT INTO tomcat_users (user_name, password) VALUES ('sarav', 'mwi@4321');
INSERT INTO tomcat_users (user_name, password) VALUES ('mwiuser', 'mwiuser');
INSERT INTO tomcat_roles (role_name) VALUES ('webuser');
INSERT INTO tomcat_roles (role_name) VALUES ('manager');
INSERT INTO tomcat_roles (role_name) VALUES ('manager-gui');
INSERT INTO tomcat_users_roles (user_name, role_name) VALUES ('mwiuser', 'webuser');
INSERT INTO tomcat_users_roles (user_name, role_name) VALUES ('sarav', 'webuser');
INSERT INTO tomcat_users_roles (user_name, role_name) VALUES ('sarav', 'manager-gui');
INSERT INTO tomcat_users_roles (user_name, role_name) VALUES ('sarav', 'manager');

COMMIT;


mysql> DROP DATABASE IF EXISTS mwidb;
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> CREATE DATABASE mwidb;
Query OK, 1 row affected (0.00 sec)

mysql> USE mwidb;
Database changed
mysql> CREATE TABLE tomcat_users (
    -> user_name varchar(20) NOT NULL PRIMARY KEY,
    -> password varchar(32) NOT NULL
    -> );
Query OK, 0 rows affected (0.13 sec)

mysql> CREATE TABLE tomcat_roles (
    -> role_name varchar(20) NOT NULL PRIMARY KEY
    -> );
Query OK, 0 rows affected (0.09 sec)

mysql> CREATE TABLE tomcat_users_roles (
    -> user_name varchar(20) NOT NULL,
    -> role_name varchar(20) NOT NULL,
    -> PRIMARY KEY (user_name, role_name),
    -> CONSTRAINT tomcat_users_roles_foreign_key_1 FOREIGN KEY (user_name) REFERENCES tomcat_users (user_name),
    -> CONSTRAINT tomcat_users_roles_foreign_key_2 FOREIGN KEY (role_name) REFERENCES tomcat_roles (role_name)
    -> );
Query OK, 0 rows affected (0.05 sec)

mysql> INSERT INTO tomcat_users (user_name, password) VALUES ('sarav', 'Sara@4321');
Query OK, 1 row affected (0.08 sec)

mysql> INSERT INTO tomcat_users (user_name, password) VALUES ('mwiuser', 'mwiuser');
Query OK, 1 row affected (0.02 sec)

mysql> INSERT INTO tomcat_roles (role_name) VALUES ('webuser');
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO tomcat_roles (role_name) VALUES ('manager');
Query OK, 1 row affected (0.02 sec)


mysql> INSERT INTO tomcat_roles (role_name) VALUES ('manager-gui');
Query OK, 1 row affected (0.13 sec)


mysql> INSERT INTO tomcat_users_roles (user_name, role_name) VALUES ('mwiuser', 'webuser');
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO tomcat_users_roles (user_name, role_name) VALUES ('sarav', 'webuser');
Query OK, 1 row affected (0.00 sec)

mysql> INSERT INTO tomcat_users_roles (user_name, role_name) VALUES ('sarav', 'manager');
Query OK, 1 row affected (0.01 sec)


mysql> INSERT INTO tomcat_users_roles (user_name, role_name) VALUES ('sarav', 'manager-gui');
Query OK, 1 row affected (0.10 sec)


mysql> COMMIT;
Query OK, 0 rows affected (0.00 sec)

Verify the tables and its record set.


mysql> show tables;
+--------------------+
| Tables_in_mwidb    |
+--------------------+
| tomcat_roles       |
| tomcat_users       |
| tomcat_users_roles |
+--------------------+
3 rows in set (0.01 sec)

mysql> select * from tomcat_roles;
+-----------+
| role_name |
+-----------+
| manager   |
| webuser   |
+-----------+
2 rows in set (0.00 sec)

mysql> select * from tomcat_users;
+-----------+-----------+
| user_name | password  |
+-----------+-----------+
| mwiuser   | mwiuser   |
| sarav     | mwi@4321 |
+-----------+-----------+
2 rows in set (0.00 sec)

mysql> select * from tomcat_users_roles;
+-----------+-------------+
| user_name | role_name   |
+-----------+-------------+
| sarav     | manager     |
| sarav     | manager-gui |
| mwiuser   | webuser     |
| sarav     | webuser     |
+-----------+-------------+
4 rows in set (0.00 sec)

TOMCAT SETUP



Download the Tomcat Application Server ZIP Distribution here and un compress it under /opt/tomcat directory

[tomcat@localhost tomcat]$ pwd
/opt/tomcat
[tomcat@localhost tomcat]$ ls -lrt
total 4
drwxr-xr-x. 9 tomcat tomcat 4096 Apr 11 10:59 apache-tomcat-7.0.69


Download the JConnecter JDBC Driver from here  and get the jar file

[sarav@localhost Downloads]$ ls -rlt
total 138660
-rw-rw-r--. 1 sarav sarav   3899019 May 13 12:44 mysql-connector-java-5.1.39.tar.gz

[sarav@localhost Downloads]$ pwd
/home/sarav/Downloads

[sarav@localhost Downloads]$ gunzip mysql-connector-java-5.1.39.tar.gz 

[sarav@localhost Downloads]$ ls -lrt
total 145824
-rw-rw-r--. 1 sarav sarav 138082565 May  1 22:07 jdk-7u79-linux-x64.rpm
-rw-rw-r--. 1 sarav sarav  11233280 May 13 12:44 mysql-connector-java-5.1.39.tar

[sarav@localhost Downloads]$ tar -xf mysql-connector-java-5.1.39.tar

[sarav@localhost Downloads]$ ls -lrt
total 145828
-rw-rw-r--. 1 sarav sarav 138082565 May  1 22:07 jdk-7u79-linux-x64.rpm
drwxr-xr-x. 4 sarav sarav      4096 May  4 04:11 mysql-connector-java-5.1.39
-rw-rw-r--. 1 sarav sarav  11233280 May 13 12:44 mysql-connector-java-5.1.39.tar

[sarav@localhost Downloads]$ ls -lrt mysql-connector-java-5.1.39/mysql-connector-java-5.1.39-bin.jar 
-rw-r--r--. 1 sarav sarav 989497 May  4 04:11 mysql-connector-java-5.1.39/mysql-connector-java-5.1.39-bin.jar

Copy the jar file to Lib directory of Tomcat

[sarav@localhost Downloads]$ cp mysql-connector-java-5.1.39/mysql-connector-java-5.1.39-bin.jar /opt/tomcat/apache-tomcat-7.0.69/lib/ls -lrt^C
[sarav@localhost Downloads]$ ls -lrt /opt/tomcat/apache-tomcat-7.0.69/lib/mysql-connector-java-5.1.39-bin.jar 
-rw-rw-r--. 1 tomcat tomcat 989497 May  4 04:11 /opt/tomcat/apache-tomcat-7.0.69/lib/mysql-connector-java-5.1.39-bin.jar


Server.xml changes

Find the section LockOutRealm and Comment  the below lines with <!--     -->

<Realm className="org.apache.catalina.realm.UserDatabaseRealm"
               resourceName="UserDatabase"/>

and Add the JDBC Realm values

<Realm  className="org.apache.catalina.realm.JDBCRealm"
                             driverName="com.mysql.jdbc.Driver"
                                                            connectionURL="jdbc:mysql://localhost:3306/mwidb" connectionName="root" connectionPassword="mwi@4321" userTable="tomcat_users" userNameCol="user_name" userCredCol="password" userRoleTable="tomcat_users_roles" roleNameCol="role_name" />




Here, 

1) jdbc:mysql://localhost:3306/mwidb is the JDBC URL and mwidb is the DataBase Name2) connectionName is the username I am using to access the Database3) connectionPassword is the password of the connectionName account mentioned.

Find the Below Resource and Comment this as well with <!--     -->

<Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved" factory="org.apache.catalina.users.MemoryUserDatabaseFactory" pathname="conf/tomcat-users.xml" />





Now restart the tomcat server with ./shutdown.sh & ./startup.sh scripts

Access the manager app at http://localhost:8080/manager/  and enter the credentials of  account "sarav" and you should be able to get through without any issues as it is having "manager-gui" privilage.  





Application Design and Deployment


Hope you have checked out my previous tutorial about web application designing and form based auth at http://www.mwinventory.in/2015/12/apache-tomcat-web-application.html 

we can use the same Application here as well (without any code changes).  you can download the war file from here.

Once you have downloaded,  copy the war file to tomcat/webapps location to get it deployed. (Hot Deployment)

[tomcat@localhost webapps]$ ls -lrt
total 24
drwxr-xr-x.  3 tomcat tomcat 4096 Apr 11 10:58 ROOT
drwxr-xr-x.  5 tomcat tomcat 4096 Apr 11 10:58 manager
drwxr-xr-x.  5 tomcat tomcat 4096 Apr 11 10:58 host-manager
drwxr-xr-x.  7 tomcat tomcat 4096 Apr 11 10:58 examples
drwxr-xr-x. 14 tomcat tomcat 4096 Apr 11 10:58 docs
drwxrwxr-x.  3 tomcat tomcat 4096 May 13 14:30 TestSecApp
[tomcat@localhost webapps]$ pwd
/opt/tomcat/apache-tomcat-7.0.69/webapps

Restart the tomcat instance after application installation and make sure the app gets deployed by checking the logs. you should be able to see the app deploying message

INFO: Deploying web application directory /opt/tomcat/apache-tomcat-7.0.69/webapps/TestSecApp

Application Testing.

Application Name is the "ContextRoot" of the application here. so you can test the application at http://localhost:8080/TestSecApp


Screen1 :  User Form, Enter the  "sarav" account credentials to test


Screen2: Sucessfull login 


Screen3: logout screen



Screen4: Test it with some incorrect credentials and you will not be able to login 



How to set HeapMemory/JVM Arguments in Tomcat

Option 1:


Create the setenv.sh under CATALINA_HOME you can find the value of  CATALINA_HOME if you execute version.sh available under the installation_path/bin.

In general,  installation_path will be your CATALINA_HOME

[tomcat@localhost bin]$ ./version.sh
Using CATALINA_BASE:   /opt/tomcat/apache-tomcat-7.0.69
Using CATALINA_HOME:   /opt/tomcat/apache-tomcat-7.0.69
Using CATALINA_TMPDIR: /opt/tomcat/apache-tomcat-7.0.69/temp
Using JRE_HOME:        /usr
Using CLASSPATH:       /opt/tomcat/apache-tomcat-7.0.69/bin/bootstrap.jar:/opt/tomcat/apache-tomcat-7.0.69/bin/tomcat-juli.jar
Server version: Apache Tomcat/7.0.69
Server built:   Apr 11 2016 07:57:09 UTC
Server number:  7.0.69.0
OS Name:        Linux
OS Version:     2.6.32-504.el6.x86_64
Architecture:   amd64
JVM Version:    1.7.0_79-b15
JVM Vendor:     Oracle Corporation
 
[tomcat@localhost bin]$ pwd
/opt/tomcat/apache-tomcat-7.0.69/bin



If you have just installed the tomcat, you may not be having setenv.sh file available under the CATALINA_HOME/bin location. So you must create it.

Here we go.

I am creating a new file called setenv.sh under bin location and defining my JVM arguments.


[tomcat@localhost bin]$ cat > setenv.sh
JAVA_OPTS="-Xms1024m -Xmx1024m -Xss228k -XX:+UseParallelGC"

[tomcat@localhost bin]$ cat setenv.sh
JAVA_OPTS="-Xms1024m -Xmx1024m -Xss228k -XX:+UseParallelGC"

[tomcat@localhost bin]$ pwd
/opt/tomcat/apache-tomcat-7.0.69/bin

 

Option 2:

you can add these parameters in the very beginning of catalina.sh file.

#!/bin/sh
...
...  

......

#   LOGGING_MANAGER (Optional) Override Tomcat's logging manager
#                   Example (all one line)
#                   LOGGING_MANAGER="-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager"
# -----------------------------------------------------------------------------
JAVA_OPTS="-Xms1024m -Xmx1024m -Xss228k -XX:+UseParallelGC"

# OS specific support.  $var _must_ be set to either true or false
.

 

Verification:

Start the tomcat instance and verify.

you can verify the JVM arguments in two ways. 

1)  In catalina log you would be able to something like this.

INFO: Command line argument: -Xms1024m
May 13, 2016 10:34:46 AM org.apache.catalina.startup.VersionLoggerListener log
INFO: Command line argument: -Xmx1024m
May 13, 2016 10:34:47 AM org.apache.catalina.startup.VersionLoggerListener log
INFO: Command line argument: -Xss228k
May 13, 2016 10:34:47 AM org.apache.catalina.startup.VersionLoggerListener log
INFO: Command line argument: -XX:+UseParallelGC
 

2)  using ps command  you can find the JVM arguments

[tomcat@localhost ~]$ ps -eaf|grep -i java
tomcat    5944  3481  3 10:34 pts/0    00:00:05 /usr/bin/java -Djava.util.logging.config.file=/opt/tomcat/apache-tomcat-7.0.69/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xms1024m -Xmx1024m -Xss228k -XX:+UseParallelGC -Djdk.tls.ephemeralDHKeySize=2048 -Djava.endorsed.dirs=/opt/tomcat/apache-tomcat-7.0.69/endorsed -classpath /opt/tomcat/apache-tomcat-7.0.69/bin/bootstrap.jar:/opt/tomcat/apache-tomcat-7.0.69/bin/tomcat-juli.jar -Dcatalina.base=/opt/tomcat/apache-tomcat-7.0.69 -Dcatalina.home=/opt/tomcat/apache-tomcat-7.0.69 -Djava.io.tmpdir=/opt/tomcat/apache-tomcat-7.0.69/temp org.apache.catalina.startup.Bootstrap start
tomcat    6008  5988  0 10:37 pts/1    00:00:00 grep -i java
[tomcat@localhost ~]$

Tuesday, May 10, 2016

Weblogic NodeManager not Starting

Issue:
  
Node manager will not start with following exception "Native version is enabled but nodemanager native library could not be loaded"


It means that you have 64 bit version of java installed and 32 bit  node manager library got loaded into classpath.


Test your Java Version First

[weblogic@localhost x86_64]$ java -version
java version "1.7.0_79"
Java(TM) SE Runtime Environment (build 1.7.0_79-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)




Error Statement
 
[weblogic@localhost bin]$ ./startNodeManager.sh
+ CLASSPATH=/usr/java/jdk1.7.0_79//lib/tools.jar:/opt/wls/wlserver/server/lib/weblogic_sp.jar:/opt/wls/wlserver/server/lib/weblogic.jar:/opt/wls/modules/features/weblogic.server.modules_10.3.6.0.jar:/opt/wls/wlserver/server/lib/webservices.jar:/opt/wls/modules/org.apache.ant_1.7.1/lib/ant-all.jar:/opt/wls/modules/net.sf.antcontrib_1.1.0.0_1-0b2/lib/ant-contrib.jar::/opt/wls
+ '[' '!' -z '' ']'
+ '[' '!' -z '' ']'
+ export CLASSPATH
+ export PATH
+ cd /opt/wls/wlserver/common/nodemanager
+ set -x
+ '[' '' '!=' '' ']'
+ '[' '' '!=' '' ']'
+ /usr/java/jdk1.7.0_79//bin/java -Xms32m -Xmx200m -Dcoherence.home=@COHERENCE_HOME -Dbea.home=/opt/wls -Djava.security.policy=/opt/wls/wlserver/server/lib/weblogic.policy -Dweblogic.nodemanager.javaHome=/usr/java/jdk1.7.0_79/ weblogic.NodeManager -v
<May 10, 2016 12:49:58 AM> <INFO> <Loading domains file: /opt/wls/wlserver/common/nodemanager/nodemanager.domains>
May 10, 2016 12:49:58 AM weblogic.nodemanager.server.NMServerConfig initDomainsMap
INFO: Loading domains file: /opt/wls/wlserver/common/nodemanager/nodemanager.domains
Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /opt/wls/wlserver/server/native/linux/i686/libnodemanager.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
<May 10, 2016 12:49:58 AM> <SEVERE> <Fatal error in node manager server>
weblogic.nodemanager.common.ConfigException: Native version is enabled but nodemanager native library could not be loaded
    at weblogic.nodemanager.server.NMServerConfig.initProcessControl(NMServerConfig.java:269)
    at weblogic.nodemanager.server.NMServerConfig.<init>(NMServerConfig.java:210)
    at weblogic.nodemanager.server.NMServer.init(NMServer.java:182)
    at weblogic.nodemanager.server.NMServer.<init>(NMServer.java:148)
    at weblogic.nodemanager.server.NMServer.main(NMServer.java:380)
    at weblogic.NodeManager.main(NodeManager.java:31)
Caused by: java.lang.UnsatisfiedLinkError: /opt/wls/wlserver/server/native/linux/i686/libnodemanager.so: /opt/wls/wlserver/server/native/linux/i686/libnodemanager.so: wrong ELF class: ELFCLASS32 (Possible cause: architecture word width mismatch)
    at java.lang.ClassLoader$NativeLibrary.load(Native Method)
    at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1965)
    at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890)
    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1880)
    at java.lang.Runtime.loadLibrary0(Runtime.java:849)
    at java.lang.System.loadLibrary(System.java:1088)
    at weblogic.nodemanager.util.UnixProcessControl.<init>(UnixProcessControl.java:24)
    at weblogic.nodemanager.util.ProcessControlFactory.getProcessControl(ProcessControlFactory.java:22)
    at weblogic.nodemanager.server.NMServerConfig.initProcessControl(NMServerConfig.java:267)
    ... 5 more

May 10, 2016 12:49:58 AM weblogic.nodemanager.server.NMServer main
SEVERE: Fatal error in node manager server
weblogic.nodemanager.common.ConfigException: Native version is enabled but nodemanager native library could not be loaded
    at weblogic.nodemanager.server.NMServerConfig.initProcessControl(NMServerConfig.java:269)
    at weblogic.nodemanager.server.NMServerConfig.<init>(NMServerConfig.java:210)
    at weblogic.nodemanager.server.NMServer.init(NMServer.java:182)
    at weblogic.nodemanager.server.NMServer.<init>(NMServer.java:148)
    at weblogic.nodemanager.server.NMServer.main(NMServer.java:380)
    at weblogic.NodeManager.main(NodeManager.java:31)
Caused by: java.lang.UnsatisfiedLinkError: /opt/wls/wlserver/server/native/linux/i686/libnodemanager.so: /opt/wls/wlserver/server/native/linux/i686/libnodemanager.so: wrong ELF class: ELFCLASS32 (Possible cause: architecture word width mismatch)
    at java.lang.ClassLoader$NativeLibrary.load(Native Method)
    at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1965)
    at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890)
    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1880)
    at java.lang.Runtime.loadLibrary0(Runtime.java:849)
    at java.lang.System.loadLibrary(System.java:1088)
    at weblogic.nodemanager.util.UnixProcessControl.<init>(UnixProcessControl.java:24)
    at weblogic.nodemanager.util.ProcessControlFactory.getProcessControl(ProcessControlFactory.java:22)
    at weblogic.nodemanager.server.NMServerConfig.initProcessControl(NMServerConfig.java:267)
    ... 5 more

+ set +x




Resolution

Goto "<MWHOME>/wlserver/server/bin"

and open  startNodeManager.sh . find weblogic.NodeManager  and replace it with "-Djava.library.path="/opt/wls/wlserver/server/native/linux/x86_64/"  weblogic.NodeManager"


Lines from the  startNodeManager.sh file


 cd "${NODEMGR_HOME}"
set -x
if [ "$LISTEN_PORT" != "" ]
 then
   if [ "$LISTEN_ADDRESS" != "" ]
    then
     "${JAVA_HOME}/bin/java" ${JAVA_VM} ${MEM_ARGS} ${JAVA_OPTIONS} -Djava.security.policy="${WL_HOME}/server/lib/weblogic.policy" -Dweblogic.nodemanager.javaHome="${JAVA_HOME}" -DListenAddress="${LISTEN_ADDRESS}" -DListenPort="${LISTEN_PORT}" -Djava.library.path="/opt/wls/wlserver/server/native/linux/x86_64/" weblogic.NodeManager -v
    else
     "${JAVA_HOME}/bin/java" ${JAVA_VM} ${MEM_ARGS} ${JAVA_OPTIONS} -Djava.security.policy="${WL_HOME}/server/lib/weblogic.policy" -Dweblogic.nodemanager.javaHome="${JAVA_HOME}" -DListenPort="${LISTEN_PORT}" -Djava.library.path="/opt/wls/wlserver/server/native/linux/x86_64/"  weblogic.NodeManager -v
   fi
 else
   if [ "$LISTEN_ADDRESS" != "" ]
    then
     "${JAVA_HOME}/bin/java" ${JAVA_VM} ${MEM_ARGS} ${JAVA_OPTIONS} -Djava.security.policy="${WL_HOME}/server/lib/weblogic.policy" -Dweblogic.nodemanager.javaHome="${JAVA_HOME}" -DListenAddress="${LISTEN_ADDRESS}" -Djava.library.path="/opt/wls/wlserver/server/native/linux/x86_64/" weblogic.NodeManager -v
    else
     "${JAVA_HOME}/bin/java" ${JAVA_VM} ${MEM_ARGS} ${JAVA_OPTIONS} -Djava.security.policy="${WL_HOME}/server/lib/weblogic.policy" -Dweblogic.nodemanager.javaHome="${JAVA_HOME}" -Djava.library.path="/opt/wls/wlserver/server/native/linux/x86_64/" weblogic.NodeManager -v
   fi
fi
set +x
fi


Rerun the script and you should be able to get it started now.

Hope it helps.
AKSarav