Monday, August 6, 2018

EBS Adding a node in Shared File System


The Oracle E-Business Suite Release 12.2 architecture is a framework for multi-tiered, distributed computing. In a shared file system, all application tier files including the INSTANCE_TOP are installed on a shared disk resource, which is mounted on each application tier node. Any application tier node can be configured to perform any of the standard application tier services, such as Forms, Web and Concurrent Processing (Batch) services. All changes made to the shared file system are immediately accessible to all application tier nodes.
Note:
•All application tier nodes in a shared file system must be running the same operating system.
•Shared Application Tier File System can not be a read-only-file system unlike in the previous releases.
•User ID and Group ID should be consistent across all nodes to avoid file access permission issues.
•The same absolute path must be retained for the shared file system mount points on each node.
•The value for the context variable "s_atName" must be same across all the application tier node context files.
General Three Tier Architecture
Application/Database Tier Components
Release 12.2 Dual File System Setup on a Single Tier
Shared Application Tier File System Layout
Step by Step Configuration
1. Start weblogic admin server and then Run pre-clone on application tier node 1 on both RUN (fs2) and PATCH (fs1) file systems.



On the RUN file system:
[applmgr@ed-olapplin1 ~]$ ./t1/tst/fs2/EBSapps/appl/APPStst_ed-olapplin1.env
[applmgr@ed-olapplin1 ~]$ cd $ADMIN_SCRIPTS_HOME
[applmgr@ed-olapplin1 scripts]$ adadminsrvctl.sh start 
[applmgr@ed-olapplin1 scripts]$ adadminsrvctl.sh status
[applmgr@ed-olapplin1 scripts]$ adpreclone.pl appsTier
On the PATCH file system:
[applmgr@ed-olapplin1 ~]$ . /t1/tst/fs1/EBSapps/appl/APPStst_ed-olapplin1.env
[applmgr@ed-olapplin1 ~]$ cd $ADMIN_SCRIPTS_HOME
[applmgr@ed-olapplin1 scripts]$ adadminsrvctl.sh status
[applmgr@ed-olapplin1 scripts]$ adadminsrvctl.sh start forcepatchfs
[applmgr@ed-olapplin1 scripts]$ adpreclone.pl appsTier
2. Check below context file parameters settings on both RUN & PATCH files systems.
<shared_file_system oa_var="s_shared_file_system">true</shared_file_system>
<APPL_TOP_NAME oa_var="s_atName">ed-olapplin1</APPL_TOP_NAME>

3. On the new node (ed-olapplin2) that is to be added do the below steps. The node to be added now will be known as the secondary application tier node. This node can be configured to run all services except for the Oracle WebLogic Administration Server.
[applmgr@ed-olapplin2 ~]$  more /etc/oraInst.loc
inventory_loc=/t1/tst/oraInventory
inst_group=oinstall
[applmgr@ed-olapplin2 ~]$ export PATH=/t1/tst/fs2/FMW_Home/webtier/perl/bin:$PATH
4. It is recommended to keep the Instance Top (INST_TOP=/t1/tst/fs2/inst/apps/tst_edolapplin1 AND /t1/tst/fs2/inst/apps/tst_ed-olapplin2) on the shared file system for Ease of maintenance and The number of application tier processes writing to this location is limited when compared with the previous releases.
5. Ensure the WebLogic Administration Server is running from both run and Patch file system on the primary application tier node [on ED-OLAPPLIN1] if not run it as shown in Step 1 above.
[applmgr@ed-olapplin2 ~]$ export PATH=/t1/tst/fs2/FMW_Home/webtier/perl/bin:$PATH
6. A sample pairsfile for the run/patch file system is instantiated into the instance home on the primary application tier node. The file is called <SID>_<primary_node_name>_.txt, and located in the <inst_top>/appl/admin/ directory. For a configuration with an instance name of VIS, the pairsfile will be /u01/12.2/fs1/inst/apps/VIS_appstier1/appl/admin/VIS_appstier1.txt.
/t1/tst/fs2/inst/apps/tst_ed-olapplin1/appl/admin/tst_ed-olapplin1.txt
# cd $INST_TOP/appl/admin
# pwd
/t1/tst/fs2/inst/apps/tst_ed-olapplin1/appl/admin
# mkdir –p /t1/tst/pairsfile
# cp /t1/tst/fs2/inst/apps/tst_ed-olapplin1/appl/admin/tst_ed-olapplin1.txt /t1/tst/pairsfile/mypairsfile.txt
7. Execute adclonectx utility to configure both run and Patch file system. Note that since we have to execute the adconectx.pl script from the new node [ED-OLAPPLIN2] that will be added that is why we are copying the mypairsfile.txt and tst_ed-olapplin1.xml file on common shared location.
[applmgr@ed-olapplin2 ~]$ export PATH=/t1/tst/fs2/FMW_Home/webtier/perl/bin:$PATH
[applmgr@ed-olapplin2 bin]$ cd /t1/tst/fs2/EBSapps/comn/clone/bin
[applmgr@ed-olapplin2 bin]$ perl adclonectx.pl addnode contextfile=/t1/tst/fs2/inst/apps/tst_ed-olapplin1/appl/admin/tst_ed-olapplin1.xml pairsfile=/t1/tst/pairsfile/mypairsfile.txt outfile=/t1/tst/pairsfile /mypairsOUTPUT.txt dualfs=yes
Below is the mypairsfile.txt file content,
# $Header: pairs_file_dualfsclone_ux_txt.tmp 120.0.12020000.2 2015/07/29 19:36:02 jmajumde noship $
# Copyright (c) 2005, 2015  Oracle and/or its affiliates.
# All rights reserved.
# Version 12.0.0
#
# This is a sample pairs file generated by the autoconfig tool based on the
# context values set on the primary application tier node.
# This pairs file can be modified and used when adding additional
# application tier nodes. Some of the parameters set in this pairs file
# assume that you are using a shared file system.
[Base]
s_base=/t1/tst
s_current_base=/t1/tst/fs2
s_other_base=/t1/tst/fs1
s_ne_base=/t1/tst/fs_ne
[General]
s_applptmp=/usr/tmp
s_appsgroup=oinstall
s_appsuser=applmgr
s_dbuser=oracle
s_dbgroup=dba
s_dbdomain=samiora.blogspot.com
s_at=/t1/tst/fs2/EBSapps/appl
s_com=/t1/tst/fs2/EBSapps/comn
s_tools_oh=/t1/tst/fs2/EBSapps/10.1.2
s_weboh_oh=/t1/tst/fs2/FMW_Home/webtier
s_fmw_home=/t1/tst/fs2/FMW_Home
s_dbGlnam=tst
s_dbSid=tst
s_dbhost=edolraclin-dbsrv
s_clonestage=/t1/tst/fs2/EBSapps/comn/clone
s_dbport=1522
s_options_symlinks=Options -FollowSymLinks
s_proxyhost=
s_proxybypassdomain=samiora.blogspot.com
s_proxyport=
s_nonproxyhosts=
s_javamailer_imapdomainname=NoImapDomain
s_javamailer_imaphost=NoImapHost
s_javamailer_reply_to=NoReplyTo
s_javamailer_outbound_user=changeOnJavaMailerInstall
s_smtphost=ed-olapplin1
s_smtpdomainname=samiora.blogspot.com
s_file_edition_type=run
s_port_pool=1
patch_s_port_pool=2
s_admhost=ed-olapplin1
s_atName=ed-olapplin1
s_shared_file_system=true
[Web Entry Point Configuration]
s_webentryurlprotocol=http
s_webentryhost=ed-olapplin2
s_webentrydomain=samiora.blogspot.com
s_active_webport=8001
s_endUserMonitoringURL=http://ed-olapplin2.samiora.blogspot.com:8001/oracle_smp_chronos/oracle_smp_chronos_sdk.gif
s_external_url=http://ed-olapplin2.samiora.blogspot.com:8001
s_login_page=http://ed-olapplin2.samiora.blogspot.com:8001/OA_HTML/AppsLogin
[Instance Specific]
# Please provide values for the context variables listed below. On the source
# instance they are instantiated as shown in the comment section below.
# These values should only be used as reference to fill out the instance
# values for the new node.
#s_temp=/t1/tst/fs2/inst/apps/tst_ed-olapplin1/temp
#s_contextname=tst_ed-olapplin1
#s_hostname=ed-olapplin1
#s_domainname=samiora.blogspot.com
#s_cphost=ed-olapplin1
#s_webhost=ed-olapplin1
#s_config_home=/t1/tst/fs2/inst/apps/tst_ed-olapplin1
#s_inst_base=/t1/tst
#s_display=ed-olapplin1:0.0
#s_forms-c4ws_display=ed-olapplin1:0.0
#s_ohs_instance=EBS_web_erp_OHS1
#s_webport=8001
#s_http_listen_parameter=8001
#s_https_listen_parameter=4444
s_temp=/t1/tst/fs2/inst/apps/tst_ed-olapplin2/temp
s_contextname=tst_ed-olapplin2
s_hostname=ed-olapplin2
s_domainname=samiora.blogspot.com
s_cphost=ed-olapplin2
s_webhost=ed-olapplin2
s_config_home=/t1/tst/fs2/inst/apps/tst_ed-olapplin2
s_inst_base=/t1/tst
s_display=ed-olapplin2:0.0
s_forms-c4ws_display=ed-olapplin2:0.0
s_ohs_instance=EBS_web_erp_OHS2
s_webport=8031
s_http_listen_parameter=8001
s_https_listen_parameter=4444
[Services]
# Please provide values for the context variables listed below .
# Enter "enabled" without the quotes to enable the service on the new node .
# Enter "disabled" without the quotes to disable the service on the new node .
# The Root service include the Node Manager .
# The Web Application Services include the Node Manager, Admin Server,
# Managed Servers ( oacore, forms, oafm, formsc4-ws).
# To enable the configuration of Node Manager and the Managed Servers,
# set s_web_applications_status to enabled
# The Web Entry Services include the OPMN and Oracle HTTP Server .
# To enable the configuration of OPMN,OHS , set s_webentry_status and s_apcstatus to enabled.
# The Batch Processing Services include the FNDFS Listener, Concurrent Mgr, ICSM# and JTF FullFillment Server
# To enable the configuration of Concurrent Manager# set s_batch_status to enabled.
# The Other Services group include the Forms Metric Server, Forms Metric Clients# Forms Server for the socket mode configuration and Mobile Web Application
# Server (MWA).
# s_adminserverstatus is set to disabled since this service can only be enabled
# on the primary application tier.
# The services enabled on the primary application tier node are as shown
# below in the comment section.
[Services Enabled on the Primary Application Tier Node]
#s_web_applications_status=enabled
#s_web_entry_status=enabled
#s_apcstatus=enabled
#s_root_status=enabled
#s_batch_status=enabled
#s_other_service_group_status=disabled
#s_adminserverstatus=enabled
#s_web_admin_status=enabled
[Services To be Enabled on the Secondary Application Tier Node]
s_web_applications_status=enabled
s_web_entry_status=enabled
s_apcstatus=enabled
s_root_status=enabled
s_batch_status=enabled
s_other_service_group_status=disabled
s_adminserverstatus=disabled
s_web_admin_status=enabled
# Additional variables can be added below as per your requirement.

8. This command adclonectx.pl needs to be run only 1 time on 2nd node without setting any environment.
Verify using the below SQL Script,
SQL> select * from APPS.FND_OAM_CONTEXT_FILES where name not in( 'METADATA','TEMPLATE') order by creation_date desc;If there is ERROR WHILE ADDING A NODE DUE TO any reason then use below command to DELETE THE NODE AND THEN ADD IT AGAIN.  Run this from node1,
[applmgr@ed-olapplin1 bin]$ cd /t1/tst/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/
[applmgr@ed-olapplin1 bin]$ perl adProvisionEBS.pl ebs-delete-node -contextfile=/t1/tst/fs2/inst/apps/tst_ed-olapplin1/appl/admin/tst_ed-olapplin1.xml -hostname=ed-olapplin2 -logfile=ebs-delete-node2_I.logAlso delete the inst directory that was created while we added the node for first time /t1/tst/fs2/inst/apps/tst_ed-olapplin2
Now Register the new topology from the newly added application tier node. Get port from Below file and change accordingly to below perl scripts.
[applmgr@ed-olapplin2]$ vi /t1/tst/fs2/FMW_Home/webtier/instances/EBS_web_tst_OHS2/config/OHS/EBS_web_tst/mod_wl_ohs.conf
[applmgr@ed-olapplin2]$. /t1/tst/fs2/EBSapps/appl/APPStst_ed-olapplin2.env
cd /t1/tst/fs2/EBSapps/comn/clone/bin
perl adclonectx.pl addnode contextfile=/t1/tst/fs2/inst/apps/tst_ed-olapplin1/appl/admin/tst_ed-olapplin1.xml pairsfile=/t1/tst/pairsfiles/mypairsfile.txt outfile=/t1/tst/pairsfile/mypairsOUTPUT.txt dualfs=yes
[applmgr@ed-olapplin2 EBS_web_tst]$ . /t1/tst/fs2/EBSapps/appl/APPStst_ed-olapplin2.env
perl /t1/tst/fs2/EBSapps/appl/fnd/12.0.0/patch/115/bin/txkSetAppsConf.pl -contextfile=/t1/tst/fs2/inst/apps/tst_ed-olapplin2/appl/admin/tst_ed-olapplin2.xml -configoption=removeMS -oacore=ed-olapplin2.samiora.blogspot.com:7202,ed-olapplin2.samiora.blogspot.com:7203,ed-olapplin2.samiora.blogspot.com:7204,ed-olapplin2.samiora.blogspot.com:7205 -oafm=ed-olapplin2.samiora.blogspot.com:7602 -forms=ed-olapplin2.samiora.blogspot.com:7402 -formsc4ws=ed-olapplin2.samiora.blogspot.com:7802 -ekanban=ed-olapplin2.samiora.blogspot.com:6802 -accessgate=ed-olapplin2.samiora.blogspot.com:6802 -yms=ed-olapplin2.samiora.blogspot.com:6802
perl /t1/tst/fs2/EBSapps/appl/fnd/12.0.0/patch/115/bin/txkSetAppsConf.pl -contextfile=/t1/tst/fs2/inst/apps/tst_ed-olapplin2/appl/admin/tst_ed-olapplin2.xml -configoption=addMS -oacore=ed-olapplin2.samiora.blogspot.com:7202,ed-olapplin2.samiora.blogspot.com:7205,ed-olapplin2.samiora.blogspot.com:7207,ed-olapplin2.samiora.blogspot.com:7209 -oafm=ed-olapplin2.samiora.blogspot.com:7602 -forms=ed-olapplin2.samiora.blogspot.com:7402 -formsc4ws=ed-olapplin2.samiora.blogspot.com:7802
more /t1/tst/fs2/FMW_Home/webtier/instances/EBS_web_tst_OHS2/config/OHS/EBS_web_tst/mod_wl_ohs.conf
perl $AD_TOP/patch/115/bin/adProvisionEBS.pl ebs-create-managedserver -contextfile=/t1/tst/fs2/inst/apps/tst_ed-olapplin2/appl/admin/tst_ed-olapplin2.xml -managedsrvname=oacore_server2 -servicetype=oacore -managedsrvport=7205 -logfile=/tmp/oacore2log.log
perl $AD_TOP/patch/115/bin/adProvisionEBS.pl ebs-create-managedserver -contextfile=/t1/tst/fs2/inst/apps/tst_ed-olapplin2/appl/admin/tst_ed-olapplin2.xml -managedsrvname=oacore_server5 -servicetype=oacore -managedsrvport=7209 -logfile=/tmp/oacore5log.log
perl $AD_TOP/patch/115/bin/adProvisionEBS.pl ebs-create-managedserver -contextfile=/t1/tst/fs2/inst/apps/tst_ed-olapplin2/appl/admin/tst_ed-olapplin2.xml -managedsrvname=oacore_server7 -servicetype=oacore -managedsrvport=7207 -logfile=/tmp/oacore7log.log
perl $AD_TOP/patch/115/bin/adProvisionEBS.pl ebs-create-managedserver -contextfile=/t1/tst/fs2/inst/apps/tst_ed-olapplin2/appl/admin/tst_ed-olapplin2.xml -managedsrvname=oacore_server9 -servicetype=oacore -managedsrvport=7202 -logfile=/tmp/oacore9log.log
for oacore_server9>>>ERROR: There is already a managed server with the name oacore_server9>>>>>>ignore this and continue below.
>>>now run autoconfig on node2 (ed-olapplin2)
>>>now without stopping node1 servies execute autoconfig on node 1 (ed-olapplin1)
>>>>start all services on NODE 2 (ed-olapplin2)
[applmgr@ed-olapplin2 ~]$ more $CONTEXT_FILE | grep oacore_server
      <oacore_server>
      </oacore_server>
         <oacore_server_ports oa_var="s_oacore_server_ports">oacore_server9:7202,oacore_server2:7205,oacore_server5:7209,oacore_server7:7207</oacore_server_ports>
            <oa_service_name oa_var="s_oacorename" type="managed_server">oacore_server9,oacore_server2,oacore_server5,oacore_server7</oa_service_name>
            <oa_managed_server_name oa_var="s_oacore_managed_servers">oacore_server9,oacore_server2,oacore_server5,oacore_server7</oa_managed_server_name>
>>>Below Additional command only to delete a managed server
perl $AD_TOP/patch/115/bin/adProvisionEBS.pl ebs-delete-managedserver -contextfile=/t1/tst/fs2/inst/apps/tst_ed-olapplin2/appl/admin/tst_ed-olapplin2.xml -managedsrvname=oacore_server2 -servicetype=oacore -logfile=/tmp/oacore_server1_remove.log


9. On all Application tier nodes, perform the following steps to register the newly added Application tier node with the Application tier TNS Listener (FNDFS listener) on each node:
On ed-olapplin2:
[applmgr@ed-olapplin2 ~]$ kill -9 -1
[applmgr@ed-olapplin2 ~]$ cd $ADMIN_SCRIPTS_HOME
[applmgr@ed-olapplin2 scripts]$ adautocfg.sh
[applmgr@ed-olapplin2 scripts]$ echo $TWO_TASK
[applmgr@ed-olapplin2 scripts]$ sh adalnctl.sh start
[applmgr@ed-olapplin2 admin]$ lsnrctl reload APPS_tst
[applmgr@ed-olapplin2 scripts]$ sh adalnctl.sh stop
On ed-olapplin1:
[applmgr@ed-olapplin1 tstuat]$ cd $ADMIN_SCRIPTS_HOME
[applmgr@ed-olapplin1 scripts]$ adautocfg.sh
[applmgr@ed-olapplin1 scripts]$ echo $TWO_TASK
[applmgr@ed-olapplin1 scripts]$ sh adalnctl.sh start
[applmgr@ed-olapplin1 admin]$ lsnrctl reload APPS_tst
[applmgr@ed-olapplin1 scripts]$ sh adalnctl.sh stop
10.If the Node Manager service is up on the Patch Edition File System of the newly added Application tier node, shut it down as follows:
[applmgr@ed-olapplin2 ~]$ . /t1/tst/fs1/EBSapps/appl/APPStst_ed-olapplin2.env
[applmgr@ed-olapplin2 ~]$ adnodemgrctl.sh stop
11. Shut down the Admin Server and the Node Manager on the Patch Edition File System of the primary node as follows:
[applmgr@ed-olapplin1 ~]$ . /t1/tst/fs1/EBSapps/appl/APPStst_ed-olapplin1.env
[applmgr@ed-olapplin1 ~]$ echo $FILE_EDITION
Patch
[applmgr@ed-olapplin1 ~]$ adadminsrvctl.sh stop forcepatchfs
[applmgr@ed-olapplin1 ~]$ adnodemgrctl.sh stop
[applmgr@ed-olapplin1 scripts]$ ps -ef | grep wlsport verify wls for patchfs is down.
12.Perform the following steps on all Database tier nodes to add the newly added node to the Access Control List
on Database server ed-olraclin-db:
[oracle@ed-olraclin-db~]$ cd /u01/oracle/12.2.0.1/tsthome
[oracle@ed-olraclin-dbtsthome1]$ . ed-olraclin-db.env
[oracle@ed-olraclin-dbtsthome1]$ cd appsutil/scripts/ed-olraclin-db/
[oracle@ed-olraclin-dbtst2_ed-olraclin-db]$ adautocfg.sh
[oracle@ed-olraclin-dbtst2_ed-olraclin-db]$ lsnrctl reload tst
13. Now Load balancer configuration for both URLs from two nodes
14. Now SSH Connectivity Requirements for Online Patching (Required)
# su - applmgr
# . /t1/tst/EBSapps.env RUN
On node 1 & node 2 both execute the below script respectively,
On ed-olapplin1# perl $AD_TOP/patch/115/bin/txkRunSSHSetup.pl enablessh -contextfile=/t1/tst/fs2/inst/apps/tst_ed-olapplin1/appl/admin/tst_ed-olapplin1.xml -hosts=ed-olapplin1,ed-olapplin2
On ed-olapplin2# perl $AD_TOP/patch/115/bin/txkRunSSHSetup.pl enablessh -contextfile=/t1/tst/fs2/inst/apps/tst_ed-olapplin2/appl/admin/tst_ed-olapplin2.xml -hosts=ed-olapplin2,ed-olapplin1
This finishes the ADDING NODE IN SHARED FILE SYSTEM SETUP.
Reference: Sharing the application tier file system in oracle EBS release 12.2. 1375769.1
For any questions on this article please email me on samiappsdba@gmail.com, and do subscribe to this blog as well.