Thursday, October 27, 2016

adop - Oracle EBS 12.2.x Online Patching utility

Applying Oracle E-Business Suite patches without a significant system downtime is referred to as online patching, and a new utility, adop, is used to apply patches.

Online patching is supported by the capability of storing multiple application editions in the database, and the provision of a dual application tier file system. At any given point in time, one of these file systems is designated as run (part of the running system) and the other as patch (either being patched or awaiting the start of the next patching cycle). Whichever is the current run file system appears to the user in exactly the same way as the single application tier file system did in Oracle E-Business Suite releases prior to 12.2.

It is more appropriate to think in terms of a patching cycle than a single patching operation. The online patching cycle consists of a number of phases:

    adop phase=prepare -> copies the application code
    adop phase=apply -> apply patches to PATCH Environment
    adop phase=finalize -> makes ready the system for cutover
    adop phase=cutover -> bounce the system and fs2 becomes RUN edition
    adop phase=cleanup -> remove obsolete objects.
    adop phase=fs_clone -> synchronize filesystems







The online patching cycle consists of five phases which are executed in order. Example of a typical online patching cycle:

source <ebs_root>/EBSapps.env run
adop phase=prepare
adop phase=apply patches=123456
adop phase=finalize
adop phase=cutover
source <ebs_root>/EBSapps.env run
adop phase=cleanup


Note that, after cutover the command line environment should be re-loaded as the run edition file system has changed.

In a multi-node deployment, adop commands are only executed from the primary node. The primary adop session uses remote execution to automatically perform required actions on any secondary node.

Multiple phases of adop can be executed in a single line command. Example of combined finalize/cutover/cleanup:

adop phase=finalize,cutover,cleanup

Prior to cutover, it is possible to execute additional “apply” and “finalize” phases as needed. Example of applying multiple patches using separate apply commands:

source <ebs_root>/EBSapps.env run
adop phase=prepare
adop phase=apply patches=123456
adop phase=apply patches=223456
adop phase=finalize
adop phase=apply patches=323456
adop phase=finalize
adop phase=cutover
source <ebs_root>/EBSapps.env run
adop phase=cleanup


Note that it is possible to apply additional patches after running the finalize phase, but if you do so then you will need to run the finalize phase again. Finalize must always be run immediately prior to cutover.


Use the below command at various stages to know the status of the adop phases,

# adop -status -detail


DEBUG ADOP
To debug Online Patching issues which utilize the adop (AD Online Patching) tool we need to collect adop log files located in
<INSTALL BASE>/fs_ne/EBSapps/log/adop directory


Below SQL will show you the status for each adop phase along with its corresponding session id. This is effectively a history of online patching in an environment
SQL> SELECT ADOP_SESSION_ID,PREPARE_STATUS,APPLY_STATUS,FINALIZE_STATUS,CUTOVER_STATUS,CLEANUP_STATUS,ABORT_STATUS,STATUS,ABANDON_FLAG,NODE_NAME from AD_ADOP_SESSIONS order by ADOP_SESSION_ID;


SQL> select * from AD_ZD_LOGS order by log_sequence desc;

Note:
       Y denotes that the phase is done
       N denotes that the phase has not been completed
       X denotes that the phase is not applicable
       R denotes that the phase is running (in progress)
       F denotes that the phase has failed
       P (is applicable only to APPLY phase) denotes at least one patch is already applied for the session id
       C denotes that the status of this ADOP session has completed

Note: Numerical statuses are only relevant for the cutover phase



To show which patches were applied in each ADOP_SESSION_ID (patching cycle)
SQL> select * from AD_ADOP_SESSION_PATCHES order by end_date desc;



SQL> SELECT ADOP_SESSION_ID, BUG_NUMBER, STATUS, APPLIED_FILE_SYSTEM_BASE, PATCH_FILE_SYSTEM_BASE, ADPATCH_OPTIONS, NODE_NAME, END_DATE, CLONE_STATUS
from ad_adop_session_patches
order by end_date desc;

Note: STATUS
N - Not Applied In the current node but applied in other nodes
R - Patch Application is going on.
H - Patch failed in the middle. (Hard Failure)
F - Patch failed in the middle but user tried to skip some failures.
S - Patch Application succeeded after skipping the failed jobs.
Y - Patch Application succeeded.
C - Reserved for clone and config_clone. Indicates clone completed


ADOP will not allow you to use adpatch from 12.2.x EBS versions. In some rare cases, only if Oracle Support advises then, set the below value and then use adpatch,

#export ENABLE_ADPATCH=YES

Then run adpatch for applying patches in 12.2. This helps in lower environments when you built only with one File System.


Additional Pointer - SSH should be enabled to allow communication from the primary node to the secondary nodes. Use txkRunSSHSetup.pl to enable SSH.



For more details on adop patching utility, refer to below oracle support knowledge base,

Oracle E-Business Suite 12.2 Patching Technology Components Guide (Doc ID 1355068.1)
Oracle E-Business Suite Release 12.2: Online Patching FAQ (Doc ID 1583902.1
How To Automatically Set the Current Run or Patch Edition / File System for EBS 12.2 (Doc ID 1545584.1)

For full Patching Procedures check this link: https://docs.oracle.com/cd/E26401_01/doc.122/e22954/T202991T531065.htm