Maintenance Mode -- EBS

Maintenance Mode is a new mode of operation introduced with Release 11.5.10, in which the Oracle Applications system is made accessible only for patching activities not allowing the users to login to any responsibility. This provides optimal performance for AutoPatch sessions, and minimizes downtime needed.

1. Scheduling System Downtime

Administrators can schedule 'System Downtime' using Oracle Applications Manager (OAM):

Site Map --> Maintenance --> Manage Downtime Schedules

When the System has been scheduled for 'Downtime', Apache should be re-started on Restricted Mode by using the Script (adaprstctl.sh).  By doing this, users attempting to log on to Oracle Applications will be automatically redirected to a System Downtime URL showing a message similar to the following one:

Scheduled Downtime Details

Start Time       : 17:30:00 12/11/2004
Expected Up Time : 09:00:00 12/12/2004
For Updates      : John.Smith@oracle.com

The system is currently undergoing a scheduled maintenance.

<Current Status>

This message can be customized with any text message.  If No Downtime has been specified, and the users try to access theApplications, the following message might also appear:

! Warning
The system has not been taken off maintenance mode completely.
Please contact your System Administrator.

2. Advantages

There are several practical points relating to the use of Maintenance Mode:

• You can toggle Maintenance Mode between Enabled and Disabled using the new Change Maintenance Mode menu in AD Administration, or the equivalent function in Oracle Applications Manager.

• Although you can run AutoPatch with Maintenance Mode disabled, there will be a significant degradation in performance.

• There is a separate logon page for Restricted Mode access while the system is in Maintenance Mode. Restricted Mode allows administrators access to specific privileged functionality in OAM, for example to view the timing report that shows the progress of a patching session.
  For more Information on Restricted Mode Access please consult Metalink Note: 364236.1
  or the OAM Online Help (OAM->Patches and Utilities -> Managing Downtime Schedules -> Restricted Mode)

3. Enabling and Disabling Maintenance Mode

Maintenance mode is Enabled or Disabled from adadmin.

When you Enable or Disable 'Maintenance Mode', adadmin will execute the script:

$AD_TOP/patch/115/sql/adsetmmd.sql sending the parameter 'ENABLE' or 'DISABLE' :

sqlplus <APPS_Schema name>/<APPS Password>@adsetmmd.sql ENABLE | DISABLE

    ENABLE  -   Enable Maintenance Mode .
    DISABLE -   Disable Maintenance Mode.

When adsetmmd.sql runs, it sets the Profile Option 'Applications Maintenance Mode' (APPS_MAINTENANCE_MODE) to 'MAINT' to Enable 'Maintenance Mode' and to 'NORMAL' to Disable it.

4.  Determining if Maintenance Mode is Running

A quick way to verify if the Environment is on Maintenance Mode or not, is by checking the value of this Profile Option as follows:

sqlplus apps/apps
SQL> select fnd_profile.value('APPS_MAINTENANCE_MODE') from dual;

If the query returns 'MAINT', then Maintenance Mode has been Enabled and the Users will not be able to Login.  If the query returns 'NORMAL' then Maintenance Mode has been De-Activated and the Users will be able to use the application.

Note:  Maintenance Mode is only needed for AutoPatch Sessions. Other AD utilities do not require Maintenance Mode to be enabled. Maintenance Mode must be 'Enabled' before running AutoPatch and 'Disabled' after the patch application was completed.

When Maintenance Mode is disabled, you can still run Autopatch by using options=hotpatch on the command line, if necessary.  However, doing so can cause a significant degradation of performance.

5. Error Messages

Always remember to Disable Maintenance Mode after any Patch application. If Maintenance Mode is not Disabled, the Application will not allow the users to use the system.  Take note that Apache must be re-started in normal mode after disabling 'Maintenance Mode' by using the Script adapcctl.sh (or adstrtal.sh)

As explained before, when 'Maintenance Mode' is enabled, a Downtime should be Scheduled from OAM and Apache should be started on Restricted Mode by using the Script (adaprstctl.sh).

If a 'DownTime' is not Scheduled from OAM and Apache has not been re-started on Restricted Mode, the Application will allow the users to Login, but it might experience unusual behaviors afterwards depending on the Patch Level.

Here are some examples of the possible error messages:

• When clicking on a Responsibility from the PHP

There are no applications available for this responsibility.  Please click on a different responsibility link to display the list of available applications.

or

You are not authorized to access the function Applications Home Page.  Please contact your System Administrator.

• When trying to access to the Application via CGI directly (not supported):

There are no valid navigations for this responsibility

Cause: The menu compilation has failed.
Cause: There is not valid menu defined for this responsibility.
Cause: There are no navigable forms associated with this responsibility.

Action: Contact your system administrator. Ensure that a valid menu,
containing navigable forms, is defined for the responsibility.
Ensure that the menu is correctly compiled.

Note:  In some cases, the behavior is slightly different.  Instead of showing the above messages, the Application might not show any Responsibilities listed for the user at all.

6. Step by Step Process

1.  Schedule the 'System Downtime' from OAM

OAM: Site Map --> Maintenance --> Manage Downtime Schedules

At the moment of the downtime, do the following:

2.  Shutdown Apache (on Normal Mode):

   adapcctl.sh stop
   or
   adstpall.sh <apps_user>/<apps_pwd>

3.  Enable 'Maintenance Mode' from adadmin

   adadmin: Options 5, 1

4. Start Apache (on Restricted Mode)

   adaprstctl.sh start

5. Apply the Patch with adpatch

6.  Stop Apache (on Restricted Mode)

  adaprstctl.sh stop

7.  Disable 'Maintenance Mode' from adadmin

   adadmin: Options 5, 2

8.  Start Apache (on Normal Mode):
 

  adapcctl.sh start
  or
  adstrtal.sh <apps_user>/<apps_pwd>

Unregister a Database From RMAN Recovery Catalog

There are multiple ways to unregister a database from an RMAN recovery catalog.

• UNREGISTER DATABASE (Catalog and Database)
• UNREGISTER DATABASE (Catalog Only)
• DBMS_RCVCAT (Catalog Only)

UNREGISTER DATABASE (Catalog and Database)

This option is available from Oracle 10g onward. If you still have access to the database you can start RMAN, connecting to both the target database and the catalog.

rman target=sys/password@TEST catalog=rman/rman@catdb

You may wish to perform some clean-up first, like deleting the existing backups.

RMAN> LIST BACKUP SUMMARY;

RMAN> DELETE BACKUP DEVICE TYPE SBT;

RMAN> DELETE BACKUP DEVICE TYPE DISK;

When ready, use the UNREGISTER DATABASE command.

RMAN> UNREGISTER DATABASE;

OR

RMAN> UNREGISTER DATABASE NOPROMPT;

UNREGISTER DATABASE (Catalog Only)

This option is available from Oracle 10g onward. If you no longer have access to the target database, you can still unregister it from the catalog using the UNREGISTER DATABASE command in RMAN.

Start RMAN, connecting only to the catalog.

rman catalog=rman/rman@catdb

Unregister the database by name.

RMAN> UNREGISTER DATABASE TEST NOPROMPT;

If there is more than one database in the catalog with the same name, you will need to use the DBID to identify the database to unregister. You can find this using the LIST INCARNATION command.

RMAN> LIST INCARNATION OF DATABASE TEST;

Once you have the DBID, you can unregister the database using the following script.

RUN

{ 

  SET DBID 1312293510;

  UNREGISTER DATABASE TEST NOPROMPT;

}

DBMS_RCVCAT (Catalog Only)

If you no longer have access to the target database, you can still unregister it from the catalog using the DBMS_RCVCAT package in SQL.

Connect to the catalog database using SQL*Plus, then query the DB_KEY and DBID values as follows.

SQL> CONNECT rman/rman@catdb

Connected.

SQL> SELECT db_key, dbid, name FROM rc_database WHERE name = 'TEST';

    DB_KEY       DBID NAME

---------- ---------- --------

     23085 1312293510 TEST

1 row selected.

SQL>

The resulting DB_KEY and DBID can then be used to unregister the database using the DBMS_RCVCAT package.

SQL> EXECUTE dbms_rcvcat.unregisterdatabase(23085 , 1312293510);

PL/SQL procedure successfully completed.

SQL>

RPM command with examples

RPM (Red Hat Package Manager) command is used for installing, uninstalling, upgrading, querying, listing, and checking RPM packages on your Linux system.

With root privilege, you can use the rpm command with appropriate options to manage the RPM software packages.

Let us take an rpm of Mysql Client and run through all our examples.

1. Installing a RPM package Using rpm -ivh

RPM file name has package name, version, release and architecture name.

For example, In the MySQL-client-3.23.57-1.i386.rpm file:

• MySQL-client – Package Name
• 3.23.57 – Version
• 1 – Release
• i386 – Architecture

When you install a RPM, it checks whether your system is suitable for the software the RPM package contains, figures out where to install the files located inside the rpm package, installs them on your system, and adds that piece of software into its database of installed RPM packages.

The following rpm command installs Mysql client package.

# rpm -ivh  MySQL-client-3.23.57-1.i386.rpm

Preparing...                ########################################### [100%]

   1:MySQL-client           ########################################### [100%]

rpm command and options

• -i : install a package
• -v : verbose
• -h : print hash marks as the package archive is unpacked.

2. Installing a RPM package without dependency 

Some time we are not able to install RPM due to dependency issue. one RPM is dependent on other. so here is solution, install RPM without checking dependency.

rpm --nodeps -ivh MySQL-client-3.23.57-1.i386.rpm

3. Query all the RPM Packages using rpm -qa

You can use rpm command to query all the packages installed in your system.

# rpm -qa

cdrecord-2.01-10.7.el5

bluez-libs-3.7-1.1

setarch-2.0-1.1

• -q query operation
• -a queries all installed packages

To identify whether a particular rpm package is installed on your system, combine rpm and grep command as shown below. Following command checks whether cdrecord package is installed on your system.

# rpm -qa | grep 'cdrecord'

4. Query a Particular RPM Package using rpm -q

The above example lists all currently installed package. After installation of a package to check the installation, you can query a particular package and verify as shown below.

# rpm -q MySQL-client

MySQL-client-3.23.57-1

# rpm -q MySQL

package MySQL is not installed

Note: To query a package, you should specify the exact package name. If the package name is incorrect, then rpm command will report that the package is not installed.

5. Query RPM Packages in a various format using rpm –query format

Rpm command provides an option –query format, which allows you to give the header tag names, to list the packages. Enclose the header tag with in {}.

# rpm -qa --queryformat '%{name-%{version}-%{release} %{size}\n'

cdrecord-2.01-10.7 12324

bluez-libs-3.7-1.1 5634

setarch-2.0-1.1 235563

6. Which RPM package does a file belong to? – Use rpm -qf

Let us say, you have list of files and you would want to know which package owns all these files. rpm command has options to achieve this.

The following example shows that /usr/bin/mysqlaccess file is part of the MySQL-client-3.23.57-1 rpm.

# rpm -qf /usr/bin/mysqlaccess

MySQL-client-3.23.57-1

• -f : file name

7. Locate documentation of a package that owns file using rpm -qdf

Use the following to know the list of documentations, for a package that owns a file. The following command, gives the location of all the manual pages related to mysql package.

# rpm -qdf /usr/bin/mysqlaccess

/usr/share/man/man1/mysql.1.gz

/usr/share/man/man1/mysqlaccess.1.gz

/usr/share/man/man1/mysqladmin.1.gz

/usr/share/man/man1/mysqldump.1.gz

/usr/share/man/man1/mysqlshow.1.gz

• -d : refers documentation.

8. Information about Installed RPM Package using rpm -qi

rpm command provides a lot of information about an installed package using rpm -qi as shown below:

# rpm -qi MySQL-client

Name        : MySQL-client                 Relocations: (not relocatable)

Version     : 3.23.57                           Vendor: MySQL AB

Release     : 1                             Build Date: Mon 09 Jun 2003 11:08:28 PM CEST

Install Date: Mon 06 Feb 2010 03:19:16 AM PST               Build Host: build.mysql.com

Group       : Applications/Databases        Source RPM: MySQL-3.23.57-1.src.rpm

Size        : 5305109                          License: GPL / LGPL

Signature   : (none)

Packager    : Lenz Grimmer

URL         : http://www.mysql.com/

Summary     : MySQL - Client

Description : This package contains the standard MySQL clients.

If you have an RPM file that you would like to install, but want to know more information about it before installing, you can do the following:

# rpm -qip MySQL-client-3.23.57-1.i386.rpm

Name        : MySQL-client                 Relocations: (not relocatable)

Version     : 3.23.57                           Vendor: MySQL AB

Release     : 1                             Build Date: Mon 09 Jun 2003 11:08:28 PM CEST

Install Date: (not installed)               Build Host: build.mysql.com

Group       : Applications/Databases        Source RPM: MySQL-3.23.57-1.src.rpm

Size        : 5305109                          License: GPL / LGPL

Signature   : (none)

Packager    : Lenz Grimmer

URL         : http://www.mysql.com/

Summary     : MySQL - Client

Description : This package contains the standard MySQL clients.

• -i : view information about an rpm
• -p : specify a package name

9. List all the Files in a Package using rpm -qlp

To list the content of a RPM package, use the following command, which will list out the files without extracting into the local directory folder.

$ rpm -qlp ovpc-2.1.10.rpm

/usr/bin/mysqlaccess

/usr/bin/mysqldata

/usr/bin/mysqlperm

.

.

/usr/bin/mysqladmin

• q : query the rpm file
• l : list the files in the package
• p : specify the package name

10. List the Dependency Packages using rpm -qRP

To view the list of packages on which this package depends,

# rpm -qRp MySQL-client-3.23.57-1.i386.rpm

/bin/sh

/usr/bin/perl

11. Find out the state of files in a package using rpm -qsp

The following command is to find state (installed, replaced or normal) for all the files in a RPM package.

# rpm -qsp MySQL-client-3.23.57-1.i386.rpm

normal        /usr/bin/msql2mysql

normal        /usr/bin/mysql

normal        /usr/bin/mysql_find_rows

normal        /usr/bin/mysqlaccess

normal        /usr/bin/mysqladmin

normal        /usr/bin/mysqlbinlog

normal        /usr/bin/mysqlcheck

normal        /usr/bin/mysqldump

normal        /usr/bin/mysqlimport

normal        /usr/bin/mysqlshow

normal        /usr/share/man/man1/mysql.1.gz

normal        /usr/share/man/man1/mysqlaccess.1.gz

normal        /usr/share/man/man1/mysqladmin.1.gz

normal        /usr/share/man/man1/mysqldump.1.gz

normal        /usr/share/man/man1/mysqlshow.1.gz

12. Verify a Particular RPM Package using rpm -Vp

Verifying a package compares information about the installed files in the package with information about the files taken from the package metadata stored in the rpm database. In the following command, -V is for verification and -p option is used to specify a package name to verify.

# rpm -Vp MySQL-client-3.23.57-1.i386.rpm

S.5....T c        /usr/bin/msql2mysql

S.5....T c        /usr/bin/mysql

S.5....T c        /usr/bin/mysql_find_rows

S.5....T c        /usr/bin/mysqlaccess

The character in the above output denotes the following:

• S file Size differs
• M Mode differs (includes permissions and file type)
• 5 MD5 sum differs
• D Device major/minor number mismatch
• L readlink(2) path mismatch
• U User ownership differs
• G Group ownership differs
• T mTime differs

13. Verify a Package Owning file using rpm -Vf

The following command verify the package which owns the given filename.

# rpm -Vf /usr/bin/mysqlaccess

S.5....T c /usr/bin/mysql

14. Upgrading a RPM Package using rpm -Uvh

Upgrading a package is similar to installing one, but RPM automatically un-installs existing versions of the package before installing the new one. If an old version of the package is not found, the upgrade option will still install it.

# rpm -Uvh MySQL-client-3.23.57-1.i386.rpm

Preparing...                   ########################################### [100%]

1:MySQL-client          ###########################################

15. Uninstalling a RPM Package using rpm -e

To remove an installed rpm package using -e as shown below. After uninstall, you can query using rpm -qa and verify the uninstall.

# rpm -ev MySQL-client

16. Verifying all the RPM Packages using rpm -Va

The following command verifies all the installed packages.

# rpm -Va

S.5....T c /etc/issue

S.5....T c /etc/issue.net

S.5....T c /var/service/imap/ssl/seed

S.5....T c /home/httpd/html/horde/ingo/config/backends.php

.

.

S.5....T c /home/httpd/html/horde/ingo/config/prefs.php

S.5....T c /etc/printcap