下面是测试11.2.0.3滚动升级到11.2.0.3.10，原来做过很多次，但是都没有写过文档。整个升级包括2个部分，1，升级前的准备工作;2，升级过程。更多的升级方案见：
环境介绍
SQL> select * from v$version;
BANNER
--------------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
PL/SQL Release 11.2.0.3.0 - Production
CORE 11.2.0.3.0 Production
TNS for Linux: Version 11.2.0.3.0 - Production
NLSRTL Version 11.2.0.3.0 - Production
SQL> !lsb_release -a
LSB Version: :core-4.0-amd64:core-4.0-ia32:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-ia32:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-ia32:printing-4.0-noarch
Distributor ID: EnterpriseEnterpriseServer
Description: Enterprise Linux Enterprise Linux Server release 5.8 (Carthage)
Release: 5.8
Codename: Carthage
1，升级前的准备工作
1.1 安装最新版本opatch工具
需要在所有的节点安装最新版本的OPATCH工具，安装前可以对老版本的OPATCH进行备份
oracle@11rac1 soft]$ unzip p6880880_112000_Linux-x86-64.zip -d $ORACLE_HOME
Archive: p6880880_112000_Linux-x86-64.zip
replace /u01/app/oracle/product/11.2.0/db_1/OPatch/jlib/oracle.opatch.classpath.jar? [y]es, [n]o, [A]ll, [N]one, [r]ename: A
inflating: /u01/app/oracle/product/11.2.0/db_1/OPatch/jlib/oracle.opatch.classpath.jar
inflating: /u01/app/oracle/product/11.2.0/db_1/OPatch/jlib/oracle.opatch.classpath.windows.jar
[grid@11rac1 soft]$ unzip p6880880_112000_Linux-x86-64.zip -d $ORACLE_HOME
Archive: p6880880_112000_Linux-x86-64.zip
replace /u01/app/11.2.0/grid/OPatch/jlib/oracle.opatch.classpath.jar? [y]es, [n]o, [A]ll, [N]one, [r]ename: A
inflating: /u01/app/11.2.0/grid/OPatch/jlib/oracle.opatch.classpath.jar
inflating: /u01/app/11.2.0/grid/OPatch/jlib/oracle.opatch.classpath.windows.jar
1.2 解压PSU包
需要在所有的节点操作，这里解压到/tmp目录下，生产环境操作，不建议解压到/TMP目录，MOS有部分AUTO报错的原因是补丁包解压到/TMP导致的
[grid@11rac1 soft]$ unzip p18139678_112030_Linux-x86-64.zip -d /tmp/patch/
1.3 验证补丁包的冲突
所有在所有的节点执行下面的操作，另外还需要注意的是AIX平台需要目录剩余空间达到20G。
racle Interim Patch Installer version 11.2.0.3.6
Copyright (c) 2013, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home : /u01/app/oracle/product/11.2.0/db_1
Central Inventory : /u01/app/oraInventory
from : /u01/app/oracle/product/11.2.0/db_1/oraInst.loc
OPatch version : 11.2.0.3.6
OUI version : 11.2.0.3.0
Log file location : /u01/app/oracle/product/11.2.0/db_1/cfgtoollogs/opatch/opatch2014-10-13_08-00-41AM_1.log
Invoking prereq "checkcomponents"
Prereq "checkComponents" for patch 17592127 passed.
OPatch succeeded.
[oracle@11rac1 18031683]$ $ORACLE_HOME/OPatch/opatch prereq CheckComponents -phBaseDir /tmp/patch/18031683 -oh $ORACLE_HOME
Oracle Interim Patch Installer version 11.2.0.3.6
Copyright (c) 2013, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home : /u01/app/oracle/product/11.2.0/db_1
Central Inventory : /u01/app/oraInventory
from : /u01/app/oracle/product/11.2.0/db_1/oraInst.loc
OPatch version : 11.2.0.3.6
OUI version : 11.2.0.3.0
Log file location : /u01/app/oracle/product/11.2.0/db_1/cfgtoollogs/opatch/opatch2014-10-13_08-01-26AM_1.log
Invoking prereq "checkcomponents"
Prereq "checkComponents" for patch 13343438 passed.
Prereq "checkComponents" for patch 13696216 passed.
Patch 13923374: Optional component(s) missing : [ oracle.network.cman, 11.2.0.3.0 ]
Prereq "checkComponents" for patch 13923374 passed.
Patch 14275605: Optional component(s) missing : [ oracle.precomp.lang, 11.2.0.3.0 ]
Prereq "checkComponents" for patch 14275605 passed.
Prereq "checkComponents" for patch 14727310 passed.
Patch 16056266: Optional component(s) missing : [ oracle.network.cman, 11.2.0.3.0 ]
Prereq "checkComponents" for patch 16056266 passed.
Patch 16619892: Optional component(s) missing : [ oracle.precomp.lang, 11.2.0.3.0 ]
Prereq "checkComponents" for patch 16619892 passed.
Patch 16902043: Optional component(s) missing : [ oracle.idm.oid, 11.2.0.3.0 ]
Prereq "checkComponents" for patch 16902043 passed.
Patch 17540582: Optional component(s) missing : [ oracle.precomp.lang, 11.2.0.3.0 ]
Prereq "checkComponents" for patch 17540582 passed.
Patch 18031683: Optional component(s) missing : [ oracle.precomp.lang, 11.2.0.3.0 ]
Prereq "checkComponents" for patch 18031683 passed.
OPatch succeeded.
[oracle@11rac1 18031683]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOH -phBaseDir /tmp/patch/17592127/custom/server/17592127 -oh $ORACLE_HOME
Oracle Interim Patch Installer version 11.2.0.3.6
Copyright (c) 2013, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home : /u01/app/oracle/product/11.2.0/db_1
Central Inventory : /u01/app/oraInventory
from : /u01/app/oracle/product/11.2.0/db_1/oraInst.loc
OPatch version : 11.2.0.3.6
OUI version : 11.2.0.3.0
Log file location : /u01/app/oracle/product/11.2.0/db_1/cfgtoollogs/opatch/opatch2014-10-13_08-04-24AM_1.log
Invoking prereq "checkconflictagainstoh"
Prereq "checkConflictAgainstOH" passed.
OPatch succeeded.
[oracle@11rac1 18031683]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOH -phBaseDir /tmp/patch/18031683 -oh $ORACLE_HOME
Oracle Interim Patch Installer version 11.2.0.3.6
Copyright (c) 2013, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home : /u01/app/oracle/product/11.2.0/db_1
Central Inventory : /u01/app/oraInventory
from : /u01/app/oracle/product/11.2.0/db_1/oraInst.loc
OPatch version : 11.2.0.3.6
OUI version : 11.2.0.3.0
Log file location : /u01/app/oracle/product/11.2.0/db_1/cfgtoollogs/opatch/opatch2014-10-13_08-04-48AM_1.log
Invoking prereq "checkconflictagainstoh"
Prereq "checkConflictAgainstOH" passed.
OPatch succeeded.
验证GRID
[oracle@11rac1 18031683]$ su - grid
Password:
[grid@11rac1 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckComponents -phBaseDir /tmp/patch/17592127 -oh $ORACLE_HOME
Oracle Interim Patch Installer version 11.2.0.3.6
Copyright (c) 2013, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home : /u01/app/11.2.0/grid
Central Inventory : /u01/app/oraInventory
from : /u01/app/11.2.0/grid/oraInst.loc
OPatch version : 11.2.0.3.6
OUI version : 11.2.0.3.0
Log file location : /u01/app/11.2.0/grid/cfgtoollogs/opatch/opatch2014-10-13_08-21-52AM_1.log
Invoking prereq "checkcomponents"
Prereq "checkComponents" for patch 17592127 passed.
OPatch succeeded.
[grid@11rac1 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckComponents -phBaseDir /tmp/patch/18031683 -oh $ORACLE_HOME
Oracle Interim Patch Installer version 11.2.0.3.6
Copyright (c) 2013, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home : /u01/app/11.2.0/grid
Central Inventory : /u01/app/oraInventory
from : /u01/app/11.2.0/grid/oraInst.loc
OPatch version : 11.2.0.3.6
OUI version : 11.2.0.3.0
Log file location : /u01/app/11.2.0/grid/cfgtoollogs/opatch/opatch2014-10-13_08-22-03AM_1.log
Invoking prereq "checkcomponents"
Prereq "checkComponents" for patch 13343438 passed.
Patch 13696216: Optional component(s) missing : [ oracle.sysman.console.db, 11.2.0.3.0 ] , [ oracle.sysman.oms.core, 10.2.0.4.4 ]
Prereq "checkComponents" for patch 13696216 passed.
Patch 13923374: Optional component(s) missing : [ oracle.sysman.console.db, 11.2.0.3.0 ] , [ oracle.network.cman, 11.2.0.3.0 ]
Prereq "checkComponents" for patch 13923374 passed.
Patch 14275605: Optional component(s) missing : [ oracle.precomp.common, 11.2.0.3.0 ] , [ oracle.precomp.lang, 11.2.0.3.0 ]
Prereq "checkComponents" for patch 14275605 passed.
Patch 14727310: Optional component(s) missing : [ oracle.sysman.console.db, 11.2.0.3.0 ] , [ oracle.sysman.oms.core, 10.2.0.4.4 ]
Prereq "checkComponents" for patch 14727310 passed.
Patch 16056266: Optional component(s) missing : [ oracle.network.cman, 11.2.0.3.0 ] , [ oracle.ovm, 11.2.0.3.0 ]
Prereq "checkComponents" for patch 16056266 passed.
Patch 16619892: Optional component(s) missing : [ oracle.marvel, 11.2.0.3.0 ] , [ oracle.precomp.common, 11.2.0.3.0 ] , [ oracle.precomp.lang, 11.2.0.3.0 ] , [ oracle.sysman.agent, 10.2.0.4.3 ] , [ oracle.sysman.console.db, 11.2.0.3.0 ] , [ oracle.sysman.repository.core, 10.2.0.4.4 ] , [ oracle.xdk, 11.2.0.3.0 ]
Prereq "checkComponents" for patch 16619892 passed.
Patch 16902043: Optional component(s) missing : [ oracle.idm.oid, 11.2.0.3.0 ] , [ oracle.owb.rsf, 11.2.0.3.0 ] , [ oracle.sysman.console.db, 11.2.0.3.0 ]
Prereq "checkComponents" for patch 16902043 passed.
Patch 17540582: Optional component(s) missing : [ oracle.sdo, 11.2.0.3.0 ] , [ oracle.precomp.common, 11.2.0.3.0 ] , [ oracle.precomp.lang, 11.2.0.3.0 ]
Prereq "checkComponents" for patch 17540582 passed.
Patch 18031683: Optional component(s) missing : [ oracle.precomp.common, 11.2.0.3.0 ] , [ oracle.precomp.lang, 11.2.0.3.0 ]
Prereq "checkComponents" for patch 18031683 passed.
OPatch succeeded.
[grid@11rac1 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOH -phBaseDir /tmp/patch/17592127 -oh $ORACLE_HOME
Oracle Interim Patch Installer version 11.2.0.3.6
Copyright (c) 2013, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home : /u01/app/11.2.0/grid
Central Inventory : /u01/app/oraInventory
from : /u01/app/11.2.0/grid/oraInst.loc
OPatch version : 11.2.0.3.6
OUI version : 11.2.0.3.0
Log file location : /u01/app/11.2.0/grid/cfgtoollogs/opatch/opatch2014-10-13_08-22-09AM_1.log
Invoking prereq "checkconflictagainstoh"
Prereq "checkConflictAgainstOH" passed.
OPatch succeeded.
[grid@11rac1 ~]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOH -phBaseDir /tmp/patch/18031683 -oh $ORACLE_HOME
Oracle Interim Patch Installer version 11.2.0.3.6
Copyright (c) 2013, Oracle Corporation. All rights reserved.
PREREQ session
Oracle Home : /u01/app/11.2.0/grid
Central Inventory : /u01/app/oraInventory
from : /u01/app/11.2.0/grid/oraInst.loc
OPatch version : 11.2.0.3.6
OUI version : 11.2.0.3.0
Log file location : /u01/app/11.2.0/grid/cfgtoollogs/opatch/opatch2014-10-13_08-22-21AM_1.log
Invoking prereq "checkconflictagainstoh"
Prereq "checkConflictAgainstOH" passed.
OPatch succeeded.
1.3 生成OCM文件
[oracle@11rac1 ~]$ $ORACLE_HOME/OPatch/ocm/bin/emocmrsp -no_banner -output /tmp/ocm.rsp
Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:
You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]: Y
The OCM configuration response file (/tmp/ocm.rsp) was successfully created.
2 开始PSU升级
2.1 备份GRID_HOME/ORACLE_HOME
这里略
2.2 切换服务
[oracle@11rac1 ~]$ srvctl status service -d power
Service power1 is running on instance(s) power1
Service power2 is running on instance(s) power2
Service powera is running on instance(s) power1
Service powerb is running on instance(s) power2
[oracle@11rac1 ~]$ srvctl relocate service -d power -s power1 -i power1 -t power2
[oracle@11rac1 ~]$ srvctl relocate service -d power -s powera -i power1 -t power2
[oracle@11rac1 ~]$ srvctl status service -d power
Service power1 is running on instance(s) power2
Service power2 is running on instance(s) power2
Service powera is running on instance(s) power2
Service powerb is running on instance(s) power2
切换服务后，需要开发商确认服务可以正常连接。
2.3 停数据库实例
SQL> alter system switch logfile;
System altered.
SQL> alter system switch logfile;
SQL> alter system checkpoint;
System altered.
kill进程
ps -ef|grep LOCAL=NO|grep power|grep -v grep|awk '{print $2}'|xargs kill -9
SQL> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
2.4 开始升级
升级过程大概需要30分钟，如果auto失败可以需要手动运行命令
[root@11rac1 tmp]# /u01/app/11.2.0/grid/OPatch/opatch auto /tmp/patch/ -ocmrf /tmp/ocm.rsp
Executing /u01/app/11.2.0/grid/perl/bin/perl /u01/app/11.2.0/grid/OPatch/crs/patch11203.pl -patchdir /tmp -patchn patch -ocmrf /tmp/ocm.rsp -paramfile /u01/app/11.2.0/grid/crs/install/crsconfig_params
This is the main log file: /u01/app/11.2.0/grid/cfgtoollogs/opatchauto2014-10-13_23-37-55.log
This file will show your detected configuration and all the steps that opatchauto attempted to do on your system:
/u01/app/11.2.0/grid/cfgtoollogs/opatchauto2014-10-13_23-37-55.report.log
2014-10-13 23:37:55: Starting Clusterware Patch Setup
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Stopping RAC /u01/app/oracle/product/11.2.0/db_1 ...
Stopped RAC /u01/app/oracle/product/11.2.0/db_1 successfully
patch /tmp/patch/17592127/custom/server/17592127 apply successful for home /u01/app/oracle/product/11.2.0/db_1
patch /tmp/patch/18031683 apply successful for home /u01/app/oracle/product/11.2.0/db_1
Stopping CRS...
Stopped CRS successfully
patch /tmp/patch/17592127 apply successful for home /u01/app/11.2.0/grid
patch /tmp/patch/18031683 apply successful for home /u01/app/11.2.0/grid
Starting CRS...
CRS-4123: Oracle High Availability Services has been started.
Starting RAC /u01/app/oracle/product/11.2.0/db_1 ...
Started RAC /u01/app/oracle/product/11.2.0/db_1 successfully
opatch auto succeeded.
可以在$ORACLE_HOME/cfgtoollogs中找到命令的输出日志，如果需要生成 更加详细的日志，可以配置OPATCH_DEBUG=TRUE
opatchauto2014-10-13_23-37-55.report.log文件记录了AUTO执行的每一条命令。
opatchauto2014-10-13_23-37-55.log记录了详细的日志文件
2.5 启动数据库与切换服务
SQL> startup
ORACLE instance started.
Total System Global Area 1043886080 bytes
Fixed Size 2234960 bytes
Variable Size 750781872 bytes
Database Buffers 285212672 bytes
Redo Buffers 5656576 bytes
Database mounted.
Database opened.
SQL>
[oracle@11rac1 ~]$ srvctl status service -d power
Service power1 is running on instance(s) power2
Service power2 is running on instance(s) power2
Service powera is running on instance(s) power2
Service powerb is running on instance(s) power2
[oracle@11rac1 ~]$ srvctl relocate service -d power -s power1 -i power2 -t power1
[oracle@11rac1 ~]$ srvctl relocate service -d power -s powera -i power2 -t power1
[oracle@11rac1 ~]$ srvctl status service -d power
Service power1 is running on instance(s) power1
Service power2 is running on instance(s) power2
Service powera is running on instance(s) power1
Service powerb is running on instance(s) power2
到目前，第一个节点已经升级完成，2节点按1节点的方法升级就可以了。
2.6 升级数据字典
cd $ORACLE_HOME/rdbms/admin
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> STARTUP
SQL> @catbundle.sql psu apply
SQL> QUIT
2.7 升级rman
$ rman catalog username/password@alias
RMAN> UPGRADE CATALOG;
3 回退方案
下面使用的auto来自动回退，后面给出执行的语句
[root@11rac2 ~]# /u01/app/11.2.0/grid/OPatch/opatch auto /tmp/patch -rollback -ocmrf /tmp/ocm.rsp
Executing /u01/app/11.2.0/grid/perl/bin/perl /u01/app/11.2.0/grid/OPatch/crs/patch11203.pl -patchdir /tmp -patchn patch -rollback -ocmrf /tmp/ocm.rsp -paramfile /u01/app/11.2.0/grid/crs/install/crsconfig_params
This is the main log file: /u01/app/11.2.0/grid/cfgtoollogs/opatchauto2014-10-14_06-20-58.log
This file will show your detected configuration and all the steps that opatchauto attempted to do on your system:
/u01/app/11.2.0/grid/cfgtoollogs/opatchauto2014-10-14_06-20-58.report.log
2014-10-14 06:20:58: Starting Clusterware Patch Setup
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Stopping RAC /u01/app/oracle/product/11.2.0/db_1 ...
Stopped RAC /u01/app/oracle/product/11.2.0/db_1 successfully
patch 17592127 rollback successful for home /u01/app/oracle/product/11.2.0/db_1
patch 18031683 rollback successful for home /u01/app/oracle/product/11.2.0/db_1
Stopping CRS...
Stopped CRS successfully
patch 17592127 rollback successful for home /u01/app/11.2.0/grid
patch 18031683 rollback successful for home /u01/app/11.2.0/grid
Starting CRS...
CRS-4123: Oracle High Availability Services has been started.
Starting RAC /u01/app/oracle/product/11.2.0/db_1 ...
Started RAC /u01/app/oracle/product/11.2.0/db_1 successfully
opatch auto succeeded.
[oracle@11rac2 sql]$ cat /u01/app/11.2.0/grid/cfgtoollogs/opatchauto2014-10-14_06-20-58.report.log
*********** Configuration Data ***********
* It shows only those targets that will be patched in this session *
crs_home=/u01/app/11.2.0/grid owner=grid opatch_ver=11.2.0.3.6
rac_home=/u01/app/oracle/product/11.2.0/db_1 owner=oracle opatch_ver=11.2.0.3.6
*********** Steps to be executed as owner unless specified as root ***********
1: /u01/app/oracle/product/11.2.0/db_1/OPatch/opatch prereq CheckComponents -ph /tmp/patch/17592127/custom/server/17592127 -oh /u01/app/oracle/product/11.2.0/db_1
2: /u01/app/oracle/product/11.2.0/db_1/OPatch/opatch prereq CheckComponents -ph /tmp/patch/18031683 -oh /u01/app/oracle/product/11.2.0/db_1
3: /u01/app/11.2.0/grid/OPatch/opatch prereq CheckComponents -ph /tmp/patch/17592127 -oh /u01/app/11.2.0/grid
4: /u01/app/11.2.0/grid/OPatch/opatch prereq CheckComponents -ph /tmp/patch/18031683 -oh /u01/app/11.2.0/grid
5: /u01/app/oracle/product/11.2.0/db_1/bin/emctl stop dbconsole
6: /u01/app/oracle/product/11.2.0/db_1/bin/emctl stop agent
7: /u01/app/oracle/product/11.2.0/db_1/OPatch/opatch prereq CheckApplicable -ph /tmp/patch/17592127/custom/server/17592127 -oh /u01/app/oracle/product/11.2.0/db_1
8: /u01/app/oracle/product/11.2.0/db_1/OPatch/opatch prereq CheckApplicable -ph /tmp/patch/18031683 -oh /u01/app/oracle/product/11.2.0/db_1
9: /u01/app/oracle/product/11.2.0/db_1/bin/srvctl stop home -o /u01/app/oracle/product/11.2.0/db_1 -s /u01/app/oracle/product/11.2.0/db_1/srvm/admin/stophome.txt -n 11rac2
10: /sbin/fuser -k /u01/app/oracle/product/11.2.0/db_1/bin/oracle : run as root
11: /tmp/patch/17592127/custom/server/17592127/custom/scripts/prepatch.sh -dbhome /u01/app/oracle/product/11.2.0/db_1
12: /u01/app/oracle/product/11.2.0/db_1/OPatch/opatch prereq checkRollbackable -id 17592127 -oh /u01/app/oracle/product/11.2.0/db_1
13: /u01/app/oracle/product/11.2.0/db_1/OPatch/opatch rollback -id 17592127 -local -silent -oh /u01/app/oracle/product/11.2.0/db_1 -invPtrLoc /u01/app/oracle/product/11.2.0/db_1/oraInst.loc
14: /u01/app/oracle/product/11.2.0/db_1/OPatch/opatch prereq checkRollbackable -id 18031683 -oh /u01/app/oracle/product/11.2.0/db_1
15: /u01/app/oracle/product/11.2.0/db_1/OPatch/opatch rollback -id 18031683 -local -silent -oh /u01/app/oracle/product/11.2.0/db_1 -invPtrLoc /u01/app/oracle/product/11.2.0/db_1/oraInst.loc
16: /tmp/patch/17592127/custom/server/17592127/custom/scripts/postpatch.sh -dbhome /u01/app/oracle/product/11.2.0/db_1
17: /u01/app/11.2.0/grid/crs/install/rootcrs.pl -unlock : run as root
18: /sbin/fuser -k /u01/app/11.2.0/grid/bin/crsctl.bin : run as root
19: /u01/app/11.2.0/grid/OPatch/opatch prereq CheckApplicable -ph /tmp/patch/17592127 -oh /u01/app/11.2.0/grid
20: /u01/app/11.2.0/grid/OPatch/opatch prereq CheckApplicable -ph /tmp/patch/18031683 -oh /u01/app/11.2.0/grid
21: /u01/app/11.2.0/grid/OPatch/opatch prereq checkRollbackable -id 17592127 -oh /u01/app/11.2.0/grid
22: /u01/app/11.2.0/grid/OPatch/opatch rollback -id 17592127 -local -silent -oh /u01/app/11.2.0/grid -invPtrLoc /u01/app/11.2.0/grid/oraInst.loc
23: /u01/app/11.2.0/grid/OPatch/opatch prereq checkRollbackable -id 18031683 -oh /u01/app/11.2.0/grid
24: /u01/app/11.2.0/grid/OPatch/opatch rollback -id 18031683 -local -silent -oh /u01/app/11.2.0/grid -invPtrLoc /u01/app/11.2.0/grid/oraInst.loc
25: /u01/app/11.2.0/grid/bin/emctl start dbconsole
26: /u01/app/11.2.0/grid/rdbms/install/rootadd_rdbms.sh : run as root
27: /u01/app/11.2.0/grid/crs/install/rootcrs.pl -patch : run as root
28: /u01/app/oracle/product/11.2.0/db_1/bin/emctl start dbconsole
29: /u01/app/oracle/product/11.2.0/db_1/bin/emctl start agent
30: /u01/app/oracle/product/11.2.0/db_1/bin/srvctl start home -o /u01/app/oracle/product/11.2.0/db_1 -s /u01/app/oracle/product/11.2.0/db_1/srvm/admin/stophome.txt -n 11rac2
RAC 11.2.0.3滚动升级11.2.0.3.10：等您坐沙发呢！