Oracle 无备份情况恢复ocr和olr

一、先关闭GI，命令不再贴出来了，参见：
http://blog.csdn.net/msdnchina/article/details/49402299

二、模拟OLR损坏

[root@hostb cdata]# ls -lrttotal 2884drwxr-xr-x 2 grid oinstall      4096 Oct 30 13:21 hostbdrwxr-xr-x 2 grid oinstall      4096 Oct 30 19:06  drwxrwxr-x 2 grid oinstall      4096 Dec 22 13:27 hosta-cluster-rw------- 1 root oinstall 272756736 Jan  8 13:38 hostb.olr[root@hostb cdata]# mv hostb.olr  back_hostb.olr ----------->  mv掉OLR[root@hostb cdata]# ls -lrttotal 2884drwxr-xr-x 2 grid oinstall      4096 Oct 30 13:21 hostbdrwxr-xr-x 2 grid oinstall      4096 Oct 30 19:06 localhostdrwxrwxr-x 2 grid oinstall      4096 Dec 22 13:27 hosta-cluster-rw------- 1 root oinstall 272756736 Jan  8 13:38 back_hostb.olr[root@hostb cdata]# /u01/app/11.2.0/grid/bin/crsctl start crs   --------->起不来。CRS-4124:  High Availability Services startup failed.CRS-4000:  Start failed, or completed with errors.[root@hostb cdata]# pwd/u01/app/11.2.0/grid/cdata[root@hostb cdata]#

[oracle@hostb ohasd]$ pwd/u01/app/11.2.0/grid/log/hostb/ohasd[oracle@hostb ohasd]$ ls -lrttotal 95720-rw-r--r-- 1 root root 10540929 Nov  4 03:51 ohasd.l09-rw-r--r-- 1 root root 10556558 Nov  8 16:28 ohasd.l08-rw-r--r-- 1 root root 10558423 Nov 13 12:39 ohasd.l07-rw-r--r-- 1 root root 10558697 Nov 18 05:44 ohasd.l06-rw-r--r-- 1 root root 10557602 Nov 22 18:15 ohasd.l05-rw-r--r-- 1 root root 10532670 Nov 27 08:24 ohasd.l04-rw-r--r-- 1 root root 10556869 Dec  2 01:30 ohasd.l03-rw-r--r-- 1 root root 10557883 Dec  6 23:49 ohasd.l02-rw-r--r-- 1 root root 10542530 Dec 21 11:49 ohasd.l01-rw-r--r-- 1 root root     6700 Jan  8 13:42 ohasdOUT.log-rw-r--r-- 1 root root  2986628 Jan  8 13:42 ohasd.log[oracle@hostb ohasd]$ vi ohasd.log2016-01-08 13:42:59.562: [ default][3149834016] Initializing OLR2016-01-08 13:42:59.564: [  OCROSD][3149834016]utopen:6m': failed in stat OCR file/disk /u01/app/11.2.0/grid/cdata/hostb.olr, errno=2, os err string=No such file or directory2016-01-08 13:42:59.564: [  OCROSD][3149834016]utopen:7: failed to open any OCR file/disk, errno=2, os err string=No such file or directory2016-01-08 13:42:59.564: [  OCRRAW][3149834016]proprinit: Could not open raw device2016-01-08 13:42:59.564: [  OCRAPI][3149834016]a_init:16!: Backend init unsuccessful : [26]2016-01-08 13:42:59.564: [  CRSOCR][3149834016] OCR context init failure.  Error: PROCL-26: Error while accessing the physical storage Operating  error [No such file or directory] [2]2016-01-08 13:42:59.565: [ default][3149834016] Created alert : (:OHAS00106:) :  OLR initialization failed, error: PROCL-26: Error while accessing the physical storage Operating System error [No such file or directory] [2]2016-01-08 13:42:59.565: [ default][3149834016][PANIC] OHASD exiting; Could not init OLR2016-01-08 13:42:59.565: [ default][3149834016] Done.

开始对本地节点（主机名为hostb）进行OLR的recreate,参考文章：
Top 5 Grid Infrastructure Startup Issues (文档 ID 1368382.1) ，搜索关键字为"OLR"

[root@hostb bin]# ./crsctl stop crsCRS-4544: Unable to connect to OHASCRS-4000: Command Stop failed, or completed with errors.

第一步：# /u01/app/11.2.0/grid/crs/install/rootcrs.pl -deconfig -force

[root@hostb install]# /u01/app/11.2.0/grid/crs/install/rootcrs.pl -deconfig -forceUsing configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_paramsPRCR-1119 : Failed to look up CRS resources of ora.cluster_vip_net1.type typePRCR-1068 : Failed to query resourcesCannot communicate with crsdPRCR-1070 : Failed to check if resource ora.gsd is registeredCannot communicate with crsdPRCR-1070 : Failed to check if resource ora.ons is registeredCannot communicate with crsdCRS-4535: Cannot communicate with Cluster Ready ServicesCRS-4000: Command Stop failed, or completed with errors.CRS-4544: Unable to connect to OHASCRS-4000: Command Stop failed, or completed with errors.Successfully deconfigured Oracle clusterware stack on this node[root@hostb install]#

此时的/u01/app/11.2.0/grid/cdata目录情况：

[grid@hostb cdata]$ ls -lrttotal 13124drwxr-xr-x 2 grid oinstall      4096 Oct 30 13:21 hostbdrwxr-xr-x 2 grid oinstall      4096 Oct 30 19:06 localhostdrwxrwxr-x 2 grid oinstall      4096 Dec 22 13:27 hosta-cluster-rw------- 1 root oinstall 272756736 Jan  8 13:38 back_hostb.olr-rw-r--r-- 1 root root      10485760 Jan 12 18:08 hostb.olr[grid@hostb cdata]$

第二步：# /u01/app/11.2.0/grid/root.sh

[root@hostb install]# /u01/app/11.2.0/grid/root.shPerforming root user operation for Oracle 11gThe following environment variables are set as:    ORACLE_OWNER= grid    ORACLE_HOME=  /u01/app/11.2.0/gridEnter the full pathname of the local bin directory: [/usr/local/bin]:The contents of "dbhome" have not changed. No need to overwrite.The contents of "oraenv" have not changed. No need to overwrite.The contents of "coraenv" have not changed. No need to overwrite.Entries will be added to the /etc/oratab file as needed by  Configuration Assistant when a database is createdFinished running generic part of root script.Now product-specific root actions will be performed.Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_paramsUser ignored Prerequisites during installationOLR initialization - successfulAdding Clusterware entries to upstartCRS-2672: Attempting to start 'ora.mdnsd' on 'hostb'CRS-2676: Start of 'ora.mdnsd' on 'hostb' succeededCRS-2672: Attempting to start 'ora.gpnpd' on 'hostb'CRS-2676: Start of 'ora.gpnpd' on 'hostb' succeededCRS-2672: Attempting to start 'ora.cssdmonitor' on 'hostb'CRS-2672: Attempting to start 'ora.gipcd' on 'hostb'CRS-2676: Start of 'ora.cssdmonitor' on 'hostb' succeededCRS-2676: Start of 'ora.gipcd' on 'hostb' succeededCRS-2672: Attempting to start 'ora.cssd' on 'hostb'CRS-2672: Attempting to start 'ora.diskmon' on 'hostb'CRS-2676: Start of 'ora.diskmon' on 'hostb' succeededCRS-2676: Start of 'ora.cssd' on 'hostb' succeededPRKO-2190 : VIP exists for node hostb, VIP name hostb-vipPRCS-1037 : Single Client Access Name VIPs already existPRCS-1028 : Single Client Access Name listeners already existOC4J could not be created as it already existsPRCR-1086 : resource ora.oc4j is already registeredPRCR-1086 : resource ora.cvu is already registeredPRCC-1014 : scan1 was already runningPRCR-1004 : Resource ora.scan1.vip is already runningPRCR-1079 : Failed to start resource ora.scan1.vipCRS-5702: Resource 'ora.scan1.vip' is already running on 'hostb'PRCC-1014 : LISTENER_SCAN1 was already runningPRCR-1004 : Resource ora.LISTENER_SCAN1.lsnr is already runningPRCR-1079 : Failed to start resource ora.LISTENER_SCAN1.lsnrCRS-5702: Resource 'ora.LISTENER_SCAN1.lsnr' is already running on 'hostb'PRCC-1014 : oc4j was already runningPRCR-1004 : Resource ora.oc4j is already runningPRCR-1079 : Failed to start resource ora.oc4jCRS-5702: Resource 'ora.oc4j' is already running on 'hostb'CRS-2672: Attempting to start 'ora.registry.acfs' on 'hostb'CRS-2676: Start of 'ora.registry.acfs' on 'hostb' succeededPRCC-1014 : cvu was already runningPRCR-1004 : Resource ora.cvu is already runningPRCR-1079 : Failed to start resource ora.cvuCRS-5702: Resource 'ora.cvu' is already running on 'hostb'Preparing s for installation...cvuqdisk-1.0.9-1Configure Oracle Grid Infrastructure for a Cluster ... succeeded[root@hostb install]#

完成后的/u01/app/11.2.0/grid/cdata目录情况：

[grid@hostb cdata]$ ls -lrttotal 5612drwxr-xr-x 2 grid oinstall      4096 Oct 30 19:06 localhostdrwxrwxr-x 2 grid oinstall      4096 Dec 22 13:27 hosta-cluster-rw------- 1 root oinstall 272756736 Jan  8 13:38 back_hostb.olrdrwxr-xr-x 2 grid oinstall      4096 Jan 12 18:19 hostb-rw------- 1 root oinstall 272756736 Jan 12 18:19 hostb.olr[grid@hostb cdata]$[grid@hostb cdata]$ crsctl stat res -t--------------------------------------------------------------------------------NAME           TARGET  STATE        SERVER                   STATE_DETAILS      --------------------------------------------------------------------------------Local Resources--------------------------------------------------------------------------------ora.DATA.dg               ONLINE  ONLINE       hostb                                      ora.DATAFILE.dg               ONLINE  ONLINE       hostb                                      ora.FRA.dg               ONLINE  ONLINE       hostb                                      ora.LISTENER.lsnr               ONLINE  ONLINE       hostb                                      ora.asm               ONLINE  ONLINE       hostb                   Started            ora.gsd               OFFLINE OFFLINE      hostb                                      ora.net1. ONLINE  ONLINE       hostb                                      ora.ons               ONLINE  ONLINE       hostb                                      ora.registry.acfs               ONLINE  ONLINE       hostb                                      --------------------------------------------------------------------------------Cluster Resources--------------------------------------------------------------------------------ora.LISTENER_SCAN1.lsnr      1        ONLINE  ONLINE       hostb                                      ora.cvu      1        ONLINE  ONLINE       hostb                                      ora.orcl.db      1        ONLINE  OFFLINE                                                        2        ONLINE  ONLINE       hostb                   Open               ora.orcl.orclxa.svc      1        ONLINE  ONLINE       hostb                                      ora.orcla.vip      1        ONLINE  INTERMEDIATE hostb                   FAILED OVER        ora.hostb.vip      1        ONLINE  ONLINE       hostb                                      ora.oc4j      1        ONLINE  ONLINE       hostb                                      ora.scan1.vip      1        ONLINE  ONLINE       hostb                                      [grid@hostb cdata]$

[grid@hostb cdata]$ lsnrctl statusLSNRCTL for :  11.2.0.3.0 - Production on 12-JAN-2016 18:21:34Copyright (c) 1991, 2011, Oracle.  All rights reserved.Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))STATUS of the LISTENER------------------------Alias                     LISTENERVersion                   TNSLSNR for Linux: Version 11.2.0.3.0 - ProductionStart Date                12-JAN-2016 18:18:21Uptime                    0 days 0 hr. 3 min. 13 secTrace Level               offSecurity                  ON: Local OS AuthenticationSNMP                      OFFListener Parameter File   /u01/app/11.2.0/grid/network/admin/listener.oraListener Log File         /u01/app/grid/diag/tnslsnr/hostb/listener/alert/log.xmlListening Endpoints Summary...  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER)))  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.188.182)(PORT=1528)))  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.188.184)(PORT=1528)))Services Summary...Service "+ASM" has 1 instance(s).  Instance "+ASM2", status READY, has 1 handler(s) for this service...Service "orcl" has 1 instance(s).  Instance "orcl2", status READY, has 1 handler(s) for this service...Service "orclXDB" has 1 instance(s).  Instance "orcl2", status READY, has 1 handler(s) for this service...Service "orclxa" has 1 instance(s).  Instance "orcl2", status READY, has 1 handler(s) for this service...The command completed successfully[grid@hostb cdata]$

[oracle@hostb ~]$ sqlplus / as sysdbaSQL*Plus: Release 11.2.0.3.0 Production on Tue Jan 12 18:29:52 2016Copyright (c) 1982, 2011, Oracle.  All rights reserved.Connected to:Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit ProductionWith the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,Data Mining and Real Application Testing optionsSQL> select open_mode from v$database;OPEN_MODE--------------------READ WRITE