环境介绍：os:rhel 5.8 db:10.2.0.5
数据库以外被停了
Errors in file /u01/app/oracle/admin/asm10g/bdump/asm10g_ckpt_9203.trc: ORA-00206: error in writing (block 3, # blocks 1) of control file ORA-00202: control file: '+DATA/asm10g/controlfile/current.260.826785429' ORA-15078: ASM diskgroup was forcibly dismounted Mon Sep 30 01:45:25 CST 2013 Errors in file /u01/app/oracle/admin/asm10g/bdump/asm10g_ckpt_9203.trc: ORA-00221: error on write to control file ORA-00206: error in writing (block 3, # blocks 1) of control file ORA-00202: control file: '+DATA/asm10g/controlfile/current.260.826785429' ORA-15078: ASM diskgroup was forcibly dismounted Mon Sep 30 01:45:25 CST 2013 CKPT: terminating instance due to error 221 Instance terminated by CKPT, pid = 9203
查看asm实例的实例发现如下的信息
NOTE: assigning ARB0 to group 2/0xb3f85b88 (DATA) Mon Sep 30 01:45:21 CST 2013 WARNING: cache read a corrupted block gn=2 dsk=1 blk=2 from disk 1 NOTE: a corrupted block was dumped to the trace file ERROR: cache failed to read dsk=1 blk=2 from disk(s): 1 ORA-15196: invalid ASM block header [kfc.c:8281] [check_kfbh] [2147483649] [2] [2162363223 != 2162394711] System State dumped to trace file /u01/app/oracle/admin/+ASM/bdump/+asm_arb0_9255.trc NOTE: cache initiating offline of disk 1 group 2 WARNING: process 9255 initiating offline of disk 1.3915950986 (DATA_0001) with mask 0x3 in group 2 WARNING: Disk 1 in group 2 in mode: 0x7,state: 0x2 will be taken offline NOTE: PST update: grp = 2, dsk = 1, mode = 0x6
通过上面的值可以发现在读取disk 1，block 为2时发现check kfbh时发现值不一样。
通过trace出来的文件可以发现下面的信息
/u01/app/oracle/admin/+ASM/bdump/+asm_arb0_9255.trc Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options ORACLE_HOME = /u01/app/oracle/product/10.2.0/db System name: Linux Node name: asm10g Release: 2.6.18-308.el5 Version: #1 SMP Fri Jan 27 17:17:51 EST 2012 Machine: x86_64 Instance name: +ASM Redo thread mounted by this instance: 0 <none> Oracle process number: 13 Unix process pid: 9255, image: oracle@asm10g (ARB0) *** SERVICE NAME:() 2013-09-30 01:45:21.569 *** SESSION ID:(38.5) 2013-09-30 01:45:21.569 ARB0 relocating file +DATA.1.1 (1 entries) *** 2013-09-30 01:45:21.576 WARNING: cache read a corrupted block gn=2 dsk=1 blk=2 from disk 1 OSM metadata block dump: kfbh.endian: 1 ; 0x000: 0x01 kfbh.hard: 130 ; 0x001: 0x82 kfbh.type: 3 ; 0x002: KFBTYP_ALLOCTBL kfbh.datfmt: 1 ; 0x003: 0x01 kfbh.block.blk: 2 ; 0x004: T=0 NUMB=0x2 kfbh.block.obj: 2147483649 ; 0x008: TYPE=0x8 NUMB=0x1 kfbh.check: 2162363223 ; 0x00c: 0x80e30b57 kfbh.fcn.base: 1412 ; 0x010: 0x00000584 kfbh.fcn.wrap: 0 ; 0x014: 0x00000000 kfbh.spare1: 0 ; 0x018: 0x00000000 kfbh.spare2: 0 ; 0x01c: 0x00000000 kfdatb.aunum: 0 ; 0x000: 0x00000000
可以发现当前check不正确，修改check值，这些使用的bbed来修改
BBED> show all
FILE# 0
BLOCK# 17
OFFSET 0
DBA 0x00000000 (0 0,17)
FILENAME /dev/mapper/asm2
BIFILE bifile.bbd
LISTFILE
BLOCKSIZE 512
MODE Edit
EDIT Unrecoverable
IBASE Dec
OBASE Dec
WIDTH 80
COUNT 512
LOGFILE log.bbd
SPOOL No
BBED> map
File: /dev/mapper/asm2 (0)
Block: 17 Dba:0x00000000
------------------------------------------------------------
Undo Segment Header
struct kcbh, 20 bytes @0
struct ktect, 44 bytes @20
struct ktetb[8388610], 67108880 bytes @64
struct ktuxc, 104 bytes @468
struct ktuxe[0], 0 bytes @572
ub4 tailchk @508
BBED> print kcbh
struct kcbh, 20 bytes @0
ub1 type_kcbh @0 0x01
ub1 frmt_kcbh @1 0x82
ub1 spare1_kcbh @2 0x03
ub1 spare2_kcbh @3 0x01
ub4 rdba_kcbh @4 0x00000002
ub4 bas_kcbh @8 0x80000001
ub2 wrp_kcbh @12 0x0b57
ub1 seq_kcbh @14 0xe3
ub1 flg_kcbh @15 0x80 (NONE)
ub2 chkval_kcbh @16 0x0584
ub2 spare3_kcbh @18 0x0000
BBED> dump
File: /dev/mapper/asm2 (0)
Block: 17 Offsets: 0 to 511 Dba:0x00000000
------------------------------------------------------------------------
01820301 02000000 01000080 570be380 84050000 00000000 00000000 00000000
00000000 c0010000 08000800 0000c001 10001000 00000000 18001800 00000000
20002000 00000000 00000000 00008000 00000000 00008000 00000000 02008000
00000000 03008000 02000000 03008000 03000000 03008000 05000000 03008000
06000000 03008000 08000000 03008000 09000000 03008000 0b000000 03008000
0c000000 03008000 0e000000 03008000 0f000000 03008000 11000000 03008000
12000000 03008000 14000000 03008000 15000000 03008000 17000000 03008000
18000000 03008000 1a000000 03008000 1b000000 03008000 1d000000 03008000
1e000000 03008000 20000000 03008000 21000000 03008000 23000000 03008000
24000000 03008000 26000000 03008000 27000000 03008000 29000000 03008000
00000000 04008000 00000000 05008000 00000000 06008000 01000000 01008000
00000000 00018000 02000000 00018000 03000000 00018000 05000000 00018000
06000000 00018000 08000000 00018000 09000000 00018000 0b000000 00018000
0c000000 00018000 0e000000 00018000 0f000000 00018000 11000000 00018000
12000000 00018000 14000000 00018000 15000000 00018000 17000000 00018000
18000000 00018000 1a000000 00018000 1b000000 00018000 1d000000 00018000
80E38657
BBED> modify /x 5786 offset 12
File: /dev/mapper/asm2 (0)
Block: 17 Offsets: 12 to 511 Dba:0x00000000
------------------------------------------------------------------------
5786e380 84050000 00000000 00000000 00000000 00000000 c0010000 08000800
0000c001 10001000 00000000 18001800 00000000 20002000 00000000 00000000
00008000 00000000 00008000 00000000 02008000 00000000 03008000 02000000
03008000 03000000 03008000 05000000 03008000 06000000 03008000 08000000
03008000 09000000 03008000 0b000000 03008000 0c000000 03008000 0e000000
03008000 0f000000 03008000 11000000 03008000 12000000 03008000 14000000
BBED> sum apply
Check value for File 0, Block 17:
current = 0x0584, required = 0x0584
手动mount磁盘组
SQL> alter diskgroup data mount; Diskgroup altered. SUCCESS: diskgroup DATA was mounted Mon Sep 30 02:01:12 CST 2013 NOTE: recovering COD for group 2/0xf8685b9d (DATA) SUCCESS: completed COD recovery for group 2/0xf8685b9d (DATA) Mon Sep 30 02:01:12 CST 2013 NOTE: starting rebalance of group 2/0xf8685b9d (DATA) at power 1 Starting background process ARB0 ARB0 started with pid=12, OS id=9313 Mon Sep 30 02:01:12 CST 2013 NOTE: assigning ARB0 to group 2/0xf8685b9d (DATA) NOTE: stopping process ARB0 Mon Sep 30 02:01:18 CST 2013 SUCCESS: rebalance completed for group 2/0xf8685b9d (DATA) Mon Sep 30 02:01:18 CST 2013 SUCCESS: rebalance completed for group 2/0xf8685b9d (DATA) NOTE: PST update: grp = 2 NOTE: PST update: grp = 2
已经看到pst的信息已经更新完成。
asm实例报ORA-15196: invalid ASM block header [kfc.c:8281] [check_kfbh] [2147483649] [2] [2162363223 != 2162394711]：等您坐沙发呢！