平武建站

KFED修复ASM磁盘头

之前RAC环境出现了故障，节点2操作系统崩溃，重装系统后，CRS添加成功，但是CRS启动有问题，排查发现节点2 ASM的+DATA diskgroup无法mount，报如下错误：
ORA-15063: ASM discovered an insufficient number of disks for diskgroup "DATA"
在ASM实例上检查磁盘组和磁盘的状态，发现+DATA diskgroup的6块盘有3块是MEMBER，有3块是PROVISIONED。

NOMOUNT状态，导致添加操作失败，而尝试在目前正常工作的节点添加磁盘，结果同样报错：

  1. SQL> alter diskgroup DATA add disk'/dev/oracleasm/disks/OADB_DATA_300G_6';
  2. alter diskgroup DATA add disk'/dev/oracleasm/disks/OADB_DATA_300G_6'
  3. *
  4. ERROR at line 1:
  5. ORA-15032: not all alterations performed
  6. ORA-15029: disk'/dev/oracleasm/disks/OADB_DATA_300G_6' is already mounted by this instance
解决办法要么是重建ASM磁盘，要么直接修改ASM磁盘头信息。重建的话耗时太长，因为有本地备库，随时可切换，最终确定通过KFED工具直接修改ASM磁盘头信息。
步骤如下：
1、首先编译kfed
#cd $ORACLE_HOME/rdbms/lib  
#make -f ins_rdbms.mk ikfed 
2、用kfed读取故障ASM磁盘头信息
  1. $kfed read /dev/oracleasm/disks/OADB_DATA_DISK_1
  2. kfbh.endian: 1 ; 0x000: 0x01
  3. kfbh.hard: 130 ; 0x001: 0x82
  4. kfbh.type: 1 ; 0x002: KFBTYP_DISKHEAD
  5. kfbh.datfmt: 1 ; 0x003: 0x01
  6. kfbh.block.blk: 0 ; 0x004: T=0 NUMB=0x0
  7. kfbh.block.obj: 2147483649 ; 0x008: TYPE=0x8 NUMB=0x1
  8. kfbh.check: 1802212223 ; 0x00c: 0x6b6b937f
  9. kfbh.fcn.base: 0 ; 0x010: 0x00000000
  10. kfbh.fcn.wrap: 0 ; 0x014: 0x00000000
  11. kfbh.spare1: 0 ; 0x018: 0x00000000
  12. kfbh.spare2: 0 ; 0x01c: 0x00000000
  13. kfdhdb.driver.provstr: ORCLDISK ; 0x000: length=8
  14. kfdhdb.driver.reserved[0]: 0 ; 0x008: 0x00000000
  15. kfdhdb.driver.reserved[1]: 0 ; 0x00c: 0x00000000
  16. kfdhdb.driver.reserved[2]: 0 ; 0x010: 0x00000000
  17. kfdhdb.driver.reserved[3]: 0 ; 0x014: 0x00000000
  18. kfdhdb.driver.reserved[4]: 0 ; 0x018: 0x00000000
  19. kfdhdb.driver.reserved[5]: 0 ; 0x01c: 0x00000000
  20. kfdhdb.compat: 168820736 ; 0x020: 0x0a100000
  21. kfdhdb.dsknum: 1 ; 0x024: 0x0001
  22. kfdhdb.grptyp: 1 ; 0x026: KFDGTP_EXTERNAL
  23. kfdhdb.hdrsts: 3 ; 0x027: KFDHDR_MEMBER
  24. kfdhdb.dskname: DATAG1_0001 ; 0x028: length=11
  25. kfdhdb.grpname: DATAG1 ; 0x048: length=6
  26. kfdhdb.fgname: DATAG1_0001 ; 0x068: length=11
  27. kfdhdb.capname: ; 0x088: length=0
  28. kfdhdb.crestmp.hi: 32928501 ; 0x0a8: HOUR=0x15 DAYS=0x17 MNTH=0xc YEAR=0x7d9
  29. kfdhdb.crestmp.lo: 2195144704 ; 0x0ac: USEC=0x0 MSEC=0x1d0 SECS=0x2d MINS=0x20
  30. kfdhdb.mntstmp.hi: 32940275 ; 0x0b0: HOUR=0x13 DAYS=0x7 MNTH=0x8 YEAR=0x7da
  31. kfdhdb.mntstmp.lo: 3201116160 ; 0x0b4: USEC=0x0 MSEC=0x34a SECS=0x2c MINS=0x2f
  32. kfdhdb.secsize: 512 ; 0x0b8: 0x0200
  33. kfdhdb.blksize: 4096 ; 0x0ba: 0x1000
  34. kfdhdb.ausize: 1048576 ; 0x0bc: 0x00100000
  35. kfdhdb.mfact: 113792 ; 0x0c0: 0x0001bc80
  36. kfdhdb.dsksize: 512000 ; 0x0c4: 0x0007d000
  37. kfdhdb.pmcnt: 6 ; 0x0c8: 0x00000006
  38. kfdhdb.fstlocn: 1 ; 0x0cc: 0x00000001
  39. kfdhdb.altlocn: 2 ; 0x0d0: 0x00000002
  40. kfdhdb.f1b1locn: 0 ; 0x0d4: 0x00000000
  41. kfdhdb.redomirrors[0]: 0 ; 0x0d8: 0x0000
  42. kfdhdb.redomirrors[1]: 0 ; 0x0da: 0x0000
  43. kfdhdb.redomirrors[2]: 0 ; 0x0dc: 0x0000
  44. kfdhdb.redomirrors[3]: 0 ; 0x0de: 0x0000
  45. kfdhdb.dbcompat: 168820736 ; 0x0e0: 0x0a100000
  46. kfdhdb.grpstmp.hi: 32928501 ; 0x0e4: HOUR=0x15 DAYS=0x17 MNTH=0xc YEAR=0x7d9
  47. kfdhdb.grpstmp.lo: 2195053568 ; 0x0e8: USEC=0x0 MSEC=0x177 SECS=0x2d MINS=0x20
  48. kfdhdb.ub4spare[0]: 0 ; 0x0ec: 0x00000000
  49. kfdhdb.ub4spare[1]: 0 ; 0x0f0: 0x00000000
  50. kfdhdb.ub4spare[2]: 0 ; 0x0f4: 0x00000000
  51. kfdhdb.ub4spare[3]: 0 ; 0x0f8: 0x00000000
  52. kfdhdb.ub4spare[4]: 0 ; 0x0fc: 0x00000000
  53. kfdhdb.ub4spare[5]: 0 ; 0x100: 0x00000000
  54. kfdhdb.ub4spare[6]: 0 ; 0x104: 0x00000000
  55. kfdhdb.ub4spare[7]: 0 ; 0x108: 0x00000000
  56. kfdhdb.ub4spare[8]: 0 ; 0x10c: 0x00000000
  57. kfdhdb.ub4spare[9]: 0 ; 0x110: 0x00000000
  58. kfdhdb.ub4spare[10]: 0 ; 0x114: 0x00000000
  59. kfdhdb.ub4spare[11]: 0 ; 0x118: 0x00000000
  60. kfdhdb.ub4spare[12]: 0 ; 0x11c: 0x00000000
  61. kfdhdb.ub4spare[13]: 0 ; 0x120: 0x00000000
  62. kfdhdb.ub4spare[14]: 0 ; 0x124: 0x00000000
  63. kfdhdb.ub4spare[15]: 0 ; 0x128: 0x00000000
  64. kfdhdb.ub4spare[16]: 0 ; 0x12c: 0x00000000
  65. kfdhdb.ub4spare[17]: 0 ; 0x130: 0x00000000
  66. kfdhdb.ub4spare[18]: 0 ; 0x134: 0x00000000
  67. kfdhdb.ub4spare[19]: 0 ; 0x138: 0x00000000
  68. kfdhdb.ub4spare[20]: 0 ; 0x13c: 0x00000000
  69. kfdhdb.ub4spare[21]: 0 ; 0x140: 0x00000000
  70. kfdhdb.ub4spare[22]: 0 ; 0x144: 0x00000000
  71. kfdhdb.ub4spare[23]: 0 ; 0x148: 0x00000000
  72. kfdhdb.ub4spare[24]: 0 ; 0x14c: 0x00000000
  73. kfdhdb.ub4spare[25]: 0 ; 0x150: 0x00000000
  74. kfdhdb.ub4spare[26]: 0 ; 0x154: 0x00000000
  75. kfdhdb.ub4spare[27]: 0 ; 0x158: 0x00000000
  76. kfdhdb.ub4spare[28]: 0 ; 0x15c: 0x00000000
  77. kfdhdb.ub4spare[29]: 0 ; 0x160: 0x00000000
  78. kfdhdb.ub4spare[30]: 0 ; 0x164: 0x00000000
  79. kfdhdb.ub4spare[31]: 0 ; 0x168: 0x00000000
  80. kfdhdb.ub4spare[32]: 0 ; 0x16c: 0x00000000
  81. kfdhdb.ub4spare[33]: 0 ; 0x170: 0x00000000
  82. kfdhdb.ub4spare[34]: 0 ; 0x174: 0x00000000
  83. kfdhdb.ub4spare[35]: 0 ; 0x178: 0x00000000
  84. kfdhdb.ub4spare[36]: 0 ; 0x17c: 0x00000000
  85. kfdhdb.ub4spare[37]: 0 ; 0x180: 0x00000000
  86. kfdhdb.ub4spare[38]: 0 ; 0x184: 0x00000000
  87. kfdhdb.ub4spare[39]: 104436 ; 0x198:0x000197f4
  88. kfdhdb.ub4spare[40]: 0 ; 0x18c: 0x00000000
  89. kfdhdb.ub4spare[41]: 0 ; 0x190: 0x00000000
  90. kfdhdb.ub4spare[42]: 0 ; 0x194: 0x00000000
  91. kfdhdb.ub4spare[43]: 0 ; 0x198: 0x00000000
  92. kfdhdb.ub4spare[44]: 0 ; 0x19c: 0x00000000
  93. kfdhdb.ub4spare[45]: 0 ; 0x1a0: 0x00000000
  94. kfdhdb.ub4spare[46]: 0 ; 0x1a4: 0x00000000
  95. kfdhdb.ub4spare[47]: 0 ; 0x1a8: 0x00000000
  96. kfdhdb.ub4spare[48]: 0 ; 0x1ac: 0x00000000
  97. kfdhdb.ub4spare[49]: 0 ; 0x1b0: 0x00000000
  98. kfdhdb.ub4spare[50]: 0 ; 0x1b4: 0x00000000
  99. kfdhdb.ub4spare[51]: 0 ; 0x1b8: 0x00000000
  100. kfdhdb.ub4spare[52]: 0 ; 0x1bc: 0x00000000
  101. kfdhdb.ub4spare[53]: 0 ; 0x1c0: 0x00000000
  102. kfdhdb.ub4spare[54]: 0 ; 0x1c4: 0x00000000
  103. kfdhdb.ub4spare[55]: 0 ; 0x1c8: 0x00000000
  104. kfdhdb.ub4spare[56]: 0 ; 0x1cc: 0x00000000
  105. kfdhdb.ub4spare[57]: 0 ; 0x1d0: 0x00000000
  106. kfdhdb.acdb.aba.seq: 0 ; 0x1d4: 0x00000000
  107. kfdhdb.acdb.aba.blk: 0 ; 0x1d8: 0x00000000
  108. kfdhdb.acdb.ents: 0 ; 0x1dc: 0x0000
  109. kfdhdb.acdb.ub2spare: 43605 ; 0x1de:0xaa55
通过kfed分别读取故障ASM磁盘头信息和正常ASM磁盘头信息，发现故障磁盘头标红地方与正常ASM磁盘头不一致，故障ASM磁盘信息如下：
kfdhdb.ub4spare[39]: 104436 ; 0x198: 0x000197f4
kfdhdb.acdb.ub2spare: 43605 ; 0x1de: 0xaa55
正常ASM磁盘头信息该项为0
3、修复步骤：
a、通过kfed read /dev/oracleasm/disks/OADB_DATA_DISK_1 > /tmp/disk1.txt 导出磁盘头信息
b、修改kfdhdb.ub4spare[39]: 104436 ; 0x198: 0x000197f4 为  kfdhdb.ub4spare[39]: 0 ; 0x198: 0x00000000  ； 修改kfdhdb.acdb.ub2spare: 43605 ; 0x1de: 0xaa55为kfdhdb.acdb.ub2spare: 0 ; 0x1de: 0x0000
c、通过kfed merge  /dev/oracleasm/disks/OADB_DATA_DISK_1 text=/tmp/disk1.txt
d、另外两块磁盘也执行如上操作
e、sqlplus / as sysasm ——select group_number, disk_number, mount_status, header_status, name, path from v$asm_disk;  发现磁盘头信息已显示为MEMBER，+DATA也成功mount
网页题目：KFED修复ASM磁盘头
文章网址：http://pwwzsj.com/article/gccsej.html

其他资讯