By Nabil Nawaz, Senior Principal Consultant, Viscosity NA

Sys Admins and Exadata DBA’s probably have run the SunDiag scripts from Oracle Support to gather diagnostic information for diagnosing physical and flash disk failures/issues. One thing to remember is that the version of the SunDiag script that is on the system may not be the latest so please download the latest one from the following Support note: Oracle Exadata Diagnostic Information required for Disk Failures [ID 761868.1] 

The output file is compressed in bz2 format and furthermore it is not easy to parse and read the all of the  files. There is however a better script that can be run to do similar diagnostic against the Flash Cards/Disks on the Storage Cells and the output is all captured in a single text logfile that is easy to read. The script is called and was written by Bob Clare, Oracle Exadata Support.  The script can be run from any node (DB or Storage) individually, but must be run as ‘root’, and can also be run in parallel on multiple nodes via dcli (much as with sundiag, as shown in Note 761868.1). Here’s how I typically run it: 

$ dcli -g ~/all_group -l root -t -x | tee /tmp/CheckDBM-All.log

where ~ corresponds to /root (assuming dcli group files have been copied to same, else use full /opt/… patch to group files as in sundiag note), and of course other group files could be substituted, e.g. cell_group, w/pipe to tee gathering all results in a single local log (so you don’t have to go gather them individually from each node).


Sample output for one of the Exadata Storage Cells below.

Target cells: [‘exa01-priv’, ‘exa02-priv’, ‘exa03-priv’, ‘exa04-priv’, ‘exacell01-priv’, ‘exacell02-priv’, ‘exacell03-priv’, ‘exacell04-priv’, ‘exacell05-priv’, ‘exacell06-priv’, ‘exacell07-priv’]


exa01-priv: (Version 2.5) started on: Tuesday, 05/07/13 at: 23:18:17 CDT …

exa01-priv: ================================ System ================================

exa01-priv: Node ‘exa01’ local time: 23:18:17 up 17 days

exa01-priv: MACHINETYPE: 1 X2-2 Half rack

exa01-priv: ——————————– OS ——————————–

exa01-priv: Linux 2.6.18- x86_64

exa01-priv: ==> /etc/enterprise-release <==

exa01-priv: Enterprise Linux Enterprise Linux Server release 5.5 (Carthage)


exa01-priv: ==> /etc/redhat-release <==

exa01-priv: Red Hat Enterprise Linux Server release 5.5 (Tikanga)

exa01-priv: ——————————– DMI ——————————–

exa01-priv: BIOS Information

exa01-priv: Vendor: American Megatrends Inc.

exa01-priv: Version: 08060108

exa01-priv: Release Date: 12/27/2010

exa01-priv: System Information

exa01-priv: Manufacturer: SUN MICROSYSTEMS

exa01-priv: Product Name: SUN FIRE X4170 M2 SERVER

exa01-priv: Serial Number: 1111ABC123

exa01-priv: ——————————– IPMI/ILOM ——————————–

exa01-priv: ipmitool version

exa01-priv: system_description = SUN FIRE X4170 M2 SERVER, ILOM v3.0.14.11.b, r62978

exa01-priv: system_identifier = Exadata Database Machine X2-2 AK00014603

exa01-priv: product_serial_number = 111ABC123EFG

exa01-priv: ——————————– Exadata Image History ——————————–

exa01-priv: Version                              :

exa01-priv: Image activation date                : 2011-08-30 06:18:15 -0500

exa01-priv: Imaging mode                         : fresh

exa01-priv: Imaging status                       : success


exa01-priv: ================================ MultiDisk (SW RAID) Info ================================

exa01-priv: ================================ MegaRAID ================================

exa01-priv: MegaCLI SAS RAID Management Tool  Ver 8.00.23 May 17, 2010

exa01-priv: ——————————– BBU Status ——————————–

exa01-priv: Absolute state of charge: 0 %

exa01-priv: Absolute State of charge: 83 %

exa01-priv: Average Time to full: Battery is not being charged

exa01-priv: Battery FRU: N/A

exa01-priv: Battery Pack Missing         : No

exa01-priv: Battery Replacement required : No

exa01-priv: Battery state:

exa01-priv: BatteryType: iBBU08

exa01-priv: Charging current: 0 mA

exa01-priv: Charging Status              : None

exa01-priv: Charging Terminated     : No

exa01-priv: Cycle Count: 0

exa01-priv: Date of Manufacture: 02/23, 2013

exa01-priv: Design Capacity: 1500 mAh

exa01-priv: Design Voltage: 4100 mV

exa01-priv: Discharging             : No

exa01-priv: Full Charge Capacity: 1317 mAh

exa01-priv: Fully Charged           : No

exa01-priv: Fully Discharged        : No

exa01-priv: Learn Cycle Active           : No

exa01-priv: Learn Cycle Requested        : No

exa01-priv: Learn Cycle Status           : OK

exa01-priv: Learn Cycle Timeout          : No

exa01-priv: Max Error: 0 %

exa01-priv: Over Charged            : No

exa01-priv: Over Temperature        : No

exa01-priv: Relative State of Charge: 95 %

exa01-priv: Remaining Capacity: 1248 mAh

exa01-priv: Remaining Capacity Alarm: 0 mAh

exa01-priv: Remaining Capacity Alarm: No

exa01-priv: Remaining Capacity Low       : No

exa01-priv: Run time to empty: Battery is not being discharged

exa01-priv: Temperature: 44 C

exa01-priv: Temperature                  : OK

exa01-priv: Voltage: 4040 mV

exa01-priv: Voltage                      : OK

exa01-priv: ——————————– Disk Status ——————————–

exa01-priv: Adapter: 0

exa01-priv: Product Name: LSI MegaRAID SAS 9261-8i

exa01-priv: BBU: Present

exa01-priv: Number of DISK GROUPS: 1


exa01-priv: DISK GROUP: 0

exa01-priv: Virtual Drive: 0 (Target Id: 0)

exa01-priv: RAID Level          : Primary-5, Secondary-0

exa01-priv: Size                : 557.75 GB

exa01-priv: State               : Optimal

exa01-priv: Number Of Drives    : 3

exa01-priv: Default Cache Policy: WriteBack, ReadAheadNone

exa01-priv: Current Cache Policy: WriteBack, ReadAheadNone

exa01-priv: Disk Cache Policy   : Disabled

exa01-priv: Slot Number: 0

exa01-priv: Media Error Count: 0

exa01-priv: Other Error Count: 0

exa01-priv: Predictive Failure Count: 0

exa01-priv: Firmware state: Online, Spun Up

exa01-priv: Inquiry Data: SEAGATE ST930003SSUN300G0B701109735H0L

exa01-priv: Foreign State: None


exa01-priv: Slot Number: 1

exa01-priv: Media Error Count: 0

exa01-priv: Other Error Count: 0

exa01-priv: Predictive Failure Count: 0

exa01-priv: Firmware state: Online, Spun Up

exa01-priv: Inquiry Data: SEAGATE ST930003SSUN300G0B701110738G7Q

exa01-priv: Foreign State: None


exa01-priv: Slot Number: 2

exa01-priv: Media Error Count: 0

exa01-priv: Other Error Count: 0

exa01-priv: Predictive Failure Count: 0

exa01-priv: Firmware state: Online, Spun Up

exa01-priv: Inquiry Data: SEAGATE ST930003SSUN300G0B7011097348K5

exa01-priv: Foreign State: None


exa01-priv: ================================ Flash Disks ================================
exa01-priv: ——————————– Physical ——————————–
exa01-priv: Name           FW Ver       LUN   Status       Serial#
exa01-priv: FLASH_1_0      D20Y  1_0   normal       111ABC123E
exa01-priv: FLASH_1_1      D20Y  1_1   normal       111ABC123E
exa01-priv: FLASH_1_2      D20Y  1_2   normal       111ABC123E
exa01-priv: FLASH_1_3      D20Y  1_3   normal       111ABC123E
exa01-priv: FLASH_2_0      D20Y  2_0   normal       111ABC123E
exa01-priv: FLASH_2_1      D20Y  2_1   normal       111ABC123E
exa01-priv: FLASH_2_2      D20Y  2_2   normal       111ABC123E
exa01-priv: FLASH_2_3      D20Y  2_3   normal       111ABC123E
exa01-priv: FLASH_4_0      D20Y  4_0   normal       111ABC123E
exa01-priv: FLASH_4_1      D20Y  4_1   normal       111ABC123E
exa01-priv: FLASH_4_2      D20Y  4_2   normal       111ABC123E
exa01-priv: FLASH_4_3      D20Y  4_3   normal       111ABC123E
exa01-priv: FLASH_5_0      D20Y  5_0   normal       111ABC123E
exa01-priv: FLASH_5_1      D20Y  5_1   normal       111ABC123E
exa01-priv: FLASH_5_2      D20Y  5_2   normal       111ABC123E
exa01-priv: FLASH_5_3      D20Y  5_3   normal       111ABC123E
exa01-priv: ——————————– LUNs ——————————–
exa01-priv: Name     Cell Disk           Status       Phys Disk
exa01-priv: 1_0      FD_00_exa01-priv    normal       FLASH_1_0
exa01-priv: 1_1      FD_01_exa01-priv    normal       FLASH_1_1
exa01-priv: 1_2      FD_02_exa01-priv    normal       FLASH_1_2
exa01-priv: 1_3      FD_03_exa01-priv    normal       FLASH_1_3
exa01-priv: 2_0      FD_04_exa01-priv    normal       FLASH_2_0
exa01-priv: 2_1      FD_05_exa01-priv    normal       FLASH_2_1
exa01-priv: 2_2      FD_06_exa01-priv    normal       FLASH_2_2
exa01-priv: 2_3      FD_07_exa01-priv    normal       FLASH_2_3
exa01-priv: 4_0      FD_08_exa01-priv    normal       FLASH_4_0
exa01-priv: 4_1      FD_09_exa01-priv    normal       FLASH_4_1
exa01-priv: 4_2      FD_10_exa01-priv    normal       FLASH_4_2
exa01-priv: 4_3      FD_11_exa01-priv    normal       FLASH_4_3
exa01-priv: 5_0      FD_12_exa01-priv    normal       FLASH_5_0
exa01-priv: 5_1      FD_13_exa01-priv    normal       FLASH_5_1
exa01-priv: 5_2      FD_14_exa01-priv    normal       FLASH_5_2
exa01-priv: 5_3      FD_15_exa01-priv    normal       FLASH_5_3
exa01-priv: ——————————– Cell ——————————–
exa01-priv: Name                  LUN   Status       Size
exa01-priv: FD_00_exa01-priv      1_0   normal       22.875G
exa01-priv: FD_01_exa01-priv      1_1   normal       22.875G
exa01-priv: FD_02_exa01-priv      1_2   normal       22.875G
exa01-priv: FD_03_exa01-priv      1_3   normal       22.875G
exa01-priv: FD_04_exa01-priv      2_0   normal       22.875G
exa01-priv: FD_05_exa01-priv      2_1   normal       22.875G
exa01-priv: FD_06_exa01-priv      2_2   normal       22.875G
exa01-priv: FD_07_exa01-priv      2_3   normal       22.875G
exa01-priv: FD_08_exa01-priv      4_0   normal       22.875G
exa01-priv: FD_09_exa01-priv      4_1   normal       22.875G
exa01-priv: FD_10_exa01-priv      4_2   normal       22.875G
exa01-priv: FD_11_exa01-priv      4_3   normal       22.875G
exa01-priv: FD_12_exa01-priv      5_0   normal       22.875G
exa01-priv: FD_13_exa01-priv      5_1   normal       22.875G
exa01-priv: FD_14_exa01-priv      5_2   normal       22.875G
exa01-priv: FD_15_exa01-priv      5_3   normal       22.875G
exa01-priv: ——————————– Grid ——————————–
exa01-priv: Name                         Status       Size  ASM Mode, DeActOut
exa01-priv: ================================ Flash Cache ================================
exa01-priv: Name                  Status       Size         Effective
exa01-priv: exa01-priv_FLASHCACHE normal       365.25G      365.25G


exa01-priv: ================================ End Of Report ================================



Comments are closed