Exadata Cell Server Lab: Grid Disk Failure & Recovery

A hands-on lab covering the CellDisk → GridDisk → ASM storage stack, Grid Disk failure simulation, Cell Disk recreation, and recovery validation.

1. Introduction

One of the most important concepts for an Exadata DBA is understanding what happens between the physical storage and ASM.

The storage hierarchy can be simplified as:

Physical Disk / LUN

        ↓

    Cell Disk

        ↓

    Grid Disk

        ↓

     ASM Disk

        ↓

   ASM Disk Group

In a simulator environment, the physical disks may be represented by files such as:

/opt/oracle/.../disks/raw/DISK06

This makes it possible to practice CellCLI storage administration without modifying physical Exadata hardware.



2. Understanding CellDisk and GridDisk

A Cell Disk represents the usable storage carved from a physical disk/LUN.

A Grid Disk is a logical slice of that Cell Disk that is presented to ASM or other consumers.

For example, a 5 GB Cell Disk can be divided into multiple Grid Disks:

CD_DISK06 — 5 GB

├── DATA

│   Offset: 48M

│   Size:   3G

├── RECO

│   Offset: 3.046875G

│   Size:   1.5G

└── DBFS

    Offset: 4.546875G

    Size:   464M

The important point is that offset is the starting position of a Grid Disk inside its Cell Disk.

For example:

DATA:

offset = 48M

size   = 3G

means the DATA Grid Disk begins at approximately 48 MB into the Cell Disk.



3. Check the Physical Disk

On the Cell Server:

ls -lh /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/

You may see:

DISK01

DISK02

DISK03

DISK04

DISK05

DISK06

DISK07

FLASH01

FLASH02

FLASH03

FLASH04

Enter CellCLI:

cellcli

Check physical disks:

LIST PHYSICALDISK;

For a specific disk:

LIST PHYSICALDISK '/opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK06' DETAIL;

Important attributes include:

name

diskType

physicalSize

status

errorCount

physicalSerial

physicalInsertTime



4. Check the Cell Disk

LIST CELLDISK;

For a specific Cell Disk:

LIST CELLDISK 'CD_DISK06_ex01celadm03' DETAIL;

Useful attributes:

LIST CELLDISK ATTRIBUTES name,size,freeSpace,status;

This allows you to determine:

  • Cell Disk size
  • Available space
  • Parent physical disk
  • Current status



5. Check the Grid Disks

List all Grid Disks:

LIST GRIDDISK;

For DISK06:

LIST GRIDDISK WHERE cellDisk='CD_DISK06_ex01celadm03'

ATTRIBUTES name,cellDisk,size,offset,status;

For detailed information:

LIST GRIDDISK 'data_CD_DISK06_ex01celadm03' DETAIL;

The important attributes are:

name

cellDisk

size

offset

status

asmDiskName

asmDiskgroupName

asmFailGroupName



6. Why Offset Matters

Suppose the Cell Disk is:

5 GB

and contains:

DATA → 3 GB

RECO → 1.5 GB

DBFS → 464 MB

The approximate layout is:

0

├── Reserved/metadata area

├── 48 MB

├──────── DATA 3 GB ────────┤

│                           │

├──────── RECO 1.5 GB ──────┤

│                           │

├──────── DBFS 464 MB ──────┤

│                           │

└───────────────────────────┘

                         5 GB

The offsets tell Cell Services where each Grid Disk begins.

Therefore, if you’re recreating Grid Disks over existing underlying data, preserving the original:

Grid Disk name

Offset

Size

is critical.



7. Simulating Grid Disk Failure

Before making changes, record the existing configuration.

For example:

LIST GRIDDISK ATTRIBUTES name,cellDisk,size,offset,status;

Save the output.

Also record:

LIST CELLDISK ATTRIBUTES name,size,freeSpace,status;

And:

LIST PHYSICALDISK ATTRIBUTES name,physicalSize,status;

This gives you a recovery reference.



8. Make Grid Disks Inactive

In a controlled training environment, you can practice changing Grid Disk state.

Do not perform destructive operations on a production Exadata system just to reproduce this lab.

The purpose of this exercise is to understand the lifecycle:

Active

  ↓

Inactive

  ↓

Dropped

  ↓

Cell Disk recreated

  ↓

Grid Disks recreated

  ↓

ASM validation

The exact CellCLI syntax should always be checked against the CellCLI version in use.



9. Drop the Cell Disk

The lab scenario can then demonstrate removing the Cell Disk object from Cell Services while leaving the underlying simulator disk file untouched.

This distinction is extremely important.

You can have:

CellCLI metadata

        ↓

Cell Disk

        ↓

Grid Disks

while separately having the underlying simulator file:

/opt/oracle/.../disks/raw/DISK06

The Cell Disk object and the underlying file are not the same object.

Therefore, removing a Cell Disk object in the simulator does not automatically mean that the physical simulator file has been deleted.



10. Verify the Physical Disk File

After the Cell Disk is removed, verify at the Linux level:

ls -lh /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK06

You can also check:

stat /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK06

And:

du -h /opt/oracle/cell11.2.3.2.1_LINUX.X64_130109/disks/raw/DISK06

The purpose is to demonstrate:

CellCLI object removed

        ≠

Underlying simulator disk file deleted



11. Recreate the Cell Disk

In the simulator, the Cell Disk can be recreated using the underlying LUN/file reference.

The exact syntax depends on the CellCLI/simulator implementation.

The important concept is:

Existing physical/LUN reference

          ↓

Recreate Cell Disk

          ↓

Same underlying storage

Do not blindly copy a simulator command into a real Exadata environment. The storage architecture and supported commands differ.



12. Recreate the Grid Disks

Once the Cell Disk has been recreated, the next task is to reproduce the original Grid Disk layout.

For example:

CD_DISK06

├── DATA

│   Offset = 48M

│   Size   = 3G

├── RECO

│   Offset = 3.046875G

│   Size   = 1.5G

└── DBFS

    Offset = 4.546875G

    Size   = 464M

The objective is to reproduce the original logical layout rather than simply creating three arbitrary Grid Disks.



13. Verify CellCLI Recovery

After recreation:

LIST PHYSICALDISK;

Then:

LIST CELLDISK;

Then:

LIST GRIDDISK;

And finally:

LIST GRIDDISK ATTRIBUTES

name,cellDisk,size,offset,status;

Compare the result with the configuration you recorded before the exercise.



14. Verify ASM

Move to the ASM/database host:

sqlplus / as sysasm

Check ASM diskgroups:

SELECT

    name,

    state,

    type,

    total_mb,

    free_mb

FROM v$asm_diskgroup;

Check ASM disks:

SELECT

    name,

    path,

    header_status,

    mode_status,

    state,

    total_mb,

    free_mb

FROM v$asm_disk

ORDER BY path;

If the environment uses the appropriate ASM discovery configuration, verify that the recreated Grid Disks are visible again.



15. Final Recovery Validation

The recovery is not complete simply because CellCLI shows:

status = active

You should validate every layer.

Physical layer

ls -lh .../DISK06

Cell Disk layer

LIST CELLDISK 'CD_DISK06_ex01celadm03' DETAIL;

Grid Disk layer

LIST GRIDDISK WHERE cellDisk='CD_DISK06_ex01celadm03'

ATTRIBUTES name,size,offset,status;

ASM layer

SELECT name,state,type,total_mb,free_mb

FROM v$asm_diskgroup;

and:

SELECT name,path,header_status,mode_status,state

FROM v$asm_disk;

Database layer

Finally, validate that the database can access the affected data normally.



16. What I Learned From This Lab

The biggest lesson isn’t the CellCLI syntax.

It’s understanding which layer you’re working on.

                    Exadata Storage Stack


              Physical Disk / LUN

                       │

                       ▼

                  Cell Disk

                       │

             ┌─────────┴─────────┐

             ▼         ▼         ▼

           DATA      RECO       DBFS

        Grid Disk  Grid Disk  Grid Disk

             │         │         │

             └─────────┴─────────┘

                       │

                       ▼

                    ASM Disk

                       │

                       ▼

                 ASM Disk Group

                       │

                       ▼

                   Database

A Cell Disk problem is not automatically an ASM problem.

A Grid Disk problem is not automatically a physical disk failure.

And deleting a CellCLI storage object in a simulator is not necessarily equivalent to deleting the underlying storage.

That separation of layers is one of the most important concepts to understand when learning Exadata administration.



⚠️ Lab Safety

This procedure is appropriate as a controlled training/simulator exercise. Do not reproduce the drop/recreate steps against a production Exadata system without an approved recovery procedure, backups, and Oracle-supported operational guidance


Comments