I have an HPE ProLiant DL360p Gen8 with the Smart Array P420i (Firmware 8.32, chip PMC-Sierra).
OS: Ubuntu Server 24.04 LTS (Kernel 6.8.0-134-generic, driver hpsa)
Target Drive: 1.2TB 10.5K SAS HDD (Model: HP EG001200JWJNQ, Firmware: HPD1)
The drive disconnected in the system, either due to temperature or something else.
It is in write protected mode and shows Failed.
I have 4 drives in the system, two mirrors for the OS 500 GB.
In the system log of the Linux kernel dmesg, the hpsa driver has errors:
hpsa 0000:02:00.0: hpsa_update_device_info: LV failed, device will be skipped.
Due to the fact that the Linux kernel executes the skipped command, the operating system does not create SCSI device files at all (there is no letter like /dev/sdd, nor low-level generic devices /dev/sg* in the /sys/class/scsi_generic/ folder).
Testing with LiveCD gives exactly the same picture — sg_map -i sees only healthy volumes, and the failure slot is completely hidden. The drive can be communicated with exclusively through the ssacli utility.
If I do delete, the drive disappears.
I cannot clean the drive via erase because it is configured (occupied by the array).
I cannot enable the array using reenable because Array status not ok (the hardware Write protect triggered).
It is impossible to use Linux native utilities (dd, sg_format), because the controller has completely isolated the SCSI-generic device of the drive.
In ORCA mode, it does not allow deleting it.
Here is the ssacli info:
Smart Array P420i in Slot 0 (Embedded) (sn: 00143803116CD50)
Internal Drive Cage at Port 1I, Box 1 (Index 0), OK
Internal Drive Cage at Port 2I, Box 0 (Index 1), OK
Port Name: 1I
Port Name: 2I
Array A (Solid State SATA, Unused Space: 0 MB)
logicaldrive 1 (465.73 GB, RAID 1, OK)
physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SATA SSD, 500 GB, OK)
physicaldrive 1I:1:2 (port 1I:box 1:bay 2, SATA SSD, 500 GB, OK)
Array B (SAS,
logicaldrive 2 (1.09 TB, RAID 0, Failed)
physicaldrive 1I:1:4 (port 1I:box 1:bay 4, SAS HDD, 1.2 TB, Failed)
Array C (SAS, Unused Space: 0 MB)
logicaldrive 3 (1.09 TB, RAID 0, OK)
physicaldrive 1I:1:3 (port 1I:box 1:bay 3, SAS HDD, 1.2 TB, OK)
SEP (Vendor ID PMCSIERA, Model SRCv8x6G) 380 (WWID: 500143803116CD5F)
Reenable or scan do not help - Array status not ok.
Tried modify erase - Not available because the drive is currently configured.
Tried sanitize, Syntax error at "sanitize".
Tried pulling the drive out for 10 seconds and inserting it back, doesn't help.
Tried turning off the server and disconnecting the power cables for 20 sec doesn't help.
When moving it to the 5th slot (from the 4th) it does not write failed but Predictive Failure.
during livecd boot - sg_map -i does not find the drive that failed.
Drives are working, OS and the third 1.2 TB, but I want to restore the fourth one for a mirror.
When booting into ORCA or looking through the controller settings during startup, there are no options as rewrite or re-enable.
The target drive status is Failed, I wanted to format it but I can't do that either.
All my critical data is located on the other working drives, and it is critically important not to damage or break them under any circumstances.
Can't fix it for a week now, please help, if you need extra info let me know, I will add it.
Smart Array P420i in Slot 0 (Embedded)
Bus Interface: PCI
Slot: 0
Serial Number: 00143803116CD50
Cache Serial Number: PBKUC0BRH5P0Q2
RAID 6 Status: Enabled
Controller Status: OK
Hardware Revision: B
Firmware Version: 8.32
Firmware Supports Online Firmware Activation: False
Rebuild Priority: Low
Expand Priority: Medium
Surface Scan Delay: 15 secs
Surface Scan Mode: Idle
Parallel Surface Scan Supported: No
Queue Depth: Automatic
Monitor and Performance Delay: 60 min
Elevator Sort: Enabled
Degraded Performance Optimization: Disabled
Inconsistency Repair Policy: Disabled
Wait for Cache Room: Disabled
Surface Analysis Inconsistency Notification: Disabled
Post Prompt Timeout: 0 secs
Cache Board Present: True
Cache Status: Permanently Disabled
Cache Status Details: Cache disabled; backup to flash memory failed
Cache Disable Reason: Permanent disable condition. The posted write cache has been disabled because the backup operation failed in a flash-backed write cache module.
Drive Write Cache: Disabled
Total Cache Size: 1.0
Total Cache Memory Available: 0.8
Battery Backed Cache Size: 0.8
No-Battery Write Cache: Disabled
SSD Caching RAID5 WriteBack Enabled: False
SSD Caching Version: 1
Cache Backup Power Source: Capacitors
Battery/Capacitor Count: 1
Battery/Capacitor Status: OK
SATA NCQ Supported: True
Spare Activation Mode: Activate on physical drive failure (default)
Controller Temperature (C): 81
Cache Module Temperature (C): 49
Capacitor Temperature (C): 29
Number of Ports: 2 Internal only
Encryption: Not Set
Driver Name: hpsa
Driver Version: 3.4.20
WWN Port: 500143803116CD50
Driver Supports SSD Smart Path: True
PCI Address (Domain:Bus:Device.Function): 0000:02:00.0
Port Max Phy Rate Limiting Supported: False
Host Serial Number: CZ240105DW
Sanitize Erase Supported: False
Primary Boot Volume: logicaldrive 1 (600508B1001CC725CF1BE202E3282EBE)
Secondary Boot Volume: None
SPDM Supports Get Slot Certificate Chain: no
SPDM Supports Get Controller Info : no
SPDM Supports Get Slot Info : no
SPDM Supports Set Import Certificate : no
SPDM Supports Set Invalidate Slot : no
Surface Scan Completion Supported: False
Persistent Event Log Policy Change Supported: False
UEFI Health Reporting Mode Supported: False
Internal Drive Cage at Port 1I, Box 1 (Index 0), OK
Drive Bays: 4
Port: 1I
Box: 1
Box Index: 0
Location: Internal
Physical Drives
physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SATA SSD, 500 GB, OK)
physicaldrive 1I:1:2 (port 1I:box 1:bay 2, SATA SSD, 500 GB, OK)
physicaldrive 1I:1:3 (port 1I:box 1:bay 3, SAS HDD, 1.2 TB, OK)
Internal Drive Cage at Port 2I, Box 0 (Index 1), OK
Drive Bays: 1
Port: 2I
Box: 0
Box Index: 1
Location: Internal
Physical Drives
None attached
Port Name: 1I
Port ID: 0
Port Connection Number: 0
SAS Address: 500143803116CD50
Port Location: Internal
Port Phy Count: 4
Port Name: 2I
Port ID: 1
Port Connection Number: 1
SAS Address: 500143803116CD54
Port Location: Internal
Port Phy Count: 4
Array: A
Interface Type: Solid State SATA
Unused Space: 0 MB (0.00%)
Used Space: 931.46 GB (100.00%)
Status: OK
Array Type: Data
Smart Path: disable
Logical Drive: 1
Size: 465.73 GB
Fault Tolerance: 1
Heads: 255
Sectors Per Track: 32
Cylinders: 65535
Strip Size: 256 KB
Full Stripe Size: 256 KB
Status: OK
Unrecoverable Media Errors: None
Caching: Disabled
Last Surface Scan Completed: False
Unique Identifier: 600508B1001CC725CF1BE202E3282EBE
Disk Name: /dev/sda
Mount Points: None
Boot Volume: Primary
Logical Drive Label: AA4C060D00143803116CD50AE57
Mirror Group 1:
physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SATA SSD, 500 GB, OK)
Mirror Group 2:
physicaldrive 1I:1:2 (port 1I:box 1:bay 2, SATA SSD, 500 GB, OK)
Drive Type: Data
LD Acceleration Method: All disabled
physicaldrive 1I:1:1
Port: 1I
Box: 1
Bay: 1
Status: OK
Drive Type: Data Drive
Interface Type: Solid State SATA
Size: 500 GB
Drive exposed to OS: False
Logical/Physical Block Size: 512/512
Firmware Revision: SVT01B6Q
Serial Number: S6P6NZFR700630E
WWID: 300143803116CD40
Model: ATA Samsung SSD 870
SATA NCQ Capable: True
SATA NCQ Enabled: True
Current Temperature (C): 38
Maximum Temperature (C): 57
SSD Smart Trip Wearout: Not Supported
PHY Count: 1
PHY Transfer Rate: 6.0Gbps
PHY Physical Link Rate: Unknown
PHY Maximum Link Rate: Unknown
Drive Authentication Status: OK
Carrier Application Version: 11
Carrier Bootloader Version: 6
Sanitize Erase Supported: False
Shingled Magnetic Recording Support: None
Multi-Actuator Drive: False
physicaldrive 1I:1:2
Port: 1I
Box: 1
Bay: 2
Status: OK
Drive Type: Data Drive
Interface Type: Solid State SATA
Size: 500 GB
Drive exposed to OS: False
Logical/Physical Block Size: 512/512
Firmware Revision: SVT01B6Q
Serial Number: S6P6NZFR700631R
WWID: 300143803116CD41
Model: ATA Samsung SSD 870
SATA NCQ Capable: True
SATA NCQ Enabled: True
Current Temperature (C): 38
Maximum Temperature (C): 63
SSD Smart Trip Wearout: Not Supported
PHY Count: 1
PHY Transfer Rate: 6.0Gbps
PHY Physical Link Rate: Unknown
PHY Maximum Link Rate: Unknown
Drive Authentication Status: OK
Carrier Application Version: 11
Carrier Bootloader Version: 6
Sanitize Erase Supported: False
Shingled Magnetic Recording Support: None
Multi-Actuator Drive: False
Array: B
Interface Type: SAS
Unused Space: 0 MB (0%)
Used Space: Unknown (0%)
Status: Failed Physical Drive
Array Type: Data
Smart Path: disable
Warning: One of the drives on this array have failed or has been removed.
Logical Drive: 2
Size: 1.09 TB
Fault Tolerance: 0
Heads: 255
Sectors Per Track: 32
Cylinders: 65535
Strip Size: 256 KB
Full Stripe Size: 256 KB
Status: Failed
Caching: Disabled
Last Surface Scan Completed: False
Unique Identifier: 600508B1001C2D73238401C46BA550F0
Drive Type: Data
LD Acceleration Method: All disabled
physicaldrive 1I:1:4
Port: 1I
Box: 1
Bay: 4
Status: Failed
Last Failure Reason: Write protected
Drive Type: Data Drive
Interface Type: SAS
Size: 1.2 TB
Drive exposed to OS: False
Logical/Physical Block Size: 512/512
Rotational Speed: 10500
Firmware Revision: HPD1
Serial Number: WFK11GYJ
WWID: 5000C500B86E9E41
Model: HP EG001200JWJNQ
PHY Count: 2
PHY Transfer Rate: 6.0Gbps, Unknown
PHY Physical Link Rate: Unknown, Unknown
PHY Maximum Link Rate: Unknown, Unknown
Drive Authentication Status: Not Applicable
Sanitize Erase Supported: False
Shingled Magnetic Recording Support: None
Multi-Actuator Drive: False
Array: C
Interface Type: SAS
Unused Space: 0 MB (0.00%)
Used Space: 1.09 TB (100.00%)
Status: OK
Array Type: Data
Smart Path: disable
Logical Drive: 3
Size: 1.09 TB
Fault Tolerance: 0
Heads: 255
Sectors Per Track: 32
Cylinders: 65535
Strip Size: 256 KB
Full Stripe Size: 256 KB
Status: OK
Caching: Disabled
Last Surface Scan Completed: False
Unique Identifier: 600508B1001C45A5717995EADF4CD3FF
Disk Name: /dev/sdb
Mount Points: None
Logical Drive Label: Logical Drive 3
Drive Type: Data
LD Acceleration Method: All disabled
physicaldrive 1I:1:3
Port: 1I
Box: 1
Bay: 3
Status: OK
Drive Type: Data Drive
Interface Type: SAS
Size: 1.2 TB
Drive exposed to OS: False
Logical/Physical Block Size: 512/512
Rotational Speed: 10500
Firmware Revision: HPD1
Serial Number: WFK16CQN
WWID: 5000C500B8AE9A25
Model: HP EG001200JWJNQ
Current Temperature (C): 38
Maximum Temperature (C): 56
PHY Count: 2
PHY Transfer Rate: 6.0Gbps, Unknown
PHY Physical Link Rate: Unknown, Unknown
PHY Maximum Link Rate: Unknown, Unknown
Drive Authentication Status: OK
Carrier Application Version: 11
Carrier Bootloader Version: 6
Sanitize Erase Supported: True
Sanitize Estimated Max Erase Time: 1 hour(s), 55 minute(s)
Unrestricted Sanitize Supported: True
Shingled Magnetic Recording Support: None
Multi-Actuator Drive: False
SEP (Vendor ID PMCSIERA, Model SRCv8x6G) 380
Device Number: 380
Firmware Version: RevB
WWID: 500143803116CD5F
Vendor ID: PMCSIERA
Model: SRCv8x6G