替换RAID中损坏的磁盘

背景

在18年3月份构建的RAID磁盘阵列，其中一个磁盘出现了磁盘主轴不能转动的问题。果然不是为了24×7工作的磁盘长期不断电使用实际出现了问题。由于预料到可能出现磁盘损坏的问题，在购买磁盘时果断购买了2年只换不修服务，然后顺利换到了新磁盘。接下来的事情就是为RAID阵列增加新的磁盘。

增加磁盘

首先确认下当前RAID的状态

root@h:~# mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Wed Mar  7 14:13:44 2018
        Raid Level : raid1
        Array Size : 3906887424 (3725.90 GiB 4000.65 GB)
     Used Dev Size : 3906887424 (3725.90 GiB 4000.65 GB)
      Raid Devices : 2
     Total Devices : 1
       Persistence : Superblock is persistent

     Intent Bitmap : Internal

       Update Time : Wed Dec 12 03:19:11 2018
             State : clean, degraded
    Active Devices : 1
   Working Devices : 1
    Failed Devices : 0
     Spare Devices : 0

Consistency Policy : bitmap

              Name : h:0  (local to host h)
              UUID : xxxxxxxx:xxxxxxxx:xxxxxxxx:xxxxxxxx
            Events : 12127

    Number   Major   Minor   RaidDevice State
       0       8       17        0      active sync   /dev/sdb1
       -       0        0        1      removed

root@h:~# mdadm --detail /dev/md0

/dev/md0:

Version : 1.2

Creation Time : Wed Mar 7 14:13:44 2018

Raid Level : raid1

Array Size : 3906887424 (3725.90 GiB 4000.65 GB)

Used Dev Size : 3906887424 (3725.90 GiB 4000.65 GB)

Raid Devices : 2

Total Devices : 1

Persistence : Superblock is persistent

Intent Bitmap : Internal

Update Time : Wed Dec 12 03:19:11 2018

State : clean, degraded

Active Devices : 1

Working Devices : 1

Failed Devices : 0

Spare Devices : 0

Consistency Policy : bitmap

Name : h:0 (local to host h)

UUID : xxxxxxxx:xxxxxxxx:xxxxxxxx:xxxxxxxx

Events : 12127

Number Major Minor RaidDevice State

0 8 17 0 active sync /dev/sdb1

- 0 0 1 removed

目前RAID1阵列处于降级工作状态。

将新磁盘安装到机器上，使用gdisk工具重新分区，保存后记录下新分区的PART-UUID。接下来使用mdadm添加新磁盘

root@h:~# mdadm --manage /dev/md0 --add /dev/disk/by-partuuid/[新PART-UUID]
mdadm: added /dev/disk/by-partuuid/[新PART-UUID]

root@h:~# mdadm --manage /dev/md0 --add /dev/disk/by-partuuid/[新PART-UUID]

mdadm: added /dev/disk/by-partuuid/[新PART-UUID]

可以看到新的磁盘已经添加到阵列中了。接下来查看阵列状态

root@h:~# mdadm --detail /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Wed Mar  7 14:13:44 2018
        Raid Level : raid1
        Array Size : 3906887424 (3725.90 GiB 4000.65 GB)
     Used Dev Size : 3906887424 (3725.90 GiB 4000.65 GB)
      Raid Devices : 2
     Total Devices : 2
       Persistence : Superblock is persistent

     Intent Bitmap : Internal

       Update Time : Wed Dec 12 22:53:08 2018
             State : clean, degraded, recovering
    Active Devices : 1
   Working Devices : 2
    Failed Devices : 0
     Spare Devices : 1

Consistency Policy : bitmap

    Rebuild Status : 0% complete

              Name : h:0  (local to host h)
              UUID : xxxxxxxx:xxxxxxxx:xxxxxxxx:xxxxxxxx
            Events : 12130

    Number   Major   Minor   RaidDevice State
       0       8       17        0      active sync   /dev/sdb1
       2       8       49        1      spare rebuilding   /dev/sdd1

root@h:~# mdadm --detail /dev/md0

/dev/md0:

Version : 1.2

Creation Time : Wed Mar 7 14:13:44 2018

Raid Level : raid1

Array Size : 3906887424 (3725.90 GiB 4000.65 GB)

Used Dev Size : 3906887424 (3725.90 GiB 4000.65 GB)

Raid Devices : 2

Total Devices : 2

Persistence : Superblock is persistent

Intent Bitmap : Internal

Update Time : Wed Dec 12 22:53:08 2018

State : clean, degraded, recovering

Active Devices : 1

Working Devices : 2

Failed Devices : 0

Spare Devices : 1

Consistency Policy : bitmap

Rebuild Status : 0% complete

Name : h:0 (local to host h)

UUID : xxxxxxxx:xxxxxxxx:xxxxxxxx:xxxxxxxx

Events : 12130

Number Major Minor RaidDevice State

0 8 17 0 active sync /dev/sdb1

2 8 49 1 spare rebuilding /dev/sdd1

看到阵列正在重建。

修改配置

接下来修改/etc/mdadm/mdadm.conf，将旧磁盘的by-partuuid方式查找的路径更换为新磁盘的PART-UUID。这样保证下次重启后, RAID也可以正常工作。

最后看下同步的进度:

root@h:~# cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]
md0 : active raid1 sdd1[2] sdb1[0]
      3906887424 blocks super 1.2 [2/1] [U_]
      [>....................]  recovery =  0.1% (7385216/3906887424) finish=527.7min speed=123146K/sec
      bitmap: 13/30 pages [52KB], 65536KB chunk

root@h:~# cat /proc/mdstat

Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]

md0 : active raid1 sdd1[2] sdb1[0]

3906887424 blocks super 1.2 [2/1] [U_]

[>....................] recovery = 0.1% (7385216/3906887424) finish=527.7min speed=123146K/sec

bitmap: 13/30 pages [52KB], 65536KB chunk

看起来还需要同步8个多小时。接下来就不用管了，等阵列自动重建完即可。
更换工作完成~