Raspberry Pi raid array with USB HDDs

Create a Raspberry Pi raid array with USB HDDs  using mdadm

raspberry pi raid assembly
raspberry pi raid assembly

Create a Raid 0, or Raid 1 or Raid 5 disk array with a Raspberry Pi 2.  Once the Raid array is assembled, you could then create a  Network attached files server or mini dna server. This project is ideal for when you want to combine the capacity of a number of drives to create a single large volume or when planning to create a file server that stores important data like family photographs or documents.

Before you begin, please read about about raid assemblies, their advantages and disadvantages. Also read about data back ups using raid compared to redundant or incremental backups.

This tutorial uses mdadm and covers setting up a raid 0 array ( where two hard drives of 1tb each are combined to form a single volume that has a 2tb capacity(thanks Mario 🙂 ). You could very easily setup a raid 1 or a raid 5 assembly by replacing the mdadm configuration. Mdadm is a Linux utility used to manage and monitor software RAID devices.

Please bookmark and read the Mdadm cheat sheet while configuring the Raid assembly.

 

Overview :

  • Connect  Hard Drives to the Raspberry Pi .
  • Install mdadm to create the raid assembly
  • Configure the raid assembly.

Time and Difficulty :

  • Time : 15 to 30 mins  once you have the Raspbian Image file and the rest of the software.
  • Newbie Difficulty level : Moderately easy ; Will need to use terminal and type in commands.

What you will need :

  • Raspberry Pi 2 Model B or one of its clones like Banana Pi M1 or M2 or Banana Pro.
  • Ethernet connection or a supported USB Wifi adapter for the Pi
  • SD or Micro SD Card that fits into your Pi (4GB or above is recommended)
  • SSH connection to the Pi or a USB Keyboard and Mouse
  • Connection to a TV or Monitor, if, not connecting via SSH.

STEP 1 : Update the Pi 

Log in to Raspbian and enter the following commands :

  1. sudo apt-get update
  2. sudo apt-get upgrade
  3. sudo apt-get dist-upgrade

The steps above will update the software and the raspbian distribution to the latest available.

reboot your pi with sudo reboot

STEP 2 : Connect the USB HDDs

Connect the USB hard disk to the Pi via a powered USB hub.  In this tutorial, we are using 2 WD portable drives of 1 TB capacity each. Run the following commands at the terminal

sudo fdisk -l (To list the drives connected to the system. The two USB drives are shown as /dev/sda1 and /dev/sdb1)

Use fdisk to list connected drives
Use fdisk to list connected drives

STEP 3 : Install Mdadm

To install mdadm, you will need to elevate to root privileges.  Enter the following commands at the terminal.

  1. sudo -i  (Elevate current login to root level)
  2. apt-get install mdadm  (Install mdadm, Choose the default “All” during installation to proceed. )
  3. mdadm -Cv /dev/md0 -l0 -n2 /dev/sd[ab]1  ( configure mdadm and create a raid array at /dev/md0 using raid0 with 2 disks ; sda1 and sdb1. To create a raid1, replace the line to read mdadm -Cv /dev/md0 -l1 -n2 /dev/sd[ab]1 )
  4. fdisk -l  (Use fdisk to check the drives, once the Raid array is created. At the very end you will see a new disk /dev/md0 with the new capacity. )once mdadm is done, use fdisk -l to verify the new raid drive
  5. once mdadm is done, use fdisk -l to verify the new raid drive
  6. The new Disk /dev/md0 doesn’t yet contain a valid partition table
  7. cat /proc/mdstat to check the status of the mdadm and drives.
  8. mkfs /dev/md0 -t ext4  (To create an ext4 file system at the newly created raid drive)use makefs to create an ext4 file system on the raid drive
  9. use makefs to create an ext4 file system on the raid drive
  10.  Use mdadm --detail /dev/md0 to check list the details of the raid array.Use mdadm --detail /dev/md0 to check the status of the raid
    Use mdadm –detail /dev/md0 to check the status of the raid

You can now automount the new created drive so that it is available at boot.

Update :  This a top tip from AxisMann in the forum below. Thanks AM!

AxisMann

What I read somewheres is that the raspberry pi version 2 was booting up two fast for the raid array to be recognized. I eliminated the problem by following someone’s instructions to add rootdelay=5 to the cmdline.txt file in the /boot directory as a work around. That seemed to work for my problem where the raid array wasn’t being recognized on reboot.

 

 

20 thoughts on “Raspberry Pi raid array with USB HDDs”

  1. I did the above procedure and all went great, but when I reboot raspberry the system doesn’t automatically recognize the existing disk array.
    # mdadm –detail /dev/md0
    mdadm: cannot open /dev/md0: No such file or directory

    *FYI:
    1) I have already update the file /etc/mdadm/mdadm.conf by entering “mdadm –detail –scan >> /etc/mdadm/mdadm.conf”
    following the instructions by http://www.ducea.com/2009/03/08/mdadm-cheat-sheet/
    2) along with the command “update-initramfs -u”
    3) and finally I created the corresponding entry in the /etc/fstab following the instructions by http://projpi.com/raspberry-pi-tips-and-hacks/raspberry-pi-automount-usb/

    The good news is that the array is there and it can be reassigned to /dev/md0 by entering
    “mdadm -A /dev/md0 /dev/sda1 /dev/sdb1”

    Is there anyone who achieved to make it “immune” to reboots?

    1. hey ZN. Write a small script that has the command and have it execute everytime the Pi boots up by adding the script to /etc/init.d/. Should work. A word of warning though, I find the Raid arrangement running on a Pi more of a “proof of concept” thing. Please test it thoroughly before storing important data. If you are planning to use the Raid for mirroring rather than striping, considering a rsync script that executes at periodic intervals. Hope this helps 🙂

      1. Hi d3v, thanks for your response! Actually I am building an owncloud server following your guide! Adding my little experience about that I think, I will follow your advice about rsync, in order to make things simpler!

        1. I know this comment is almost a year old but I found this post while doing some research into setting up this exact project. Raspi 3 with software raid across 4 cheap 64GB thumb drives and an owncloud server running. How’d yours go? Get everything working?

    2. What I read somewheres is that the raspberry pi version 2 was booting up two fast for the raid array to be recognized. I eliminated the problem by following someone’s instructions to add rootdelay=5 to the cmdline.txt file in the /boot directory as a work around. That seemed to work for my problem where the raid array wasn’t being recognized on reboot.

      1. Thanks AxisMann. That is a great tip. Will update the post. I have tested it as a proof of concept project. Personally, I recommend, using rsync to create a mirror instead of Raid for all projects where the intent is to create a backup drive which replicates and stores data.

    3. Looks like you’re having the issue AxxisMann is describing above:
      ” add rootdelay=5 to the cmdline.txt file in the /boot directory”

  2. Hi. I built a raid 1 array following your instructions and others on Rasberry Pi. It’s working and by editing fstab accordingly, the array mounts when I reboot. I am using one of those boxes with 1 USB connection for the computer and a capacity of 4 drives. There are only 2 present and they are identical 2 TB drives.

    The problem is that the drives seem to be running all the time. The HD activity LEDs are constantly flickering. The box has a fan which I have set on automatic but, because the drives are constantly on and, I suppose, hot, the fan has adjusted to the highest speed.

    Do you have any idea why the drives are constantly running? I assumed the only time they’d run is when I’m putting data on and a copy of the data is being placed on the second drive.

  3. To my previous question, I should add that when I run the command sudo mdadm –detail /dev/md0 the output always says “State : active, resyncing” and “Resync Status : 53% complete”. I’ve noticed on your image there is no display of either. It looks like something is hanging up. Here’s my complete outout:
    sudo mdadm –detail /dev/md0
    /dev/md0:
    Version : 1.2
    Creation Time : Thu Sep 24 16:28:17 2015
    Raid Level : raid1
    Array Size : 1953382400 (1862.89 GiB 2000.26 GB)
    Used Dev Size : 1953382400 (1862.89 GiB 2000.26 GB)
    Raid Devices : 2
    Total Devices : 2
    Persistence : Superblock is persistent

    Intent Bitmap : Internal

    Update Time : Sat Sep 26 00:37:14 2015
    State : active, resyncing
    Active Devices : 2
    Working Devices : 2
    Failed Devices : 0
    Spare Devices : 0

    Resync Status : 53% complete

    Name : PiServer:0 (local to host PiServer)
    UUID : de86f87a:b07c16f3:a9e05954:25cae8c2
    Events : 21938

    Number Major Minor RaidDevice State
    0 8 1 0 active sync /dev/sda1
    1 8 17 1 active sync /dev/sdb1

    1. Hey Bob. try using hd-idle to get the drives to spin down when not in use. As far as Raid is concerned, we have only tried it as a “proof of concept” project. For all practical projects, we have been using Rsync set to run periodically and make recursive backups. Hope this helps 🙂 cheers. dev

  4. I used thumb drives instead of hard drives for my storage needs since all I want to do was backup my documents. I settled on two 32gb thumb drives arranged in a raid 1 array. I used the Linux Backup utility to backup my documents directory weekly. The first backup was kind of slow but after that, the subsequent incremental backups go rather quickly. It works great. I can recover an older version of a document when necessary. It’s quite and always available. I plugged it into my UPS so that it hardly every reboots. It’s been up 124 days. Of course, a used hard drive might have been cheaper backup device for my purposes.

  5. Is it a necessity to plug all drives in to a single USB hub? If I wanted to use my Pi as a RAID NAS only, could I plug each constituent HDD in to a different USB slot? (assuming I was using a maximum of four HDDs)

    I’m looking for a way to offload my media to a quiet corner of the house, and just run a networked media client in the lounge / bedroom. I’m not too worried about data write rates, but access should be as optimal as possible.

    1. The idea behind a USB hub is that it can be POWERED (with the appropriate model) to provide juice to the hard drives. Even the Pi 3 still runs a single USB channel so even though there are 4 ports, they are not truly independent. The Pi models also do not provide much power to the USB. You can get away with a single 3.5″, keyboard, and mouse, but running a second 3.5″ reliably isn’t recommended even if it works. Hence, again, the reasoning to use a powered hub so at least the lack of energy issue is solved.

      1. Sorry should be 2.5″ laptop-style drive, not 3.5″ desktop – which would be much too demanding for any USB power.

  6. My drives are “busy” and I can’t set them up in a raid…

    root@raspberrypi:~# mdadm -Cv /dev/md0 -l0 -n2 /dev/sd[ab]1
    mdadm: super1.x cannot open /dev/sda1: Device or resource busy
    mdmon: ddf: Cannot use /dev/sda1: Device or resource busy
    mdmon: Cannot use /dev/sda1: It is busy
    mdadm: cannot open /dev/sda1: Device or resource busy

  7. I followed the steps but when i got to the mdadm create it wouldn’t work

    root@raspberrypi:~# mdadm -Cv /dev/md0 -l1 -n2 /dev/sd[ab]1
    mdadm: super1.x cannot open /dev/sda1: Device or resource busy
    mdmon: ddf: Cannot use /dev/sda1: Device or resource busy
    mdmon: Cannot use /dev/sda1: It is busy
    mdadm: cannot open /dev/sda1: Device or resource busy

    1. Ok… i should have checked this before i posted a comment….

      It seems that somehow the sda1 and sdb1 got mounted. I simply umounted them and continued with mdadm -Cv /dev/md0 -l1 -n2 /dev/sd[ab]1

      root@raspberrypi:~# df -h
      Filesystem Size Used Avail Use% Mounted on
      /dev/root 29G 3.5G 25G 13% /
      devtmpfs 214M 0 214M 0% /dev
      tmpfs 218M 0 218M 0% /dev/shm
      tmpfs 218M 4.7M 213M 3% /run
      tmpfs 5.0M 4.0K 5.0M 1% /run/lock
      tmpfs 218M 0 218M 0% /sys/fs/cgroup
      /dev/mmcblk0p1 63M 21M 43M 33% /boot
      tmpfs 44M 0 44M 0% /run/user/1000
      /dev/sdb1 2.7T 74M 2.6T 1% /media/pi/2487ee8e-c76b-4da6-b66d-9caff4fb640a
      /dev/sda1 2.7T 74M 2.6T 1% /media/pi/f63fb264-fdb5-4741-95e8-7280446e9e88
      root@raspberrypi:~# umount /dev/sda1
      root@raspberrypi:~# umount /dev/sdb1
      root@raspberrypi:~# df -h
      Filesystem Size Used Avail Use% Mounted on
      /dev/root 29G 3.5G 25G 13% /
      devtmpfs 214M 0 214M 0% /dev
      tmpfs 218M 0 218M 0% /dev/shm
      tmpfs 218M 4.7M 213M 3% /run
      tmpfs 5.0M 4.0K 5.0M 1% /run/lock
      tmpfs 218M 0 218M 0% /sys/fs/cgroup
      /dev/mmcblk0p1 63M 21M 43M 33% /boot
      tmpfs 44M 0 44M 0% /run/user/1000
      root@raspberrypi:~# mdadm -Cv /dev/md0 -l1 -n2 /dev/sd[ab]1
      mdadm: /dev/sda1 appears to contain an ext2fs file system
      size=2930232320K mtime=Tue May 31 19:15:27 2016
      mdadm: Note: this array has metadata at the start and
      may not be suitable as a boot device. If you plan to
      store ‘/boot’ on this device please ensure that
      your boot-loader understands md/v1.x metadata, or use
      –metadata=0.90
      mdadm: /dev/sdb1 appears to contain an ext2fs file system
      size=2930232320K mtime=Tue May 31 19:15:25 2016
      mdadm: size set to 2930101248K
      mdadm: automatically enabling write-intent bitmap on large array
      Continue creating array? y
      mdadm: Defaulting to version 1.2 metadata
      mdadm: array /dev/md0 started.
      root@raspberrypi:~# fdisk -l

      Disk /dev/ram0: 4 MiB, 4194304 bytes, 8192 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 4096 bytes
      I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      Disk /dev/ram1: 4 MiB, 4194304 bytes, 8192 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 4096 bytes
      I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      Disk /dev/ram2: 4 MiB, 4194304 bytes, 8192 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 4096 bytes
      I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      Disk /dev/ram3: 4 MiB, 4194304 bytes, 8192 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 4096 bytes
      I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      Disk /dev/ram4: 4 MiB, 4194304 bytes, 8192 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 4096 bytes
      I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      Disk /dev/ram5: 4 MiB, 4194304 bytes, 8192 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 4096 bytes
      I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      Disk /dev/ram6: 4 MiB, 4194304 bytes, 8192 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 4096 bytes
      I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      Disk /dev/ram7: 4 MiB, 4194304 bytes, 8192 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 4096 bytes
      I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      Disk /dev/ram8: 4 MiB, 4194304 bytes, 8192 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 4096 bytes
      I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      Disk /dev/ram9: 4 MiB, 4194304 bytes, 8192 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 4096 bytes
      I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      Disk /dev/ram10: 4 MiB, 4194304 bytes, 8192 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 4096 bytes
      I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      Disk /dev/ram11: 4 MiB, 4194304 bytes, 8192 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 4096 bytes
      I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      Disk /dev/ram12: 4 MiB, 4194304 bytes, 8192 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 4096 bytes
      I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      Disk /dev/ram13: 4 MiB, 4194304 bytes, 8192 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 4096 bytes
      I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      Disk /dev/ram14: 4 MiB, 4194304 bytes, 8192 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 4096 bytes
      I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      Disk /dev/ram15: 4 MiB, 4194304 bytes, 8192 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 4096 bytes
      I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      Disk /dev/mmcblk0: 29.3 GiB, 31440502784 bytes, 61407232 sectors
      Units: sectors of 1 * 512 = 512 bytes
      Sector size (logical/physical): 512 bytes / 512 bytes
      I/O size (minimum/optimal): 512 bytes / 512 bytes
      Disklabel type: dos
      Disk identifier: 0xfa1c056e

      Device Boot Start End Sectors Size Id Type
      /dev/mmcblk0p1 8192 137215 129024 63M c W95 FAT32 (LBA)
      /dev/mmcblk0p2 137216 61407231 61270016 29.2G 83 Linux

      Disk /dev/sda: 2.7 TiB, 3000558944256 bytes, 732558336 sectors
      Units: sectors of 1 * 4096 = 4096 bytes
      Sector size (logical/physical): 4096 bytes / 4096 bytes
      I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      Disklabel type: dos
      Disk identifier: 0x9239540a

      Device Boot Start End Sectors Size Id Type
      /dev/sda1 256 732558335 732558080 2.7T 83 Linux

      Disk /dev/sdb: 2.7 TiB, 3000558944256 bytes, 732558336 sectors
      Units: sectors of 1 * 4096 = 4096 bytes
      Sector size (logical/physical): 4096 bytes / 4096 bytes
      I/O size (minimum/optimal): 4096 bytes / 4096 bytes
      Disklabel type: dos
      Disk identifier: 0xf0126886

      Device Boot Start End Sectors Size Id Type
      /dev/sdb1 256 732558335 732558080 2.7T 83 Linux

      Disk /dev/md0: 2.7 TiB, 3000423677952 bytes, 732525312 sectors
      Units: sectors of 1 * 4096 = 4096 bytes
      Sector size (logical/physical): 4096 bytes / 4096 bytes
      I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Leave a Reply