30-04-2021

Synology Storage Manager

Instantly sync cloud-managed data to multiple Synology NAS and access files as if they were locally stored, reducing on-premises storage footprint, network use, and costs. Instant data recovery Mount a Hybrid Share folder to any Synology NAS and gain instant access to your data on Synology C2 Storage. DiskStation Manager (DSM) is an intuitive web-based operating system for every Synology NAS, designed to help you manage your digital assets across home and office.

Overview of Synology's DiskStation Manager 6.0 NAS management software ruinning on the DS216+, more information on & Ava. Go to: Storage Manager = HDD/SSD = Select Disk 2 = Select Health Info There may be some hint there as to the problem with that drive. Also, it would be a good idea to run a 'Smart Test' on drive 2. Go to: Storage Manager = HDD/SSD = Select Disk 2 = Select S.M.A.R.T.

Small steps to big protection for your storage

Data scrubbing, as the name suggests, is a process of inspecting volumes and modifying the detected inconsistencies. As times goes by, some data may fall victim to slow degradation that gradually deteriorates data integrity. Worse still, they occur silently without any warning. Take photos as an example. It could be a real disaster if it happens to one of your precious photos capturing the indelible memories. The two images below are the original photo and the corrupt one that suffers from bit rot. Read on to see how data scrubbing prevents your digital assets from data corruption.

Before we go into detail about data scrubbing, let us introduce RAID arrays to you first. RAID stands for redundant array of independent disks. Simply put, it combines multiple drives into a single storage pool, offering fault tolerance and data redundancy. Here we’re going to briefly introduce RAID 5.

RAID 5: It requires at least three drives and utilizes parity striping at the block level. When writing a block of sequential data into the array, for example, RAID 5 will write it into A1, A2, A3, B1, B2, B3 in sequence. Likewise, it reads data in the same order. What about Pa, Pb, and Pc? They are parity blocks distributed across the drives. When writing A1, A2, and A3, RAID 5 will use the following XOR to calculate Pa and write it to the corresponding block.

Pa = A1 (XOR) A2 (XOR) A3 (Function 1)

If one of the drives fails, RAID 5 will repair the missing data by using Pa and contents of the remaining two drives. Suppose the drive containing A2 breaks, then we can perform the following XOR calculation to reconstruct it:

A2 = A1 (XOR) A3 (XOR) Pa (Function 2)

The recovered contents are what we call a redundant copy. This is how RAID 5 achieves redundancy, protecting your data against drive failure.

RAID Scrubbing

Now that we have a basic understanding of the characteristics of RAID 5, then we can go on to talk about data consistency. First of all, we know that the parity information in each drive should satisfy Function 1 shown above. If it holds true, then we can safely say that the data in the array is consistent. Upon the failure of a single drive, we can use Function 2 to calculate the redundant copy and recover the contents accordingly. If it proves wrong, then there’s a problem of data inconsistency because the reconstructed data will be incorrect.

Failure to recover your data is something serious, so it’s vital to retain data consistency. RAID scrubbing scans all the contents in an array, making sure all the parity stripes satisfy Function 1. If it fails to fulfill the XOR function, it will be recalculated again and again until all the values are consistent.

“Will my data always remain intact if I set a regular RAID scrubbing schedule?”

Unfortunately, the answer is no. We cannot make sure that the data written to the drives will always be accurate. Some data corruption goes unnoticed. It can occur during the write-to-drive process without being reported. This kind of errors are caused by various reasons: hardware errors, electromagnetic interference, and many more.

The problem is that RAID scrubbing can only ensure data consistency. That is, it cannot tell which data block is incorrect. If a block is corrupted, every other block will be “consistently corrupt” as well. Sole reliance on RAID scrubbing may pose a potential risk. Say today we want to reconstruct Pa using A1, A2, and A3 (as shown in Function 1 above), but if any one of A1, A2, or A3 is corrupt, then performing the function will go awry, only to yield the wrong content and make things even worse.

This is where Btrfs data scrubbing comes in.

As you may have noticed, not only can you see whether your RAID type supports RAID scrubbing, but you can also know if file system scrubbing is supported from the volume information under the Data Scrubbing tab in Storage Manager¹.

Btrfs data scrubbing

File system data scrubbing employs the checksum mechanism to check the volumes in the Btrfs file system. If any data that is inconsistent with the checksum is detected, the system will try to use the redundant copy to repair the data. Once you enable data checksum when creating a shared folder, the Btrfs file system will calculate a checksum (data checksum) for every written file, and further protect that data checksum with another checksum (metadata checksum).

Every time data scrubbing is conducted, the file system will recalculate the checksum and compare it with the previously stored data checksum. Meanwhile, the data checksum will cross-check its corresponding metadata checksum to make sure the data checksum itself is intact. In other words, if the recalculated checksum does not accord with the data checksum, a cross-check with its metadata checksum will be followed to see whether it is the file or data checksum that goes wrong. Once data corruption is detected, the system will try to repair the corrupt data by retrieving the redundant copy (RAID 5).

One thing to note, though, is that Btrfs data checksum may take a toll on system performance. It’s not suggested to enable data checksum if it’s a shared folder storing databases, virtual machines, or surveillance video recordings. Rest easy if you only store documents or photos in shared folders or if you use these folders for file access or sharing, as it has a very modest influence on performance.

Keeping data integrity risk at bay

Synology Storage Manager Disk Not Initialized

Can’t decide which data scrubbing you should employ? No worries. You can have it both ways. Synology’s Data Scrubbing²integrates Btrfs data scrubbing and RAID scrubbing to ensure data integrity. When running data scrubbing on a Btrfs volume, file system data scrubbing will be performed first to make sure the data is accurate. RAID scrubbing will be implemented next to achieve data consistency. They work together to mitigate the risk of silent data corruption and help you maintain a healthy storage system.

Tell us what you think about data scrubbing on Synology Community.

¹ RAID scrubbing is not supported in mirroring-based RAID (RAID 1 & RAID 10) arrays. It’s only available for parity-based RAID types.

² Synology’s Data Scrubbing is supported on DSM 6.2.

My Synology DS918+ has two Seagate Ironwolf NAS 6TB drives in it, currently configured in a Storage Pool. With two drives, one is essentially a mirror of the other, so the total storage available is only 6TB. Well 5TB if you 'count' the way hard drive manufacturers do.

It's now time to add to this storage. I called C-Zone at PJ Digital Mall and was told they had the drive in stock. Unfortunately, when I got there, it turns out there was a mix up and they didn't have actually have them there, only in their Lowyatt Plaza branch. Which meant a drive into the city.

In the rain.

Synology Storage Manager Download

Fortunately it was a Sunday so there weren't too many cars on the road. In the end I managed to get my hands on another Seagate Ironwolf NAS 6TB drive.

I started by pulling out the empty caddy for slot #3. Then a small flat screwdriver was used to prise off the panels at the side.

The new drive then is placed in the caddy, with the holes aligned, and then the panels at the side are pushed in until they lie flat on the sides.

The caddy then slides back into the slot, going all the way in until the SATA connectors mate.

It takes about 20-30 seconds for the drive to power up and be recognized by the system, at which point the DISK 3 LED lights up.

Logging into the NAS and firing up the Storage Manager's HDD/SSD screen shows that Drive 3 is recognized but 'Not Initialized'.

To add the drive, I selected the Storage Pool screen and selected 'Add Drive' from the Action button-menu:

Then choose the new 'Drive 3' and press Next:

Confirm by pressing OK:

Then press Apply:

Synology Storage Manager Remove Volume

And we're returned to the Storage Pool screen. There's a moving symbol at the four squares to indicate it's busy doing stuff. The Drive first changes from 'Not Initialized' and then to 'Normal', after which the Storage Pool is Expanded to encompass the new drive. This process can take a long time as it has to recalculate the parity bits across the 3 drives.

Synology Storage Manager Repair

The existing data on the volume is still available while it's going through this process, although access speed is reduced. I was still able to stream HD video from my Plex server running on the NAS without any issues (4K would end up buffering every 20 seconds or so), although putting any load on the NAS may slow down the time it takes to expand the Storage Pool. I estimate that it took around 36 hours for my three 6TB Ironwolf NAS drives.

After that, the Storage pool had a capacity of 11TB:

Synology Storage Manager

The Volume is still using only the initial 5.24TB of space, however:

Synology Storage Manager Data Scrubbing

The last step is to select 'Volume 1' in the Volume screen, then choose 'Configure' from the 'Actions' menu. Click the 'Max' button next to the 'Modify allocated size' field, and press OK:

The NAS then starts expanding the file system to use all the available space in the Storage Pool. This took about 15 seconds, and finally:

And that's all there is to it.

The DSM UI on the Synology NAS makes expanding the storage an absolute doddle compared to using mdadmon a PC-based linux system. My ideal setup is now to use just SSDs for fast storage within the PC, and store everything else on the NAS. Of course you need wired gigabit networking connecting everything together, but for most desktop PCs this is not an issue. Hopefully this allows me to reduce the size of my desktop PCs eventually.