Contents
Backup, Backup, Backup!
Hard drives, SSD’s, Flash Drives, SD Cards, and computers do fail without warning. Make sure you regularly backup your data, photos, videos, etc.
- Data should exist in a minimum of three devices/places. The 3-2-1 backup strategy means you have a total of three copies of your files. Two of which are local, but on different devices/drives. And one additional copy offsite.
- One of the places should be off-site.
- Your off-site backup should not be live synced.
- Test your backup procedures and medium to make sure,
- You’re backing up what you expected.
- You can successfully restore from the backup.
- RAID is not a backup.
Note: Any files with sensitive information, should be encrypted locally, before they’re uploaded/synchronized to the cloud.
Some options for Cloud storage are:
- Backblaze – Use a strong Private Encryption Key under Settings > Security in the client program.
- Mega (fully encrypted)
- Carbonite.com
- DopBox
- Box.com
- Google Drive
- Rclone syncs your files to cloud storage
Reference
- DATA STORAGE & BACKUPS: strategies for Filmmakers & Photographers
- How to Backup and Restore iOS Data – 2020 Complete Guide
NAS (Network Attached Storage)
A NAS (Network Attached Storage) system is a data storage device connected to a network that allows storage and retrieval of data from a centralized location for authorized network.
Best Practices
- Use RAID Level 6 or ZFS as the file system on NAS/SAN devices.
- Do not use RAID 5, if RAID 6 or ZFS is available!
- Try and get drives from different batches/lots, by not purchasing all the hard drives from the same vendor at the same time. Why? If a drive in the NAS fails, and is replaced, the NAS has to rebuild the RAID volume. The added stress of rebuilding my cause another drive to fail, thus, there may be insufficient data on the remaining drives to complete the rebuild of the replaced drive. Drives purchased at the same time, are probably from the same batch, and would’ve been in service for the same length of time. When a drive fails, what are the chances that another from the same batch may also fail?
CMR (Conventional Magnetic Recording) vs SMR (Shingled Magnetic Recording) Drives for NAS
- Do not use SMR drivers for NAS!!!! Use only CMR type drives.
- Seagate says Network Attached Storage and SMR don’t mix
- Seagate’s IronWolf and IronWolf Pro NAS drives are all certified SMR-free.
- Buyer beware—that 2TB-6TB “NAS” drive you’ve been eyeing might be SMR
- Western Digital admits 2TB-6TB WD Red NAS drives use shingled magnetic recording
- Surreptitiously Swapping SMR into Hard Drive Lines Must Stop
References
- Still confused? NAS (Network Attached Storage) explained in 3 minutes!
- Still confused about NAS? NAS explained in 3 minutes (SPANdotCOM)
- NAS 101: What is Network Attached Storage? (Lon.TV)
- NAS and SAN Introduction (Eli the Computer Guy)
- RAID (Redundant Array of Independent Disks) is a data storage virtualization technology that combines multiple physical disk drives into one or more logical units for the purposes of data redundancy, performance improvement, and expanded storage.
NAS Hardware
- The Best NAS for Most Home Users (TheWireCutter.com)
- Storinator is an open-storage solution with space for 15, 30, 45 or 60 drives.
- Synology creates network-attached storage (NAS), IP surveillance solutions
- NAS Builds Using the Fractal Design Node 304 case
SATA Expansion Cards
To add additional SATA ports, use a Serial Attached SCSI (SAS) Host Bus Adapter (HBA). Instead of SATA ports, these cards have SAS ports, to which you connect four SATA drives per port. So, a SAS HBA with only two SAS ports will allow you to hook up to eight SATA hard drives.
Certain cards will work out of the box, such as the LSI SAS 9300-8i HBA (SAS3008) and LSI SAS 9201-8i HBA (SAS2008). Other cards will have to be flashed with an alternative firmware to enable IT-mode. That way, the hard drives connected to the card will appear in Unraid as individual disks and the hardware RAID that an HBA supports, will be completely disabled.
- Don’t use cheap PCIe SATA expansion cards with Unraid (get a cheap HBA instead)
- Firmware for LSI Logic SAS9211-8I 8PORT Int 6GB Sata+SAS Pcie 2.0 can be downloaded from Broadcom.com
NAS Operating Systems
TrueNAS (formerly FreeNAS)
TrueNAS is an operating system, based on FreeBSD, that can be installed on virtually any hardware platform to share data over a network.
- TrueNAS combines RAID redundancy, ZFS, Replication, Snapshots, Automatic Corruption Repair, and optional High Availability.
- ZFS Primer
- It is highly recommend you use ECC RAM with “mission-critical” ZFS with TrueNAS.
- Systems with ECC RAM will correct single bit errors on the fly, and will halt the system before they can do any damage to the array if multiple bit errors are detected.
- The ZFS filesystem (which TrueNAS uses) requires 1GB of ECC RAM for every Terabyte of storage.
Reference
- Slideshow explaining VDev, zpool, ZIL and L2ARC and other easy mistakes!
- A Complete Guide to FreeNAS Hardware Design, Part I: Purpose and Best Practices (FreeNAS.org)
- FreeNAS 11.2 – How to Install (FreeNASTeam)
- FreeNAS Installation & Usage (Nerd on the Street – Tech)
- FreeNAS 9 Software Tutorial & Overview One (Tek Syndicate)
- Mirroring the FreeNAS USB Boot Device
- The ‘Hidden’ Cost of Using ZFS for Your Home NAS
Unraid
Unraid OS allows sophisticated media aficionados, gamers, and other intensive data-users to have ultimate control over their data, media, applications, and desktops, using just about any combination of hardware.
- Mix and match drives of different sizes, speeds, brands, and protocols.
- Secure access to your data in the event of theft or seizure with device-level encryption.
- Unraid Guides: The Ultimate Home NAS Solution
- unRAID… The Good, The Bad, & The Ugly
- FreeNAS vs. Unraid: GRUDGE MATCH!
SnapRAID is a backup program for disk arrays. It stores parity information of your data and it recovers from up to six disk failures.
Hard Drive Reliability
Long-term Archival Storage
Standard writable optical media, CDs & DVDs, etc., are not reliable for long term storage. The die used with optical media degrade with exposure to light, temperature extremes, moisture, etc. Because of this, you may not be able read parts or whole files/data from the media within several months to 5 years.
M-DISC (Millennial Disc) by Millenniata, now Verbatim, is the standard for digital archival storage. Unlike traditional optical media, which utilize dyes that can break down over time, data stored on an M DISC is engraved on a patented inorganic write layer – it will not fade or deteriorate. This unique engraving process renders these archival grade discs practically impervious to environmental exposure, including light, temperature and humidity.
- M-DISC optical media are readable in conventional optical drives.
- ISO/IEC 16963 standard longevity tests have proven the durability of M DISC technology, and it withstood rigorous testing by the US Department of Defense. Based on ISO/IEC 16963 testing, M DISC media has a projected lifetime of several hundred years.
Drives with M-DISC support and M-DISC Media
- Pioneer BDR-XD07S Portable 6X Blu-ray Burner External Drive Bundle with 100GB M-DISC
- LG Electronics 8X USB 2.0 GP65NG60
Cloud Storage
- ownCloud is a suite of client–server software for creating file hosting services and using them. ownCloud is functionally similar to the Dropbox, with the primary functional difference being the Server Edition of ownCloud is free and open-source. It also supports extensions that allow it to work like Google Drive, with online document editing, calendar and contact synchronization, and more.
Synchronizing Data
- Allway Sync uses algorithms to synchronize your data between desktop PCs, laptops, USB drives, remote FTP/SFTP and WebDAV servers, various online data storage, and more.
Drive Encryption
- Bitlocker is Microsoft’s encryption tool that’s included in Windows
- VeraCrypt is a free open source disk encryption software for Windows, Mac OSX and Linux.
- Windows Encryption Showdown: VeraCrypt vs Bitlocker
- PowerShell cmdlet: Get-BitLockerVolume