NAS Boxes 20240320

UPS option: https://www.pbtech.co.nz/product/UPSPWR16130/Eaton-5E-Gen-2-Tower-UPS-1200VA–660W-3-ANZ-Outlet

20240322

Before: We are spread across multiple NAS boxes

  • DS218j (2 x 2TB < 1 yr old)
    • SHARE: Companies
    • SHARE: Sara
    • SHARE: PNP_Firmware
    • SHARE: Other
  • QNAP TS-431P3 (2 x 18TB, < 3 years old)
    • SHARE: MainShare
      • Movies: 11,029 | 6.91TB
      • Motorsport: 2,640 | 6.06TB
      • Samsung Frame Images: 155,854 | 140GB
      • Total: 199,135 | 13.3TB
  • DS216se (2 x Seagate Ironwolf 12TB 2018)
    • SHARE: W
    • 144,186 | 3.21TB
    • stuff that really should’ve been cleaned up and migrated previously to QNAP TS-431P3
  • DS212j (2 x 1TB unknown age)
    • SHARE: DumpStuffHere (NSFW) 71,363 | 440GB
  • DS207+ (2 x Seagate Archive 8TB 2015?)
    • Parked contents as when last retired this unit, expect all contents were migrated to DS216se at that time
  • QNAP TS-230
    • Not in use at this time

Amazon.au ordered 2 x Toshiba Enterprise HDD 20TB, MG10ACA20TE AU$1250 incl GST for the pair

  • Toshiba probably wont support 5yr warranty for non enterprise customers? so cancelled that order bugger

ServerPartsDeals.com: ordered 2 x Seagate Exos X20 ST18000NM003D 18TB 7.2K RPM SATA 6Gb/s 3.5in Recertified Hard Drive @ US$398.00 / NZ$674.22 landed for the pair. Wouldve preferred Iron Wolf pro but they were out of stock at the time (and $US10 cheaper!)

SeaToolsSeagate Exos X20 ST18000NM003D
ZVTB4YLV
Seagate Exos X20 ST18000NM003D
ZVT04GRD
Quick TestPASSPASS
Long Self TestPASS (25 hours)PASS (25 hours)

Pre cleanup= DONE

  • DS218j: no actions needed
  • QNAP TS431P3: Delete anything obsolete
  • DS216se: Delete anything obsolete
  • DS212j: Delete anything obsolete
  • Tidy up where quick (Delete / relocate)

Prep

Backup existing QNAP TS-431P3 = DONE

Reminder re W drive

Went DS207 —> DS216SE –> TS431P3

  1. Cleanup all on DS212j move remainder to DS216se DONE
  2. Move 2 x 8TB from DS207+ to DS212j as new install, mirrored DONE
  3. Backup all QNAP TS-431P3 DONE
  4. Remove 2 x 1TB MX500 SSDs from TS-431P3 DONE

Temporary backups:

WhatBacked up to where
DS212j Fashion452GB\Ds216se\w\__20240327 NAS Backups\DS212j 2TBDONE
TS431P3
\NAS2021\MainShare\Motorsport
6.1TB\Ds216se\w__20240327 NAS Backups\NAS2021\MotorsportDONE
TS431P3
\NAS2021\MainShare\Movies TV
7TB\DS212J\DS212J8TBDONE
TS431P3
\NAS2021\MainShare\Samsung The Frame Images
140GB\DS218J\DS218j OtherDONE
TS431P3
\NAS2021\MainShare everything else
215GB\DS218J\DS218j Other\DONE
Old W drive from last migration
DS207+
5TB Seagate USB ExternalDONE

Setup QNAP TS431-P3

QNAP TS-431P3 RAID 6 or RAID 10?

  • RAID 6:
    • Pro: Can lose any 2 drives
    • Con: Array build time much slower & write times slower = ok as is mostly used for static storage
  1. Power down, Remove 2 x SSDs and install 2 x EXOS 18TB then power on again = DONE 20240410
    • Control Panel > System > Backup/Restore > Restore to Factory Default.
    • Click Reinitialize NAS and follow setup procedure
  2. Storage Pool: 1 pool, all 4 drives Raid 6 = 32.72TB
    • Wait for Resync: Started 20240410 10am ended before 20240412 05am = over 1.5 days
    • Set RAID Resync Priority = Resync First
    • Enable BITMAP in Storage & Snapshots > Storage > Storage/Snapshots = DONE 20240412
  3. Set up Volumes/Shares
  4. Restore from Temporary backups
  5. NOTE TO SELF: Remember to setup Scrubbing schedule

Volumes / Shares

Static vs Thin vs Thick volumes?

  • I don’t know much about linux file systems so I’m gonna fall back on (very) old school patterns
    • Static: Previously I’ve just allocated 1 giant volume using the entire available storage
      • CON: File system checks are across all the NAS contents
    • THIN:
      • I do not need to snapshot these volumes
      • CON?: expands as needed but to me that sounds like a recipe for getting multiple volumes physically interleaved across the disk
    • THICK:
      • Pre-allocated and can be expanded and reduced (reduction might take some time?)
      • For large files (Movies & Motorsport) pre-allocation should result in sequential writes in those pools especially applicable when doing initial population from backups

Order of Volume creation, does it matter?

  • Assuming allocation of Thick volumes is sequential from outside (faster) to inner (slower) of platter
    • This is making some assumptions as to how the underlying RAID 6 array is utilised but if there is no actual benefit no harm no foul, so will proceed assuming the assumption is valid
      • AI (GPT / Stable Diffusion) vols first as will likely be the most active volumes create those first
      • Then General Use vols
      • Then vols for more static content with large files (and usually sequential write/read )(MoviesTV/Motorsport) volumes later
      • Last vol is a ~3TB volume that can be reduced/removed later if we need to expand either static content vols

First Vol used by QTS:

The system volume is a regular static or thick volume that QTS uses to store system data such as logs, metadata, and thumbnails. By default, applications are installed to the system volume. If no system volume exists, either because the NAS has recently been initialized or the system volume was deleted, QTS will assign the next static or thick volume that you create as the system volume.

Capacity:

  • Formatted capacity = 32.72TB
  • QNAP QTS has consumed for I do not know what about 210.56GB before we create first volume

Bytes Per Inode

  • I allocated a Vol with 5TB / 64 Bytes per INode and it resulted in an indicated 4.98TB capacity
  • I allocated a Vol with 5.02TB and 32 Bytes pr INode and it resulted in an indicated 4.98TB capacity
  • I allocated a Vol with 5.04TB and 32 Bytes per INode and it resulted in an indicated 5TB capacity

From this I roughly extrapolate:

  • 64K Bytes Per INode overhead = 0.4% of allocated space
  • 32k Bytes Per INode overhead = 0.8% of allocated space
  • 16k Bytes Per INode, overhead = 1.6% of allocated space

I like round numbers, so willl over allocate my TB per volume to achieve available per vol capacity closer to my target

So allocated AI_Models as 5.04TB, Volume shows as 5TB available but Shared folder creation shows as 4.99 available, so will drop / create vol again with 5.05

VolumeUsed
(202403)
AllocBytes Per INodeMapped
SystemVolume280GB8k (for thumbnails)
Running remainder
(of 32TB)
AI_Models2TB5.05 (5TB avail)32kU
AI_Other<1TB2.05TB (2.01TB avail)16kV
GeneralUse<2TB3.08TB (3.01TB avail)16kP
Movies_TV7TB10.05TB (10.0TB avail)64kQ
Motorsport6.1TB9.05TB (9TB avail)64kR
Temporary3TB16kT

Restore from Temporary backups:

Backup fromTo
Fashion
\Ds216se\w\__20240327 NAS Backups\DS212j 2TB
452GB
Motorsport
\Ds216se\w__20240327 NAS Backups\NAS2021\Motorsport
6.1TBMotorSportStart 20240417 0745
Speed: 1TB/9hrs ~40MB/s
Movies TV
\DS212J\DS212J8TB
7TBMoviesTVStart 20240412 1600
End 20240417 0500
Speed: 1TB/16hrs ~20MB/s
Samsung The Frame Images
\DS218J\DS218j Other
140GB
Orig TS431-P3 leftovers
\DS218J\DS218j Other
215GB
Old W drive from last migration
5TB External USB

Unit/Drive allocation after:

WhichDiscsRAIDWhat
DS218j2 x Seagate IronWolf 2TB (2023)Mirrored
2TB
Companies etc
Anything for Cloud backup
TS-431P32 x Seagate Iron Wolf Pro 18TB (202105-06)
2 x ServerPartsDeals Recertified Seagate Exos X20 ST18000NM003D 18TB (202403)
RAID 6
36TB
Everything else
TS-230RetireSELL
DS216seRetire
2 x Seagate Ironwolf 12TB 2018
Mirrored
12TB
Motorsport off site backup?
DS212jRetire
2 x Seagate Archive 8TB 2015
Mirrored
8TB
SELL
DS207+Retire
2 x WD 1TB
SELL
DS107RetireSELL

20240321

Intent: Anything to be Cloud backed goes to DS218J

  • Moved DS216SE SHARE Companies to new SHARE DS218j_Companies
  • Moved anything Sara specific found anywhere to new SHARE DS218j_Sara
  • Moved anything PNP firmware etc found anywhere to new SHARE DS218j_PNP_Firmware
    • Looks a little light so i better find where it al lis
  • Also some deletes of stuff that can go as found anywhere
  • Moved random Motorsport as found to TS-431P3 MainShare\Motorsport

As at 20240320

QNAP TS-431P3 (bought 2021-08-05)

  • 2 x MX500 1TB Drive slot 1 + 2 (probably bought after 2021-08-05)
    • Not used currently, idea is to use with a Docker web server for Samsung The Frames coordinated displays (browser on Frames to this webserver)
  • 2 x Seagate Iron Wolf Pro 18TB
    • 202105 purchase to drive slot 3, 202107 purchase to drive slot 4
    • 5 yr warranty
    • ST18000NE00
  • 8GB Ram: 2021-08-16 Pulled original 2GB DDR3-1600 SODimm and put in an 8GB stick removed from the Dell XPS 2710
  • SHARE MainShare

QNAP TS-230 202105 ~NZ$300 incl

  • Not in use as at 20240320

Synology DS218J (PBTech 2018 for Weekes, came to me 202402)

  • Not in use as at 20240320
  • 2 x 2TB IronWolf (202210 PBTech)

Synology DS216SE

  • 192.168.2.2
  • 2 x Seagate Ironwolf 12TB ST12000VN0007-2GS116 10.9TB (2015)
  • SHARE: Companies
  • SHARE: W

Synology DS212J (Ex Weekes)

  • Current store for Fashion shows also some Motorsport
  • 2 x WDC WD10EARX-00N0YB0 931.5 GB HDD

Synology DS207

  • 192.168.2.4
  • Not in use, files still as when last was main?
  • CIFS 1.0 only
  • 2 x Seagate Archive HDD V2 ST8000AS0002-1NA 8TB
  • Is playing up so time to retire

Synology DS107

  • Not in use
  • CIFS 1.0 only
  • No drive