How I Organize My Media library

About building a full chain Private Digital Media Entertainment Platform.
Reading this post require basic computer knowledge.

Table of Contents

0. Choosing Hardware

0. Choosing Hardware

0.1 Main Server

A media library is essentially a storage pool, but it also takes on performance-dependent functions such as media decoding and playback. So we needed to balance storage and computing needs. I bought two servers from a friend that were obsolete from the server room: DELL R720. The specific configuration is as follows:

⚠️An outdated server is a cheap option but it will consume a huge amount of power and overall, it does not save money, so please do not use my configuration list directly.

*Only important parts
Server 1️⃣
Case: DELL R72
CPU：Intel(R) Xeon(R) CPU E5-2690 v2 X2
Mem：DDR3 64G
GPU：NVIDIA GeForce RTX2060
RAID Card：PERC H710 mini
Networks：Build-in 1Gbps X 4
Networks：Mellanox ConnectX-3 MCX311A 10Gbps SFP X1

Server 2️⃣
Case: DELL R72
CPU：Intel(R) Xeon(R) CPU E5-2690 v2 X2
Mem：DDR3 64G
GPU：X
RAID Card：PERC H710 mini
Networks：Mellanox ConnectX-3 MCX311A 10Gbps SFP X4

The factors you need to consider are as follows：

CPU
- Will undertake the main system computing tasks, with no exceptionally high requirements. The ability to perform virtualization work to run the system is sufficient.
Memory
- The more the merrier. Cheap anyway. LOL
GPU
- It’s imperative to have the latest graphics card! A newer graphics card means more video decoding power.
RAID Card
- The more expensive, the better.
Networks
- Depends on your home network limit, but the faster the port, the better.

0.2 Networking

0.2.1 LAN

Most of the time, I use media libraries on my LAN. So for me, LAN speed is very important. Most of my media library is 2K resolution movies, very few of them are 4K. so 1Gbps on LAN is enough.

The network hardware and network topology diagram are as follows：

Topology diagram

All links work at 1G except between servers which are 10gbps. The 10gbps link between the servers is used to ensure that the server running the media library software can take out the content on the file server at the maximum rate.

All cables comply with ISO/IEC 11801 CAT7_A standard.

0.2.2 WAN

Due to the shortage of ipv4 and the complex network environment in China, the contents of this section will not be universally applicable, so please adjust the operation steps according to your situation.

IPv4 Access

~~According to the policy of Guangxi Telecom（The highest market share ISP）, you only need to provide suitable reasons for using IPV4 to active it.~~ （2022.02.08, Guangxi Telecom stop providing IPv4： Announcement）

Bridging is banned by ISP.

Bridging [broken]

GPON Client： HS8145V
GPON Client Software Version：V3.17.C10S118
————
Router ：ikuai
software version ：3.6.3 x64 Build202204071133

PPPoE log is as follows :

Already hide some data.
timestamp	interface   content
1627081878	ads00000000 Connection terminated.
1627081878	ads00000000 Sent PADT
1627081883	ads00000000 PPP session is 50778
1627081883	ads00000000 Connected to 00:00:00:00:00:00 via interface wan1_1
1627081883	ads00000000 Using interface ads0000000
1627081883	ads00000000 Connect: ads00000000 <--> wan1_1
1627081883	ads00000000 syncppp not active
1627081884	ads00000000 Remote message: Authentication failed!
1627081884	ads00000000 PAP authentication failed
...
1627081920	ads00000000 Timeout waiting for PADO packets
1627081920	ads00000000 Unable to complete PPPoE Discovery

ISPs may monitor PPPoE to avoid illegal uses. Only direct router dial-up can be used.

0.2.3 VLAN

Building a secure encrypted VLAN can provide an alternate way to access the server. The media server is for family and friends only, so using Tailscale is sufficient for the needs.

It is recommended to use your most convenient SSO login method to avoid trouble when logging in on other clients. （For example, I can’t use Google login when I log in to Tailscale on the Chinese server🤣）

Of course, VLANs are not required. I use VLANs only to prevent ISPs from blocking IPs after detecting abnormal traffic.

0.3 Storage

Storage is the most important part of this hardware set and must ensure that data is not corrupted even if other hardware fails.

Ensuring data integrity requires a combination of hardware and software, and this section only describes how the hardware is designed.

0.3.1 Selecting Storage Media

HDD

HDD’s storage technology is mostly: PMR SMR MAMR HAMR ….。Dizzy?😵‍💫

Just remember not to buy SMR and take HAMR MAMR with a grain of salt. SMR technology is a technology proposed by disk manufacturers to reduce costs and increase unit capacity, the technology is not friendly to long-term storage of data may lead to data loss as well as SMR disk performance is extremely low, and usually requires a large capacity cache.

So how do we avoid buying disks that use SMR technology? The answer is to just buy enterprise-grade disks. （For example, Western Digital’s HC550）

Based on my experience and Backblaz’s disk failure rate report, I recommend using Western Digital’s HDD（Also known as HGST）.

“The Force is strong with this one.”³ The 6TB Seagate (model: ST6000DX000) continues to defy time with zero failures during Q1 2022 despite an average age of nearly seven years (83.7 months). 98% of the drives (859) were installed within the same two-week period back in Q1 2015. The youngest 6TB drive in the entire cohort is a little over four years old. The 4TB Toshiba (model: MD04ABA400V) also had zero failures during Q1 2022 and the average age (82.3 months) is nearly as old as the Seagate drives, but the Toshiba cohort has only 97 drives. Still, they’ve averaged just one drive failure per year over their Backblaze lifetime.
“Great, kid, don’t get cocky.”⁴ There were a number of padawan drives (in average age) that also had zero drive failures in Q1 2022. The two 16TB WDC drives (models: WUH721816ALEL0 and WUH721816ALEL4) lead the youth movement with an average age of 5.9 and 1.5 months respectively. Between the two models, there are 3,899 operational drives and only one failure since they were installed six months ago. A good start, but surely not Jedi territory yet.
“I find your lack of faith disturbing.”⁵ You might have noticed the AFR for Q1 2022 of 24.31% for the 8TB HGST drives (model: HUH728080ALE604). The drives are young with an average age of two months, and there are only 76 drives with a total of 4,504 drive days. If you find the AFR bothersome, I do in fact find your lack of faith disturbing, given the history of stellar performance in the other HGST drives we employ. Let’s see where we are in a couple of quarters.
“Try not. Do or do not. There is no try.”⁶ The saga continues for the 14TB Seagate drives (model: ST14000NM0138). When we last saw this drive, the Seagate/Dell/Backblaze alliance continued to work diligently to understand why the failure rate was stubbornly high. Unusual it is for this model, and the team has employed multiple firmware tweaks over the past several months with varying degrees of success. Patience.

https://www.backblaze.com/blog/backblaze-drive-stats-for-2021/

SSD & Cache

— TBC