Tag Archives: unraid

HEVC media optimization

Posted on by — No Comments ↓

Do you have a large collection of video media files not using HEVC (H265) yet? There is a massive amount of disk space coming your way if you flick over to the new video codec format.

HEVC definitely lives up to its name, for most media you can expect a 70% or more disk savings from transcoding from an old codec. There are some catches though… If you want your TV to play it direct (i.e. straight off the file) the codec will need to be supported by it. You can of course get around this by using a media server such as Plex or Emby which will transcode from HEVC back to a compatible format.

Why would you transcode to HEVC? – again, disk space. HEVC as stated above can reduced you Media footprint significantly. You could boost your quality and save your disk space at the same time by recording at a higher resolution then applying the HEVC codec.

I created a powershell script to transcode my media to HEVC using my AMD graphics card. The advantage of doing this is that transcoding completed by my GPU is significantly faster than my CPU. I do not have the graphics card in my media server, so instead connect via SMB and let my gaming machine run the transcoding from remote…

The powershell script uses ffmpeg to ;

  • transcodes video stream to hevc using AMD h/w encoder
  • copys all existing audio and subtitles (i.e. no conversion)
  • works in batches (to prevent constant scanning of files) – able to set max batch size and processing time before re-scanning disk
  • overwrites source with new HEVC transcode if move_file = 1 (WARNING this is default!)
  • checks to see if video codec is already HEVC (if so, skips)
  • writes transcode.log for successful transcode (duration and space savings)
  • writes skip.log for already hevc and failed transcodes (used to skip in next loop, errors in transcode.log)

Check here for updates and script – https://github.com/dwtaylornz/hevctranscode

Unifi Video Controller NVR for UNRAID

Posted on by — No Comments ↓

If you run UNRAID at home and you have a UniFi Camera system then check out my latest container….

https://hub.docker.com/r/superd/unifi-nvr/

A dockerised UniFi NVR

Ubiquiti UniFi-Video-Controller (NVR) — Docker Container

Ubuntu 16.04, UniFi-Video-Controller 3.8.3

Setup / Quick Start

docker run \
--net=host \
-v /var/lib/unifi-video/:<YOUR DATA DIR> \
-v /var/log/unifi-video/:<YOUR LOG DIR> \
superd/unifi-nvr

Troubleshooting

UNRAID – If you have issue with MongoDB continually restarting please check your data mapping. I have seen issues where user shares do not work correctly. Please try mapping direct to a single disk or to cache drive to ensure smooth operation.

i.e. instead of /mnt/user/usershare/nvr/data  use  /mnt/disk1/usershare/nvr/data

Update 4/02/2020 ;

I would recommend using this docker image – https://hub.docker.com/r/pducharme/unifi-video-controller/ 

I have found that i no longer require a direct disk mapping as stated above, the built in DB seems to work fine on user shares. Unraid 6.8.2 

New Commandline as per follows; 

docker create --name='unifi-video-controller' 
--net='host' 
-e TZ="Pacific/Auckland" 
-e HOST_OS="Unraid" 
-e 'PUID'='99' 
-e 'PGID'='100' 
-v '/mnt/user/appdata-unifi/unifi-video/':'/var/lib/unifi-video':'rw' 
-v '/mnt/user/appdata-unifi/unifi-video/videos/':'/usr/lib/unifi-video/data/videos':'rw' 
'pducharme/unifi-video-controller'

I use ‘host’ network mappings just as there are a ton of ports and i’m lazy – use bridge ports if you want to reduce footprint. 

Docker – Running Ubiquiti NVR and Plex

Posted on by — No Comments ↓

downloadBye bye virtual machines and their inherent OS bloat. Docker and containerization is here…

The trick to containerization is picking the right workload (as with most things). Think about data, its state and where it lives and whether there are any benefits to running as a container.

Both Ubiquiti’s NVR and Plex’s media server software run’s some base application, this app within its own container then maps to data (which can exist outside the instance) that is consistent.

The fun continues when you can update a container (updating the running application), but keeping the data intact at another location. This can really help with version control etc where you can sometimes just point the new container at the data and turn off the old instance. Rollback? easy. Turn off new container and roll back to old.

Of course things are easier if you are running applications that do not change the data.  Both NVR and Plex only index and capture new data (in consistant format), which makes moving between application versions much easier.

The nature of containerization means that the full power of the host is taken into regard. This is different to regular visualization where each guest is limited to the virtual hardware it is assigned. There are of course challenges where resource is congested, but this can also happen in the latter (cpu scheduling, under / over allocation of resources).

Availability also has to be built with containers in mind, with load balances and instances across multiple hosts.

Update : this site has now now been migrated from a VM to 2 x docker containers…. One for MySQL Backend and one for WordPress FrontEnd. Containers can be linked – so the WordPress container can access MySQL container via its own local port. Very cool.

unraid smart check – dead WD green drive

Posted on by — No Comments ↓

errors on unraid GUI – sometimes its a loose cable, sometimes its an issue with the drive.

Run this command to check smart status

smartctl -a -d ata /dev/sda
or if you are using a newer SATA controller
smartctl -a -A /dev/sda

http://lime-technology.com/wiki/index.php/Troubleshooting

unfortunately in my case, looks like drive is pretty much dead… not too bad for a drive almost 5 years old.

its pretty typical of a WD green drive in its default config to die in this type of environment, no plans to replace it with a similar type drive. You can see below the incredibly high LCC count which indicates the drive header has parked this many times over its life. This is probably part of the problem – there is a tool you can run (check this vid, link for  WDIDDLE3 also in comments – http://www.youtube.com/watch?v=J2eYyRI_F98) which disables the intellipark feature of the green drive. I never disabled the park timeout before this drive died (which defaults to 8 seconds!) — note: i have disabled it completely on my other green drives.

=== START OF INFORMATION SECTION ===

Model Family: Western Digital Caviar Green
Device Model: WDC WD10EADS-00M2B0
Serial Number: WD-WCAV51020991
LU WWN Device Id: 5 0014ee 2588170a5
Firmware Version: 01.00A01
User Capacity: 1,000,204,886,016 bytes [1.00 TB]
Sector Size: 512 bytes logical/physical
Device is: In smartctl database [for details use: -P show]
ATA Version is: ATA8-ACS (minor revision not indicated)
SATA Version is: SATA 2.6, 3.0 Gb/s
Local Time is: Thu Oct 30 18:48:41 2014 NZDT
SMART support is: Available – device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: FAILED!
Drive failure expected in less than 24 hours. SAVE ALL DATA.
See vendor-specific Attribute list for failed Attributes.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 168 154 051 Pre-fail Always – 12560032
3 Spin_Up_Time 0x0027 149 105 021 Pre-fail Always – 5508
4 Start_Stop_Count 0x0032 099 099 000 Old_age Always – 1253
5 Reallocated_Sector_Ct 0x0033 119 119 140 Pre-fail Always FAILING_NOW 648
7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always – 0
9 Power_On_Hours 0x0032 041 041 000 Old_age Always – 43079
10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always – 0
11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always – 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always – 371
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always – 363
193 Load_Cycle_Count 0x0032 001 001 000 Old_age Always – 1932037
194 Temperature_Celsius 0x0022 118 076 000 Old_age Always – 29
196 Reallocated_Event_Count 0x0032 001 001 000 Old_age Always – 463
197 Current_Pending_Sector 0x0032 199 193 000 Old_age Always – 323
198 Offline_Uncorrectable 0x0030 199 190 000 Old_age Offline – 186
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always – 0
200 Multi_Zone_Error_Rate 0x0008 003 001 000 Old_age Offline – 39455

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Short offline Completed: read failure 90% 24914 789707146

Here is a good post on another forum about the issue (which also seems to hit some of the new RED drives);

https://forums.freenas.org/index.php?threads/hacking-wd-greens-and-reds-with-wdidle3-exe.18171/

I have disabled intellipark on the rest of my green drives (since they are close to 5 years and probably near failure). I have some new RED drives which i have increased the time out to 300 seconds. (most come with 300 sec timeout, but older firmware is at 8 seconds). From what I’ve been reading there is no physical difference between WD red and green drives, only the firmware differs. So if you are going to put some green drives into a NAS / RAID or Server environment ensure you run wdidle3 and either disable or change timeout on intellipark to 300 seconds. (then its pretty close to a red drive)

To check current status

wdidle3 /r

to disable intellipark

wdidle3 /d

to set to 300 (max) timeout

wdidle3 /s300

Why is UNRAID cool for home?

Posted on by — No Comments ↓

I’ve played with many storage technologies at home, ZFS being one of my favs when it comes to performance. But i’ve been looking for something that suits a typical home environment where power usage and capacity is usually more important than performance. Thats where UNRAID has come in…

UNRAID give me these advantages;

  • Different sized disks in a single pool (only requires largest disk as parity)
  • Files are distributed over all the disks – so even if you lost more than a single drive you still still have some of your data. Note : with parity drive you can handle a single drive failing without loosing anything, sorta like RAID4 – non distributed parity.
  • Power usage, since the files are stored on specific disks not all the disks need to power on to give you your file.
  • Runs on a USB stick – no large operating system required.
  • Crashplan module can be installed to provide backup options.

http://lime-technology.com/