RAW DAS/NAS Storage

[damoncooper]

Now that the reality of the X5R storage requirements has started to sink in, direct attached storage or network attached storage expansion seems more critical than ever to nail down.

I'm looking at a few possible solutions for >60TB editable-speed storage including

- the Qnap 8-bay with Thunderbolt 2. Expandable to 400TB+ with additional 8-bay enclosures. A bit pricey though. Main unit is $5200 with 6TB X 8 disks. 8-bay expansion chassis is $700 which is reasonable.

- the LaCie 12Big Thunderbolt 3 to 72TB (announced at NAB 2016 but pricing not yet announced)

- DIY assembly of RAID 12-bay DAS/NAS chassis (about $1500) + 12 X 6TB drives (about $2200).

- Other?

Thoughts, current solutions, recommendations, WTF comments etc welcome here.

 

[Dr. Ifly Drones]

I can identify with your quandary. I'm currently swapping 2TB Passports and it's cumbersome. The challenge as I see it is that technology will improve at a significant speed and you want a system that will provide acceptable ROI over it's functional life span. For me that means as close to the cheapest solution available. A good example is the Inspire 1 I own; it's not a year old yet but, is functionally obsolete in comparison of the most recent DJI equivalents.

 

[Raphael Dahan]

Now that the reality of the X5R storage requirements has started to sink in, direct attached storage or network attached storage expansion seems more critical than ever to nail down.

I'm looking at a few possible solutions for >60TB editable-speed storage including

- the Qnap 8-bay with Thunderbolt 2. Expandable to 400TB+ with additional 8-bay enclosures. A bit pricey though. Main unit is $5200 with 6TB X 8 disks. 8-bay expansion chassis is $700 which is reasonable.

- the LaCie 12Big Thunderbolt 3 to 72TB (announced at NAB 2016 but pricing not yet announced)

- DIY assembly of RAID 12-bay DAS/NAS chassis (about $1500) + 12 X 6TB drives (about $2200).

- Other?

Thoughts, current solutions, recommendations, WTF comments etc welcome here.

So, if I understand, to trait corctely a rew file from our x5r SSD , I must be equipped with all those components?

 

[pixl45]

Been through all of this. Firstly, I recommend avoiding LaCie -- they are the brand in front of Seagate, and Seagate drives have a terrible reputation for quality/failure -- there are a number of lawsuits going right now due to big failures in the enterprise world.

For video editing and real time editing you'll want something with a Thunderbolt interface and super reliable drives. The drives you want for this are HGST -- the industry's most reliable. The G-Tech Brand uses all HGST drives (HGST owns them, and WD owns HGST) and offers a decent selection of RAID arrays.

If you instead decide to go to an NAS. both QNAP and Synology are very good. Synology is a little easier to understand for those without IT backgrounds. The WD Red drives are made for NAS Raids, and are top rated for this purpose. I'm currently using a Synology DS415+ (populated with 4 x 4tb WD Reds )over gigabit ethernet, and it has been plenty speedy for me, BUT I predominantly use it for 100mb plus still photos. I'm not sure that any NAS will give you great throughput for 4k video.

The advantage of NAS is that it's really easy to expand, you can set it up to be a file server for multiple machines, and it can even be a remote file/web server.

I think what you will probaly end up doing is both things. A fast thunderbolt RAID, and an NAS. Ka-ching! For the volumes you're after.... $10k at least.

PS.... whatever you do, DONT buy Drobo.

 

[GizaDog]

Here is my simple power setup. All drives connected by ThunderBolt.

 

[pixl45]

Here is my simple power setup. All drives connected by ThunderBolt.

Nice set-up Giza. Do you find 64gb of RAM enough?

 

[gruvpix]

Yea I'd go NAS all day. Lacie stuff is garbage, had three (yes 3) of their externals fail on us, resulting in a $1000 recovery fee each time. I hate Lacie and everything they stand for!
G-drive is what we use now, no problems with those, but a NAS system would be the cat's ***.
What I would do is just forget about the original raw files and transcode everything to ProRes HQ. That way you still have a lot of flexibility to colour grade and your file sizes are much much smaller. Unless you are a stickler for having the originals, in which case prepare for your arm and leg amputation.

 

[GizaDog]

Nice set-up Giza. Do you find 64gb of RAM enough?

Its great. Never had a problem or OS crashing of any kind.

 

[AllanM]

Whatever you get, you've then got to decide how you back <that> up. Raid 1 may mirror, so a drive can fail and OK, but it won't help any sort of file or partition corruption which could be catastrophic.
For photography, I work off raid 1 on a linux box over gigabyte ethernet - that's fine, You'd not notice the difference between NAS & local.
But for video I copy the stuff from the raid to a local 1T SSD and copy back when session over.

Nightly the NAS is incrementally backed up to another machine which is in the garage.
I use WD blacks for Raid 1.

 

[pixl45]

Whatever you get, you've then got to decide how you back <that> up. Raid 1 may mirror, so a drive can fail and OK, but it won't help any sort of file or partition corruption which could be catastrophic.
For photography, I work off raid 1 on a linux box over gigabyte ethernet - that's fine, You'd not notice the difference between NAS & local.
But for video I copy the stuff from the raid to a local 1T SSD and copy back when session over.

Nightly the NAS is incrementally backed up to another machine which is in the garage.
I use WD blacks for Raid 1.

I believe you mean gigaBIT ethernet. Wouldn't it be great if we could get ethernet speeds of 1 gigabyte/sec??!!

 

[AllanM]

^^ Yes, the above was a late night typo. There is 10Gb ethernet around but just a little on the dear side at the mo. (X520-DA2). This would work fine unswitched, client to server.

 

[The Editor]

I don't understand.
Bill Gates said that 48k of memory is more than anyone will ever need!

^^ Yes, the above was a late night typo. There is 10Gb ethernet around but just a little on the dear side at the mo. (X520-DA2). This would work fine unswitched, client to server.

And the cables are not cheap either!

 

[Richard Hurst]

My X5R is on its way and this stuff goes right over my head. I'm just a hobbyist who will create something and possibly store the original footage as well as the premiere project and final .MP4 file. I'm not out filming every week so would I be fine with just getting something like the G-Technology G Drive Pro or G-raid Studio 6TB??

 

[pixl45]

My X5R is on its way and this stuff goes right over my head. I'm just a hobbyist who will create something and possibly store the original footage as well as the premiere project and final .MP4 file. I'm not out filming every week so would I be fine with just getting something like the G-Technology G Drive Pro or G-raid Studio 6TB??

If you're going to be shooting and editing 4k video, you'll be using up that 6TB pretty quickly. Usually when manufacturers say a RAID drive is 6TB, it is at most a 3TB mirrored 2-bay. But beyond the storage requirements, you're going to need likely minimum 32gb of RAM, a very fast video card, a fast processor, and a high res (4k or even 5k) monitor. You'll need a computer with a fast external interface -- Thunderbolt 2 or USB 3 on Mac, USB 3 on PC. Yes, Thunderbolt 2 is much faster than USB 3, but both interfaces are faster than the throughput of the drives you'll be using. The other advantages of Thunderbolt are that you can daisy chain multiple devices, and it supports two-way data simultaneously -- which is a big help especially in high demand situations like line editing,

Drives need to be really good 7200 rpm HDD drives. As stated above, HGST are the reliability kings with prices to match. For pure performance, WD Blacks are thought to be the best, and WD Reds are the #1 choice for storage like NAS. If you feel like spending some money for some speed, get an SSD (biggest you can find) for your live working files.

If you wanna keep everything as cheap as possible. you can go with a PC box -- build your own -- so that you have an upgrade path. Buy at least a 4 bay RAID -- you don't have to fill all the bays initially. Just don't cheap out on your RAM or video card -- without the right ones your projects will go nowhere

 

[Richard Hurst]

If you're going to be shooting and editing 4k video, you'll be using up that 6TB pretty quickly. Usually when manufacturers say a RAID drive is 6TB, it is at most a 3TB mirrored 2-bay. But beyond the storage requirements, you're going to need likely minimum 32gb of RAM, a very fast video card, a fast processor, and a high res (4k or even 5k) monitor. You'll need a computer with a fast external interface -- Thunderbolt 2 or USB 3 on Mac, USB 3 on PC. Yes, Thunderbolt 2 is much faster than USB 3, but both interfaces are faster than the throughput of the drives you'll be using. The other advantages of Thunderbolt are that you can daisy chain multiple devices, and it supports two-way data simultaneously -- which is a big help especially in high demand situations like line editing,

Drives need to be really good 7200 rpm HDD drives. As stated above, HGST are the reliability kings with prices to match. For pure performance, WD Blacks are thought to be the best, and WD Reds are the #1 choice for storage like NAS. If you feel like spending some money for some speed, get an SSD (biggest you can find) for your live working files.

If you wanna keep everything as cheap as possible. you can go with a PC box -- build your own -- so that you have an upgrade path. Buy at least a 4 bay RAID -- you don't have to fill all the bays initially. Just don't cheap out on your RAM or video card -- without the right ones your projects will go nowhere

Thanks. I'm using the 5K 27" iMac with 32GB Ram and the upgraded 4Gb graphics card currently. For normal photo/files/programmes backup I currently use 2 G-Tech Mini 2TB drives, one on a time machine backup and the other running a super duper back up. I've currently got a budget around £350-£500 for storage options

 

[pixl45]

Thanks. I'm using the 5K 27" iMac with 32GB Ram and the upgraded 4Gb graphics card currently. For normal photo/files/programmes backup I currently use 2 G-Tech Mini 2TB drives, one on a time machine backup and the other running a super duper back up. I've currently got a budget around £350-£500 for storage options

Great machine.... just get a min. 4-bay RAID... and expand as needed. As others have said, backing up your RAID is a very good idea. The back-up doesn't have to be fast. but should be local. NAS is ideal, and there are plenty of smaller ones around. WD Blacks are made in up to 6TB sizes, though locally I can only source max 4TB @ $250 CDN each. If you're looking to buy fully packaged, this one fits your budget and is a quality device -- but little headroom for expansion. As I said before, it's really just 3 TB, and still doesn't give you a proper back-up.

None of it's cheap.... but it's a lot cheaper than it used to be -- my old design studio used a SCSI 8 GB DAT system for the entire office that cost $3000. These days you can fit that much on a $10 thumb drive.

 

[Richard Hurst]

Great machine.... just get a min. 4-bay RAID... and expand as needed. As others have said, backing up your RAID is a very good idea. The back-up doesn't have to be fast. but should be local. NAS is ideal, and there are plenty of smaller ones around. WD Blacks are made in up to 6TB sizes, though locally I can only source max 4TB @ $250 CDN each. If you're looking to buy fully packaged, this one fits your budget and is a quality device -- but little headroom for expansion. As I said before, it's really just 3 TB, and still doesn't give you a proper back-up.

None of it's cheap.... but it's a lot cheaper than it used to be -- my old design studio used a SCSI 8 GB DAT system for the entire office that cost $3000. These days you can fit that much on a $10 thumb drive.

hi. So with the 6TB Studio I'd use both disks for storage and editing from and then use a NAS to back it up?

 

[pixl45]

Depends which kind of RAID you use. Many will go RAID 1, and that way always have a back-up of your work on the second disk. But as @AllanM pointed out above, that second drive is a mirror of the first, so it will also mirror partition errors that are not complete drive failures. It's better than nothing, but not a really failsafe system. A full back-up (can be NAS, or any drive set-up that is bigger than all of your drives accumulated capacity) is best. You'll sleep better.

The thing to remember is that all drives fail, eventually. IT guys will often set a schedule to replace drives whether they are working or not. I personally consider any drive older than 2 years old to be a ticking time-bomb.

 

[Richard Hurst]

Depends which kind of RAID you use. Many will go RAID 1, and that way always have a back-up of your work on the second disk. But as @AllanM pointed out above, that second drive is a mirror of the first, so it will also mirror partition errors that are not complete drive failures. It's better than nothing, but not a really failsafe system. A full back-up (can be NAS, or any drive set-up that is bigger than all of your drives accumulated capacity) is best. You'll sleep better.

The thing to remember is that all drives fail, eventually. IT guys will often set a schedule to replace drives whether they are working or not. I personally consider any drive older than 2 years old to be a ticking time-bomb.

I'm not sure of the RAID numbering system but couldn't I just have the full 6TB on the studio as storage and not have one 3TB drive mirroring the other?, then get a separate drive to back that all up?

On a separate note isn't their an issue with El capitan and setting up RAID anyway? (more issues for me)

 

[Raphael Dahan]

Great machine.... just get a min. 4-bay RAID... and expand as needed. As others have said, backing up your RAID is a very good idea. The back-up doesn't have to be fast. but should be local. NAS is ideal, and there are plenty of smaller ones around. WD Blacks are made in up to 6TB sizes, though locally I can only source max 4TB @ $250 CDN each. If you're looking to buy fully packaged, this one fits your budget and is a quality device -- but little headroom for expansion. As I said before, it's really just 3 TB, and still doesn't give you a proper back-up.

None of it's cheap.... but it's a lot cheaper than it used to be -- my old design studio used a SCSI 8 GB DAT system for the entire office that cost $3000. These days you can fit that much on a $10 thumb drive.

Thanks for your pro information,
But I'm not a pro on editing, but since I purchased the X5R I felt that I must upgrade my informatics system and I am waiting for my new Mac Pro with this configuration:
12 cores 2,7 GHz with 30 Moin cache N3
64Go(4x16Go) of memory ECC DDR3.
1T of stocking flash PCLe.
Two graphic processors AMD FirePro D700 with 6Go of VRAM GDDR5 each.
So what else do I need for editing, not a full movies of 2 hours but some nice clips of max 15 minutes.
Once I edited them I can stock them were ever on a chip hard disk , way sould I spend more K € just for stocking my finished projects?
Thanks in advance.