DIGITAL DATA STORAGE: what is your long term solution ??

[CVT]

Multiple 1-1.5TB external drives....

10K:dance1:

[digitalron]

The best approach would be a combination of both optical and drive-based. My personal approach is to create two optical-based copies, one serves as the master archive, the other is a working archive. If I know I will be working on the files regularly, I also store a copy on an external drive and use those.

The master archive should preferably be a high-quality DVD copy, written at intermediate speed. It would be a wise idea to store this in an airtight container with silica gel desiccant to avoid moisture damage. It would also be a good idea to check on this copy regularly, at least one every two or three years. This disk should be used solely to create working copies of the file, and should be used as rarely as possible. This master archive copy can also be replaced by a new master if the disk is showing signs of scratches.

Never rely on a single optical disc copy, nor just on a drive-based copy. repeated use of the optical disk may degrade the disk (scratches, etc) and the data layer (through repeated exposure to the laser). A drive on the other hand can be damaged in a multitude of ways -- the mechanical parts may fail, the platters may get scratched, the magnetic surface may degrade, the disk may be exposed to strong magnetic fields, etc.

Surprisingly, the preferred long-term storage device (archival purposes) for a lot of companies remain the venerable magnetic tape. However, for personal purposes, I stick with the combination of DVDs and mechanical drives. I don't use online storage for security purposes, especially some of my photography works which are a bit "sensitive."

[oj88]

^ Just a couple cents worth of thought...

In that case, I'd suggest you skip storing data on optical media altogether. Byte for byte, hard disk storage is cheaper and is far more reliable (higher MTBF) than say a high quality DL DVD-R or the drive it is authored on.

For example, storing 2TB of data, you'll need approx. 223 DL DVD-R disks. Imagine having to check and re-master 223 disks every couple of years!

Just some simple/rough computation:

Typical branded Dual-Layer DVD-R disk = Php60 each x 223 disks = Php 13,380

An over the counter 2TB SATA HDD would cost under Php8k. Get two for around Php15k and you can have two identical backup copies. Store one off-site. Needless to say, it's easier to manage archived data using folders, instead of having to look for a single file across multiple physical DVD disks.

[digitalron]

Agreed on the cost factor that mechanical disks are hands-down cheaper than optical discs on a per GB basis. However, if cost is not an issue, or if the data is really valuable, I'd still keep an optical copy. In my case, I'm also selective in what I copy to optical disks, so the cost isn't too big. I've had CDs burned as far back as 2002 and these are still readable.

Manufacturers often cite 20-50 years for their (regular) optical discs; I'd personally put it at 10-15 for regular media (at least based on my own experience). Maybe some gold-based archival discs may last up to 70 years, though it's expensive at roughly US$2 a disc (Verbatim Gold DVDs are around US$80 per spindle of 50 pieces).

The reason I don't rely on HDD-only backup is their electro-mechanical nature: multiple parts are vulnerable to failure, and the threat of strong magnetic fields is scary. Even atmospheric issues can be problematic as high humidity can cause parts to rust, while mold and fungi can grow.

In our company, as an example, we do have disk-based backup (using multi-terabyte EMCs in three locations plus portable NAS) but for compliance/audit purposes, we also back-up to tape. For operational efficiency, some files are also backed up to DVD and served by what we call a DVD-jukebox.

I suppose it all boils down to economics: if the data (video, audio, files, etc) is easily replaceable or is not super-valuable, then HDD-based backups should suffice; if the data is very valuable, then a multi-pronged solution is advisable.

[ghosthunter]

But there are findings and reports even by the optical disc manufacturers themselves that the CDRs and DVDRs that are currently being manufactured and used for data backup only last two to five years.

The foil layer might last fifty years but the plastic itself is the one being degraded faster than expected. Some have "rot" starting from the disc's label design. Other problems have the laminate layers separating and causing the foil to corrode after exposure to air.

[digitalron]

But there are findings and reports even by the optical disc manufacturers themselves that the CDRs and DVDRs that are currently being manufactured and used for data backup only last two to five years.

The foil layer might last fifty years but the plastic itself is the one being degraded faster than expected. Some have "rot" starting from the disc's label design. Other problems have the laminate layers separating and causing the foil to corrode after exposure to air.

I very much agree that some discs do "rot" and "bronze" after only five years, or even less. This is because of substandard components used to manufacture either the recording substrate (which can tear or oxidize), the laminate, or the disc polycarbonate (which can scratch, shatter, or get misshapen).

The handling of the disc will also play a great deal in how long it will last. Repeated exposure to humidity, heat, light, changing temperature, and an acidic atmosphere (as well as acidic/oily fingers) will cause deterioration to set in quickly.

However, most of these suspect discs are the cheaper re-recordable/re-writable types. Archival quality discs are more resilient, with double foil layers (such as gold as the external protective film), thicker laminates, better polycarbonates, etc. This, coupled with careful storage and handling (gloved hands!), should present a relatively reasonable way to archive data. They're also more portable to lug around and the dangers of damaging it on transit are a bit less than a drive.

That said (and as oj88 correctly pointed out), it is a bit of a PITA to check a couple hundred of discs, so an alternative is to store data in at least two (preferably three) large capacity external hard disk drives. One (or two) will be stored safely as the master archive (foam-padded, humidity-controlled boxes, preferably different physical locations) while one can be for regular use. Just note that the internal lubricants of a drive can dry out after 5-6 years, so it might be a good idea to check on the master archive(s) at least once a year and on the 5th year or so, clone it to a new drive.

[oj88]

I consider optical media as very short-term storage. It's only ideal for portable and disposable use, and is in fact the functional equivalent of the floppy disk from a decade ago.

Consider the time spent burning just 40GB of data (equivalent to roughly 5 dual-layer DVDs)... It WILL take a while. Then if you are the careful one, you'd want to verify the data after it has been recorded, eating up several more minutes of your time. And THEN, you have to be there, shuffling and changing disks, etc.

Now, the context of this thread is probably directed towards individual users thinking of ways to preserve their personal data (pictures, videos, digital documents, etc.), and not corporations with the financial capacity to buy the best backup solution. So that kinda puts "temp-controlled vaults" for personal use out of the question. :D

Anyway, HDD technology have matured dramatically compared to MFM/RLL drives of yesteryears. So much so that the stories about the two feared enemy of the HDD, the magnet and impact/shock, are usually put out of proportion. It is true that a sufficiently powerful level of magnetism placed in just the right way will corrupt data inside a HDD, but it has to be strong enough or the proximity close enough before any harm is done. One of the strongest magnets known to man, neodymium, is being used INSIDE the hard disk in conjunction with a voice coil to move the head actuator. I used to collect these from defunct hard drives and was stupid enough to get pinched by a pair of these magnets. They're really strong! But yeah, for those in the know, they are installed in a way that the magnetic flux are practically facing away from the platter, so the residual magnetic field is way below the threshold of the platter.

But I do agree that HDDs, still being mechanical, are bound to fail eventually. That's why it is vital to keep copies of your backup on two separate HDDs. It's statistically improbable that both HDDs will die at the same time, even if they were assembled on the same day by the same person. Even so, it's so much more easier to work with HDDs. Data transfer is fast and reliable, the capacity is scalable, it's relatively cheap, and it's available anywhere. The convenience and performance alone far outweighs any if its shortcomings.

[ghosthunter]

Talking about neodymium magnets and hard disks, I was able to corrupt a 2.5" HDD using a single neodymium magnet place on top of it.

Your data on your HDDs may survive 70 to 100 Gs of mechanical impact but it won't survive a few seconds exposure to a neodymium magnet.

[flakez]

gamit ko DVD saka meron din akong 500gb na external HD. considering on buying another 1tb HD.