DNG files. Problems ?

Joseph Chamberlain, DDS wrote:
[snip]

Have any of you converted your images to DNG ? In case you have, what have been your experiences with DNG and what do you think about it in terms of security of your original raw files ?

[snip]

I’ll repeat the response I gave to your identical question in the other forums:

Most of the "fear, uncertainty, doubt" that I see about DNG in forums arises from inexperience with it, or lack of some significant information. (Some of the FUD can be attributed to people with responsibility for, or an allegience to, products that either don’t support it at all, or do so only in a limited way).

People need to be comfortable with their workflow. I don’t think people who keep their original raw files can be criticised. There are probably some formats where some data is not preserved in the DNG file. Adobe don’t appear to have published suitable reference material on this. What there is tends to be relatively brief statements from Thomas Knoll, etc. When DNG was launched, Adobe recommended people to keep their original raws. I haven’t seen that advice officially withdrawn, and doing so might invite product liability problems. Software from camera makers mostly doesn’t support it, so if people want to be able to use that, they must keep the original raw files.

DNG isn’t really "new" any longer. It was launched more than 14 months ago. The specification is on the 2nd version. ACR-for-DNG and the DNG Converter are on the about the 5th version (assuming you don’t count
3.0). There are perhaps 80+ non-Adobe products that support DNG in some
manner:
http://www.barry.pearson.name/articles/dng/

Month-by-month, the world becomes a more "DNG-friendly" place, with more people using it, more products supporting it, more endorsements and recommendations, etc. And I doubt that Adobe has stopped adding useful features to its support!

Adobe couldn’t drop support for DNG and remain in the raw processing business. Many photographers wouldn’t upgrade to a version of their Adobe products that dropped DNG, because so many of their photographs and the settings and other metadata within them depend on DNG. Many of those photographers are opinion leaders. It would seriously damage Adobe’s credibility. (But why would Adobe want to do so anyway?)

(I started to use DNG nearly 14 months ago. I stopped retaining my original raw files about 6 months ago, soon after upgrading to CS2 /
3.1, which retained (apparently) all of the data from my original raw
files in the DNGs).

—
Barry Pearson
http://www.barry.pearson.name/photography/
http://www.birdsandanimals.info/

Dear members:

In a discussion thread on one of Apple’s forum on the new product Aperture some posters reported having problems with DNG files and that they didn’t feel safe committing their digital raw files to this new file specification. Some felt that converting to DNG and erasing the original raw files was a risky proposition since the new standard could either have bugs or fail to receive support and end up being dropped.

snip
This is a good thing. It is gathering support and bringing forth ideas that will create a file format for RAW files in it’s concept that it compares to what ASCII and ANSI has done for decades.

Be sure to check this link on Adobe web.
http://www.adobe.com/products/dng/main.html

I’m not an expert here, but the common sense position seems to me to be this. Keep the original data (in my case CR2 files). If at some later stage Canon stops supporting them, I’ll change to DNG then. If however DNG becomes the TIFF of RAW, again I’ll convert then. None of us need to convert now

At the moment the excellent Canon Zoom Browser (which IMHO knocks spots off the cumbersome Bridge) allows me to find my CR2 files and their data as well as TIFFs and JPEGs so there is no value in conversion at thgis juncture

Gordon P

"rumpledickskin" wrote in message

Be sure to check this link on Adobe web.
http://www.adobe.com/products/dng/main.html

Gordon Pembury wrote:

I’m not an expert here, but the common sense position seems to me to be this. Keep the original data (in my case CR2 files). If at some later stage Canon stops supporting them, I’ll change to DNG then. If however DNG becomes the TIFF of RAW, again I’ll convert then. None of us need to convert now

[snip]

There is a specific advantage to a DNG-based workflow that isn’t obtained other ways. That is, the ability to hold a range of metadata within the raw (DNG) file itself. This can make file management easier. (It is one of the reasons I upgraded from CS to CS2).

This metadata can include all the ACR edits and settings, plus any camera calibration used there. Plus other metadata used for "rights", such as name, copyright information, website. Plus other metadata used for asset management, such as IPTC-like keyworsd for the shoot and the individual images.

While some products can add metadata to camera makers’ raw files, (for example Nikon Capture can do that to NEFs), it can be risky doing that to a format that had to be reverse-engineered. Both DNG and XMP are publicly documented, so it is safer.

There are only a few non-Adobe products that support the XMP metadata within DNG files, but I expect the number will gradually increase: http://www.barry.pearson.name/articles/dng/xmp_dng.htm

—
Barry Pearson
http://www.barry.pearson.name/photography/
http://www.birdsandanimals.info/

"Joseph Chamberlain, DDS" wrote in message

Have any of you converted your images to DNG ? In case you have, what have been your experiences with DNG and what do you think about it in terms of security of your original raw files ?

With DNG (and I’m assuming that this is the same for the Apple world) you have the option of embedding the entire contents of the original (in your case) *.CR2 file – so that it can be extracted again in future if need be.

I switched to DNG some time ago, and never looked back … they’re smaller (with lossless compression) and they don’t create pesky ‘companion’ xmp files.

Even in the extremely unlikely event of the dng format being dropped and unsupported then it’s still a trivial matter to convert any old files to a newer format using existing apps.

On 12/9/05 3:48 AM, in article
, "Barry Pearson"
wrote:

There is a specific advantage to a DNG-based workflow that isn’t obtained other ways. That is, the ability to hold a range of metadata within the raw (DNG) file itself. This can make file management easier. (It is one of the reasons I upgraded from CS to CS2).

This metadata can include all the ACR edits and settings, plus any camera calibration used there. Plus other metadata used for "rights", such as name, copyright information, website. Plus other metadata used for asset management, such as IPTC-like keyworsd for the shoot and the individual images.

While some products can add metadata to camera makers’ raw files, (for example Nikon Capture can do that to NEFs), it can be risky doing that to a format that had to be reverse-engineered. Both DNG and XMP are publicly documented, so it is safer.

There are only a few non-Adobe products that support the XMP metadata within DNG files, but I expect the number will gradually increase: http://www.barry.pearson.name/articles/dng/xmp_dng.htm
—
Barry Pearson

Barry and others:

This is what has drawn my attention to DNG. The idea of having two files for every image makes the management of ones database or photo library a lot more complicated. When exporting or transferring photos one must always remember to export along with them the respective sidecar files otherwise these images will be opened again as "virgin" images requiring to be worked on.

The problems I see are the (1) current lack of support for the DNG format and (2) reports that users have experienced problems in the form of bugs related to either the conversion to or from DNG.

I know that with DNG the user as the option between embedding the RAW images on a permanent base (in which case they become non-retrievable) or to import the RAW image into DNG while maintaining the original file structure so that they can be exported in the future in the same exact state they were when they were converted. The problem is that in choosing the second option one really has no absolute guarantee that problems either with the current specification or the DNG converter will not cause the file to become corrupted and incapable of being retrieved in the future. This is the concern I have.

In regards to the option of converting to DNG and keeping the original Raw files as a back-up to safeguard against possible future problems, I see this as a contradiction to the essence of what DNG is supposed to stand for. In case I were to follow this approach I would then end up with not 2 but actually 4 copies of each image which represents a serious storage problem when you are dealing with 17 MB images as I am. The reason I say this is because I would keep the RAW images in my system and make a back-up copy on an external hard drive. I would then do the same with the DNG files since these are essentially different from the RAW files as they contain all metadata with Raw conversion settings and other metadata that relates to each image. I would want to loose all the information and settings I have applied to my images. In case I feel I have to back-up because of my fear that something might happen to my DNG converted images than it makes more send to avoid using it altogether.

The type of guarantee I would like to have is that in converting to DNG I will have "absolutely" no problem in the future if I choose to extract the original RAW files without them having been corrupted or touched in anyway by this conversion process.

These are basically the concerns I have.

Best regards,

Joseph

—

Dr. Joseph Chamberlain
Oral and Maxillofacial Surgery

Joseph Chamberlain, DDS wrote:

On 12/9/05 3:48 AM, in article
, "Barry Pearson"
wrote:

There is a specific advantage to a DNG-based workflow that isn’t obtained other ways. That is, the ability to hold a range of metadata within the raw (DNG) file itself. This can make file management easier. (It is one of the reasons I upgraded from CS to CS2).

This metadata can include all the ACR edits and settings, plus any camera calibration used there. Plus other metadata used for "rights", such as name, copyright information, website. Plus other metadata used for asset management, such as IPTC-like keyworsd for the shoot and the individual images.

While some products can add metadata to camera makers’ raw files, (for example Nikon Capture can do that to NEFs), it can be risky doing that to a format that had to be reverse-engineered. Both DNG and XMP are publicly documented, so it is safer.

There are only a few non-Adobe products that support the XMP metadata within DNG files, but I expect the number will gradually increase: http://www.barry.pearson.name/articles/dng/xmp_dng.htm
—
Barry Pearson

Barry and others:

This is what has drawn my attention to DNG. The idea of having two files for every image makes the management of ones database or photo library a lot more complicated. When exporting or transferring photos one must always remember to export along with them the respective sidecar files otherwise these images will be opened again as "virgin" images requiring to be worked on.

The problems I see are the (1) current lack of support for the DNG format and (2) reports that users have experienced problems in the form of bugs related to either the conversion to or from DNG.

I know that with DNG the user as the option between embedding the RAW images on a permanent base (in which case they become non-retrievable) or to import the RAW image into DNG while maintaining the original file structure so that they can be exported in the future in the same exact state they were when they were converted. The problem is that in choosing the second option one really has no absolute guarantee that problems either with the current specification or the DNG converter will not cause the file to become corrupted and incapable of being retrieved in the future. This is the concern I have.

In regards to the option of converting to DNG and keeping the original Raw files as a back-up to safeguard against possible future problems, I see this as a contradiction to the essence of what DNG is supposed to stand for. In case I were to follow this approach I would then end up with not 2 but actually 4 copies of each image which represents a serious storage problem when you are dealing with 17 MB images as I am. The reason I say this is because I would keep the RAW images in my system and make a back-up copy on an external hard drive. I would then do the same with the DNG files since these are essentially different from the RAW files as they contain all metadata with Raw conversion settings and other metadata that relates to each image. I would want to loose all the information and settings I have applied to my images. In case I feel I have to back-up because of my fear that something might happen to my DNG converted images than it makes more send to avoid using it altogether.

The type of guarantee I would like to have is that in converting to DNG I will have "absolutely" no problem in the future if I choose to extract the original RAW files without them having been corrupted or touched in anyway by this conversion process.

These are basically the concerns I have.

Best regards,

Joseph

—

Dr. Joseph Chamberlain
Oral and Maxillofacial Surgery

I think you are over complicating the whole issue. DNG is a digital negative storage format, subtly altered from the original, proprietary RAW file data into one with an emerging but universal storage format.

To suggest one of the biggest software inventors in the world and certainly the largest and most important in the field of graphics and pre-press might somehow have produced a standardized format for storing digital images that is inferior to or worse than any of the half a dozen proprietary file systems is preposterous.

Keep your original RAW files if you must but they are only data, not image files. It is when you develop them that they become images. There is nothing basically wrong with standardizing your workflow to convert RAW files to DNG and storing the developed images as maximum strength JPEG files, keeping the originals as DNG files somewhere away from your place of work.

If you start storing all your files on a central computer, you run the risk of overloading the hard drive, losing the files if the drive collapses and generally compromising a decent archive system.

Once a hard drive reaches 50% capacity, it begins to slow down until at about 90% capacity is gets slow enough to be a pain. Might I suggest the following?

Sort your keepers as you transfer files from the cards to your computer. I presume Macs have as many programs available to do this as a PC does. Convert those keepers to DNGs and burn them to a CD or DVD. CDs generally outlast DVDs for storage.

Use the RAW files to develop into image and store those images on your computer for retrieval as you need them. This workflow requires you to lose your paranoia and delete the RAW files after converting them to storage files (DNGs). This way you might get control of your archive before it gets out of control.

—
Alienjones…
Writer of the last FAQ.

Yup. Way too many worries. It’s not like RAW is going to drop dead tomorrow and all your data become worthless magnetic (or optic) trash. So keep’em. Why wouldn’t Photo(yourprogramhere) retain compatibility with your camera’s format.

First of all. If you are worried one iota about losing confidence in DNG… don’t convert! Period.

Give yourself a trial period and convert everything you are currently shooting while keeping an orignal RAW as well.

Second. You KNOW there is going to be someone out there who can/will create a reconverter – bits are bits.
Celcius to Farenheit and back again. It’s not a VCR connection.