PS reports DIB resolution incorrectly?

AM
Posted By
Andrew Morton
Jan 30, 2004
Views
675
Replies
6
Status
Closed
What I do:-
I am writing an image to a file as a Windows bitmap from a program I wrote. I want to set the resolution in the file header to 300ppi. Now, in the header the resolution in specified in pixels per metre [1], so I calculate 300ppi=11811px/m (multiply by 39.37 in/m [2]). I then convert this to \x23\x2E\x00\x00 (Intel format dword) and write it in the header (both x- and y-resolutions)

What goes wrong:-
So far, so good. BUT when the file is opened in Photoshop 6.0.1 the resolution is reported as 300.125ppi. If I change the resolution in Photoshop to exactly 300ppi then the data in the file is \x20\x2E\x00\x00, which is 11808px/m (=299.924ppi in reality).
Same problem at 72ppi: the maths says \x13\x0B\x00\x00 but PS writes \x12\x0B\x00\x00
If I pretend that there are 39.36 in/m then 300ppi converts to what PS claims, but 72ppi does not.

Question:-
Is there some convention regarding resolution settings whereby a fiddle-factor is used?

Andrew

[1]: BITMAPHEADERINFO
http://msdn.microsoft.com/library/default.asp?url=/library/e n-us/gdi/bitmaps_1rw2.asp

[2]: NIST Guide to SI Units
http://physics.nist.gov/Pubs/SP811/appenB.html

How to Improve Photoshop Performance

Learn how to optimize Photoshop for maximum speed, troubleshoot common issues, and keep your projects organized so that you can work faster than ever before!

T
toby
Jan 31, 2004
"Andrew Morton" …
What I do:-
I am writing an image to a file as a Windows bitmap from a program I wrote. I want to set the resolution in the file header to 300ppi. Now, in the header the resolution in specified in pixels per metre [1], so I calculate 300ppi=11811px/m (multiply by 39.37 in/m [2]). I then convert this to \x23\x2E\x00\x00 (Intel format dword) and write it in the header (both x- and y-resolutions)

What goes wrong:-
So far, so good. BUT when the file is opened in Photoshop 6.0.1 the resolution is reported as 300.125ppi. If I change the resolution in Photoshop to exactly 300ppi then the data in the file is \x20\x2E\x00\x00, which is 11808px/m (=299.924ppi in reality).
Same problem at 72ppi: the maths says \x13\x0B\x00\x00 but PS writes \x12\x0B\x00\x00
If I pretend that there are 39.36 in/m then 300ppi converts to what PS claims, but 72ppi does not.

Question:-
Is there some convention regarding resolution settings whereby a fiddle-factor is used?

Hi Andrew

Photoshop works with resolution values as fixed point numbers, with 16 fraction bits. One could use this code to convert from the BMP field: pb->imageHRes = rint( hdr->biXPelsPerMeter / (39.37/65536.) ); This preserves as much precision as possible, and produces exactly 300.0 when given your computed biXPelsPerMeter of 11811.

There is no fudge factor but I suspect the calculation is just sloppily coded in Photoshop’s BMP format module…perhaps it uses fixed point arithmetic (which has very low precision) instead of floating point as above. Congratulations, you’ve probably found a bug.

One workaround might be for you to use a different intermediate file format. Even TIFF is quite easy to create with something like libtiff (http://www.libtiff.org/).

Toby

Andrew

[1]: BITMAPHEADERINFO
http://msdn.microsoft.com/library/default.asp?url=/library/e n-us/gdi/bitmaps_1rw2.asp
[2]: NIST Guide to SI Units
http://physics.nist.gov/Pubs/SP811/appenB.html
AM
Andrew Morton
Feb 2, 2004
Thanks for replying, Toby.

What is even more peculiar is that in the Image->Image Size… dialogue, PS6.0.1 correctly recognizes 300ppi as 118.11px/cm but still writes 11808px/m in the biXPelsPerMeter field. Further investigation needed…

Trying to enter very small/large resolutions in the relevant box reveals that PS6 can cope with resolutions in the range 1..30000ppi (~0.393..11811.023 px/cm). Anything lower or higher found in the file is assumed to be 72ppi. For resolutions which PS does not cope with, if the file is saved then it will rewrite the resolution in the file header as 72ppi, regardless of over- or underflow. Well, actually it writes 2834px/m which is 71.984ppi rather than 2835px/m which is closer to 72ppi at 72.009ppi.

OMG, it gets worse: set a file to 1.01ppi and save it. In the file header, it says \x28 px/m. Now open and save again with PS and it changes the resolution to \x027 px/m. Similarly for 300.1ppi which it writes the first time as \x2E24 (300.025ppi) then after save/open changes its mind and calls it 300.125ppi which it writes as the number \x2E25 (300.051ppi). Yes, I know decimal fractions are recurring fractions when written in binary.

It seems that resolutions are quantized into steps of 0.125ppi [1], involving being chopped then rounded (up?) rather than rounded.

Come on Adobe, show us your code, I want to know where you’re rounding/chopping things up/down/off, then I could have an "Adobe (TM)Photoshop (R) compliant resolution stamp" checkbox in my program.

At least it’s consistent between Mac and PC PS 🙁

Andrew

[1] Not entirely unreasonable at higher resolutions, I suppose paper could change size more than that with varying moisture content.
T
toby
Feb 4, 2004
"Andrew Morton" …
Thanks for replying, Toby.

What is even more peculiar is that in the Image->Image Size… dialogue, PS6.0.1 correctly recognizes 300ppi as 118.11px/cm but still writes 11808px/m in the biXPelsPerMeter field. Further investigation needed…

Trying to enter very small/large resolutions in the relevant box reveals that PS6 can cope with resolutions in the range 1..30000ppi (~0.393..11811.023 px/cm). Anything lower or higher found in the file is assumed to be 72ppi. For resolutions which PS does not cope with, if the file is saved then it will rewrite the resolution in the file header as 72ppi, regardless of over- or underflow. Well, actually it writes 2834px/m which is 71.984ppi rather than 2835px/m which is closer to 72ppi at 72.009ppi.

That’s to be expected, as the 32-bit fixed point format which stores resolution values can only represent up to about 32767.99998 anyway. They limited various quantities to a round 30,000 including the well-known restriction on image size (up to PS7).

OMG, it gets worse: set a file to 1.01ppi and save it. In the file header, it says \x28 px/m. Now open and save again with PS and it changes the resolution to \x027 px/m. Similarly for 300.1ppi which it writes the first time as \x2E24 (300.025ppi) then after save/open changes its mind and calls it 300.125ppi which it writes as the number \x2E25 (300.051ppi). Yes, I know decimal fractions are recurring fractions when written in binary.

….and if they’re doing the entire conversion in fixed point, they don’t recur very far, either.

It seems that resolutions are quantized into steps of 0.125ppi [1], involving being chopped then rounded (up?) rather than rounded.

Come on Adobe, show us your code, I want to know where you’re rounding/chopping things up/down/off, then I could have an "Adobe (TM)Photoshop (R) compliant resolution stamp" checkbox in my program.

At least it’s consistent between Mac and PC PS 🙁

My you have spent a lot of time investigating. Unfortunately they can’t show us the code and chances are they won’t fix it either – although you could try reporting it to
http://www.adobe.com/misc/bugreport.html

Cut your losses and move straight to the workaround. I certainly wouldn’t consider emulating the problem.

T

Andrew

[1] Not entirely unreasonable at higher resolutions, I suppose paper could change size more than that with varying moisture content.
AM
Andrew Morton
Feb 4, 2004
My you have spent a lot of time investigating….

Exasperation 🙂

although you could try reporting it to
http://www.adobe.com/misc/bugreport.html

Given it a go, although I won’t be holding my breath waiting for an explanation.

Cut your losses and move straight to the workaround.

Agreed.

I certainly wouldn’t consider emulating the problem.

It looks like Macromedia know something about it because Fireworks does exactly the same thing when writing a file. And I bet they won’t tell me why either. Oh well, look-up table it is…

Thanks,

Andrew
AM
Andrew Morton
Feb 5, 2004
It turns out that, to convert from ppi to px/m, pretending that there are 39.36096 in/m then rounding gives the Photoshop value.

Andrew
T
toby
Feb 5, 2004
"Andrew Morton" …
My you have spent a lot of time investigating….

Exasperation 🙂

although you could try reporting it to
http://www.adobe.com/misc/bugreport.html

Given it a go, although I won’t be holding my breath waiting for an explanation.
Cut your losses and move straight to the workaround.

Agreed.

I certainly wouldn’t consider emulating the problem.

It looks like Macromedia know something about it because Fireworks does exactly the same thing when writing a file. And I bet they won’t tell me why either. Oh well, look-up table it is…

What do you mean look-up table? The conversion is simply a multiplication by a scale factor (see my earlier post). Adobe may have screwed it up, that doesn’t mean you have to.

If you’re concerned that Photoshop won’t read the correct resolution from your generated file, take no chances and use a different intermediate file format. Even PSD format is sufficiently documented, although I recommend something like TIFF.
T

Thanks,

Andrew

Master Retouching Hair

Learn how to rescue details, remove flyaways, add volume, and enhance the definition of hair in any photo. We break down every tool and technique in Photoshop to get picture-perfect hair, every time.

Related Discussion Topics

Nice and short text about related topics in discussion sections