PS7 Benchmarks for power users : Dual Xeon, G5, P4, AMD

ED
Posted By
ellen devito
Sep 19, 2003
Views
2410
Replies
99
Status
Closed
This is for you real power users out there.

I’ve been putting my brand new Dual Xeon rig through some performance tests, and I thought that some of you out there might find this useful, as this is what I would have wanted to know.

my setup:

Dual Xeon 3.06 procs with HT enabled
4GB DDR RAM
Supermicro X5-DALG
Seagate 15K 73GB
Western Digital Caviar 250GB
Superspeed RamDisk XP Pro (drive R:) set as primary scratch: 1.5 GB.

PS7Bench 1.1 Advanced: Benchmarks using a 50MB test file and 21 different actions times added up:

Dual Xeon 3.06 with HT (my rig)>>>>>>>90.8 seconds G5 2x 2.0ghz>>>>>>>>>>>>>>>>>>>>>>>>>95.1 seconds
P4 3.06 with HT>>>>>>>>>>>>>>>>>>>>>>>124.6 seconds
Athlon MP 2x 2200+>>>>>>>>>>>>>>>>>>>>134.6 seconds
Athlon 3000+>>>>>>>>>>>>>>>>>>>>>>>>>>148 seconds
G5 single 1.6ghz>>>>>>>>>>>>>>>>>>>>>>180.1 seconds

for all the details see:

<http://www.chaosmint.com/benchmarks/powermac-g5-ps7bench/> <http://www.aceshardware.com/read.jsp?id=50000371>

I find PS7Bench to be the best benchmark out there.
Its a simple 5KB action file, that creates a 50MB file, and then uses the timer function within PS itself to measure performance.

Its a TRUE crossplatform action, there can be no arguing over compilers, codes, and fancy benhcmarking software. Otherwise go get a degree in engineering to understand the online debate if Apple really lied in their G5 benchmarks:

<http://spl.haxial.com/apple-powermac-G5/>

if someone tells me that on their G5, Ps7Bench #18 (Watercolor) takes 26 seconds on the EXACT SAME 50MB FILE, then I can realistically see performance and how it relates to ME.

Remember that RAM and scratch disk are irrelevant in these tests -as a 50MB file never uses up enough RAM, so this a great benchmark for sheer processor power.

BUT: what about files 500MB+ ?

I have 4GB of RAM. the MAXIMUM Win XP Pro allows.

AND as I have learned on the forums, no program can address more than 2GB -yes, even the G5, with its ads of "8 GB of RAM!" -6 GB of that goes to waste! you can’t use it. looks like we do need true 64 bit OS’s AND SOFTWARE.

Even worse, PS doesnt even actually USE all that 2 GB of RAM it "should." PS only wants to give me 1777MB when memory usage is set to 100%. Up to that 1777MB, my rig flies. but once PS uses it all up, performance dies considerably fast. ressing a file up to 225MB takes 2 seconds, ressing it up to 557MB takes 100 seconds….

my conclusions:

Looks like all that talk about separating the scratch disk is for real, but ONLY ONLY ONLY when PS really runs out memory. till then, all my tests ran within fractions of a second within each other:

I set the scratch first to my 15K seagate, then my 7200 WD, then the ramdisk. even though the seagate is some TWICE as fast as the Western Digital drive, the WD improves bec. it is a separate drive. and the Ramdisk is even a better scratch bec. its actually creating a "separate drive" out of the extra RAM PS can’t use!

So, on huge files when u run out of memory:

1) A separate hard drive as scratch disk improves performance some 25-100% when PS had used up its RAM (past 1777MB) usually on files 250MB+

2) Using a Ramdisk definitely improves performance, despite Chris Cox’s assurance to the contrary in this forum before. Using Superspeed Ram Disk XP Pro, and setting it as the primary scratch disk, files 500MB+ performed some 25%+ better than a separate hard disk as scratch-

BUT:

NOWHERE NEAR as well as when PS had enough RAM. strangely, "even though" PS is being "tricked" by the Ramdisk into using the actual RAM as a scratch, it doesnt work nearly as well (some 10-20x slower than when under the 1777MB memory limit!)

So the irony is: for all these "best computer" posts in the future: it doesn’t exist! until Chris Cox and co. write PS to address more than 2GB, and windows finally goes to 64 bit, and you get a 64bit chip, you power users out there will just have to wait.

Must-have mockup pack for every graphic designer 🔥🔥🔥

Easy-to-use drag-n-drop Photoshop scene creator with more than 2800 items.

P
Phosphor
Sep 19, 2003
How many times do I have to say that PSBench is a lousy bencmark?

It puts far too much weight on filters that nobody uses. And the PSBench files are now way too small for modern systems (200 Meg is a minimum for repeatable numbers, 400 to 800 Meg even better).

PS Can’t use all 2 Gig because the OS puts the binary (application, DLLs, etc.) in that same 2 Gig. We use all that we can…..

And Apple didn’t lie in their benchmarks, however much the WinTrolls want to believe that. I verified Apple’s Photoshop results myself.

Also, mister "spl.haxial.com" was debunked months ago as just another troll (heck, if you get the bottom of his arguments it’s pretty clear that he’s a troll).
LH
Lawrence Hudetz
Sep 19, 2003
Q: When is a standard not a standard?

A: When there is more than one.

Until a standards committee develops a set of standards that ignore platforms and treats the device as a black box, we will continue discussions like this ad infinitum, ad nauseam! Reminds me of the fights about film speed/development years ago. I’m not sure that one ever really got settled. I think it got ignored due to decreased interest as digital came along.
RH
r_harvey
Sep 19, 2003
Benchmarks are like statistics.

Panatomic-X and D-76 (with a pinch of anti-fog) are the one true path.
LH
Lawrence Hudetz
Sep 19, 2003
There are lies, damned lies, and statistics.

Someone said that.

Panatomic-X. A relic today. Not available any more. For the second time in it’s life.

Panatomic-X and FG-7. Forget the anti-fog. Slows down too much. (See what I mean? Hasn’t been settled!!)
Y
YrbkMgr
Sep 19, 2003
I contend that there is absolutely no utility in trying to compare Windows and Mac platforms from a benchmarking perspective.
LH
Lawrence Hudetz
Sep 19, 2003
I would tend to agree, Tony. Even comparing between same os different hardware is hard to reconcile in some cases.
P
Phosphor
Sep 19, 2003
Another thread about this on the Mac side : <http://www.adobeforums.com/cgi-bin/webx?128@@.2ccd71b5> in which I posted a link to the Ars Technica forums where they normalize the results and discuss about another flaw of PS bench: it uses Photoshop’s timer function that has only one decimal…

I would have expected more boost trough the ramdisk as scratchdisk… Could you post numbers out of a test on a 500Mb file comparing the different drives as a scratch?

Did you test other ram disks? (cenatek, etc…)
ME
mike.engles
Sep 19, 2003
Hello

There is a similar thread in the Apple forum.
Chris Cox always says that PSbench is no good, because it uses filters that no one uses a great deal in real life.
In any case why does it matter as we are just testing. All machines run the same test on the same image. All that is different are the machines. We need to ensure that the machine configurations are the same, or as close as possible.

PsBench is a action file, which will run any filters.
Why does Chris not suggest which filters would be representative. I have done a small comparison using 50 mB images of various pixel sizes and pixel contents.

It is at this link.
<http://www.btinternet.com/~mike.engles/bench.jpg>

It seems to me that filters like sharp, blur, resize, rotate are pretty consistent. A couple seem to do better according to the image pixels. CMYK on the vector image with a lot of white runs very fast, relative to the others.

Perhaps we should also agree, if we are going to use Photoshop for benchmarking, that at least we use the same image.
There is no reason why we do not use one of the images on the PS disk, resized to 50,100 or 200mb

Mike Engles
ED
ellen devito
Sep 20, 2003
I agree about the filters -I’m sure there could be a better version of PSBench that would use more common actions -but for now, its at least something you, me and the candlestickmaker can actually benchmark for ourselves -instead of relying on PcMag or online zealots…

yes, i also tried cenatek’s ramdisk, but it crashed xp over and over. I too was disapointed by the ramdisk.

My numbers:

I created a resize action where I take a 50MB image and res it up in 10% increments over and over. this is normally how i would res something up anyway, sans say Genuine Fractals…

the "same drive" is where PS7, the OS and the scratch reside on the same drive (my Seagate 15K), "diff drive" is where i set scratch to my Western Digital 7200 drive, and the last is the Superspeed ramdisk.

all times are in seconds.

FileSize>>>>>>>SAME DRIVE>>>>>>>DIFF DRIVE>>>>>>RAMDISK

100 MB>>>>>>>>>>>>>>>>1.6>>>>>>>>>>>>>>1.6>>>>>>>>>>>>>1.6
214 MB>>>>>>>>>>>>>>>>3.1>>>>>>>>>>>>>>3.1>>>>>>>>>>>>>3.1
260 MB>>>>>>>>>>>>>>>17.0>>>>>>>>>>>>>24.2>>>>>>>>>>>>25.0
314 MB>>>>>>>>>>>>>>140.0>>>>>>>>>>>>>84.9>>>>>>>>>>>>73.1
380 MB>>>>>>>>>>>>>>>63.8>>>>>>>>>>>>>43.7>>>>>>>>>>>>10.1
460 MB>>>>>>>>>>>>>>167>>>>>>>>>>>>>>132.9>>>>>>>>>>>129.9
557 MB>>>>>>>>>>>>>>200>>>>>>>>>>>>>>170>>>>>>>>>>>>>153.8

from 100 MB to 214 MB, there was no real difference. 260MB seems to be "the cutoff" when PS used up all its 1777MB of RAM, as times go up over 5x. strangely, the "same drive" wins the first one @ 260MB.

on only the 380 MB increase does the RAMdisk radically outperform the separate drive, from 10 secs to 40 secs.

even more interesting, I monitored just how much scratch PS wants. even when it is using memory (ie up to the 1777 limit) it still creats huge scratch files irregardless:

a 214 MB resize used 1340 MB of my RAM and some 1500 MB of cache! whereas the 380 MB file needed 2910 cache plus the 1700 MB RAM already used up.

from my quick estimates, it seems PS needs some 6-8x of CACHE space, in addition to all the RAM it uses > a 50 MB – 536 MB "ressup" uses over 6 GB of ram and cache! so even if you have a ramdisk, u need a gigantic one unfortunately.
RH
r_harvey
Sep 20, 2003
Well, the Mac side won’t mention that with Panatomic-X, in Microdol-X at 1:3, it’s hard to see the grain in a scan–or the edge of anything, either.
ME
mike.engles
Sep 20, 2003
Hello

So the dual Xeon with 4 GB of ram is not quite twice as fast as my Pentium IV 2.53Ghz overclocked to 2.63Ghz with 1Gb ram, which takes 176 secs.

I would have expected it to be at least four times as fast for that kind of money.

The best value seems to be the Pentium 3.06Ghz

Mike Engles
H
Ho
Sep 20, 2003
It puts far too much weight on filters that nobody uses.

So what? If you and I road race on a highway that no one ever drives on, and you lose, then you’ve still lost. You may complain that my car was set up better for the course than yours was, and that may be true. Better luck next time.

Why does Chris not suggest which filters would be representative?

I asked this same question way over a year ago. I’m beginning to think that any test which even hints that a PC can stand on the same ground as a Mac is viewed as a virus by the Mac Illuminati.

Fact is, I really could not care less. I built my PC in ’99 and it probably couldn’t keep up with anything you all are using. But I enjoy being a Sunday driver on the information superhighway…
ME
mike.engles
Sep 20, 2003
Hello

"I’m beginning to think that any test which even hints that a PC can stand on the same ground as a Mac is viewed as a virus by the Mac Illuminati."

Strange how I am beginning to think the same thoughts.
I cannot see how anyone can complain about a benchmark if all tests are carried out using the same script. In this case a action file for Photoshop. Unless of course they already have an idea of the weaknesses of the systems being tested.

Perhaps Apple did not use PSbench because they had a idea that Apple Machines to do not do this well.

Mike Engles
P
Phosphor
Sep 20, 2003
Also, remember the fact that the bench weights too much on the heavy filters, thus a normalizing might be needed. Having more accurate results might be a plus too!

I had an idea of an accurate selection of filters by using Mathias’ de-noising actions, or some stairstep interpolations, as Ellen did, on stock photoshop images. The use of a script might maybe give us data with smaller increments that 0.1 seconds… But if the test starts with subsequent resizes, it might alleviate that problem…

Thanks for the further tests, ellen!

It is normal that PS is always using scratch: the scratch is the PRIME memory of Photoshop, while the RAM IS the cache… confusing, but true…

You might check your preferences: what is the number of image cache, and the number of steps of the history? it might explain the gigantic sizes you get…

Once again, seeing the numbers you get, a 64 bits version of Photoshop might be useful in the (not so distant) future…
BC
bart.cross
Sep 20, 2003
The benchmarks only really tell you what computers are better with what filters. I’ve seen enough of these benchmarks to realize there is not one computer alone that excels in all parts of a benchmark, but using them can guide your decesion towards a computer purchase.

Still happy with my dual PIII 1Ghz, 2Gb RAM, ABIT VP6, WIN2K. All your benchmarks be damned, you will not spoil my happiness. 🙂
P
Phosphor
Sep 20, 2003
Actually, the G5 does quite well on PSBench — IF you do the benchmarking cleanly. (Which Apple did as part of their benchmarking)

But most of the WinTrolls aren’t being too clean in their testing.

ArsTechnica has some decent benchmarking going on with PSBench – but they changed the way the numbers are used to make it a MUCH fairer test.

(And Ho, there are so many holes in your logic I really don’t know where to start….)
ME
mike.engles
Sep 21, 2003
Hello Chris

What does "do the benchmarking cleanly mean"?
PSBench is just a script in which each test is done three times. The error can come if one is not careful or quick enough to take just the process time and not include the redraw time.
The final result is just an addition of the average of the results.

The other problem is chosing a image size so that it is big enough to give reasonable test numbers, without having to have Photoshop resort to heavy use of the scratch disk. It is necessary that the machines being tested have sufficient ram.

Everything else seems fair to me.

I always thought that your objections to PSBench was the image being used. It turns out that your objections are the non representative nature of some of the filters used. What does it matter if all the machines do the same test.

You say that the G5 does quite well at PSbench. What does that mean? It suggests that the G5 doing PSbench is not the fastest.

If it was the fastest then Apple would quote it. Apple quotes a benchmark that is flattering to the G5, but not the an IBM machine.
That is just marketing and hype.

That does not seem to me to be ‘clean’.

Mike Engles
P
Phosphor
Sep 21, 2003
Mike – I mean not crippling some machines, making the test as fair as possible.

PSBench may be the same – but the disks, memory config, energy saver settings, speed of the RAM, etc. may not be the same.

It matters because PSBench just sums the times. Again: PSBench might consider the faster machine slower just because it does ONE filter slower. That is sloppy benchmarking.

G5 running PSBench wins at all the common functions, and ties on a few of the less common filters. Who "wins" depends on how you interpret those results.

No, Apple’s benchmarks are not hype – they are VERY fair and very valid.

They don’t quote PSBench because it’s a suckful benchmark.
LH
Lawrence Hudetz
Sep 21, 2003
Again, when is a standard not a standard???

r_harvey: anyone using Microdol on Panatomic-x hasn’t the foggiest…..actually, the negs do look foggy!!
RH
r_harvey
Sep 21, 2003
r_harvey: anyone using Microdol on Panatomic-x hasn’t the foggiest…..actually, the negs do look foggy!!

Yeah, that experiment didn’t last long. I’d rather use sandpapery-ol’ Rodinal. I did a lot of tests, and ended up with a custom, handmade, slightly-watery variation of D-76. For Plus-X, I used Ethol UFG, and occasionally Acufine, but realized that I really should have had a bigger piece of film.
LH
Lawrence Hudetz
Sep 21, 2003
FG-7 or HC-110. Almost identical scale. I preferred FG-7 because HC-110 smelled like a tar pit! Grain very manageable, and development times pretty good.

Now, it’s almost exclusively color, like Velvia or the new Provia. I convert, and I have access to the most fantastic set of filters ever.
H
Ho
Sep 21, 2003
(And Ho, there are so many holes in your logic I really don’t know where to start….)

I always leave you an opening. I don’t know why you’re so reluctant to take it 🙂
RH
r_harvey
Sep 21, 2003
HC-110 smelled like a tar pit!

The best thing about HC-110 was the long shelf life. You could keep the syrup under the sink for months. I used it for years, when I was only doing a couple of rolls a week.

After too many years working with computers (writing things like libraries, benchmarks, utilities), and scanning prints, I’m trying to get back to the real stuff. It looks like the way for right now is good color negative film, scanned directly; if you need a quickie trophy-holding shot, use a digital camera.

…and I have access to the most fantastic set of filters ever.

While a Weston fan, I never was a purist. It seems crazy that if what you got on film didn’t work how you visualized it, that you should chuck it. I’m going through old slides and pulling stuff out of the shadows that should have been there all along. Hey, that’s what I saw, why shouldn’t it be on the print?
LH
Lawrence Hudetz
Sep 21, 2003
It was Ansel that previsualized his work. I don’t know that Weston ever did. But, he used one film, one developer, ant one paper(Azo)also developed in the same developer as the film (Amidol). I’m not absolutely sure that his film developer was Amidol, but I am sure the paper developer was. So, it was pretty easy to visualize the outcome.

I prefer the sensibilities of another AA expression. "The film is the score, the print is the performance". I know for a fact that AA’s prints varied vastly as the years went by. So previsualization may be an ideal, but it sure cuts into second guessing with improved materials. (Like digital)

My approach is to develop a heart connection with the subject, then strive to bring it out in the print. But even there, sometimes another feeling wants out and who am I to deny it?

Ho, I don’t see what Chris is talking about. Logic full of holes? What I perceive is an opinion couched in metaphor. Seeing logical holes in that is to miss the point entirely.

IMHO.
ME
mike.engles
Sep 21, 2003
Hello Chris

PsBench uses Photoshop to test the capabilites of computers. If the choice of filters are not to your taste, suggest some that are.

PSbench tests all the common functions.
The MacAddict test is quite clear on the conditions of the test and ran 5 common function tests. I ran the same tests om my Pentium on a image slighly bigger than used on the MacAddict test. My machine which is a Pentium IV 2.53 GHZ with 1GB ram and Xp pro produced figures not far off from the Dual G5 with 4 GB of ram.
So much so that one of the respondents to the thread could not believe my result.

Even in this thread my machine on the face of it beats the single processor G5.

I really do feel that you tend to show your colours a little to much in these benchmark wars. You are perfectly entitled to have your opinions and preferences, but I am not sure that a Adobe employee who works on both versions of Photoshop should be so obviously partisan.

It does make us Windows users(Wintrolls-your words) feel that we are not as important as Apple user.

This is the link to the Macaddict site for those who might be interested <http://www.macaddict.com/news/news_007.html>

These are my results for a 115MB image, synthesised by me.

Rotate 90CW————- .6 secs
CMYK ——————17.7 secs
Sharp200,5.0 ———–5.1 secs
Blur 25 —————-4.9 secs
Blur 1 —————–2.2 secs

I undid and purged between tests. I did these will writing this on line.

Mike Engles
RH
r_harvey
Sep 21, 2003
It was Ansel that previsualized his work. I don’t know that Weston ever did.

Adams formalized the process, but it’s the same thing right-brainers have done for thousands of years.

But, he used one film, one developer

Edward Weston <http://www.edward-weston.com/> was not a technical guy.

I know for a fact that AA’s prints varied vastly as the years went by.

I saw an exhibit of very late Ansel Adams prints, and they were really dark. It took a lot of lights in that place.

When Kodak Technical Pan first came out, I ran two test rolls, found it sharp and grainless, but way too contrasty. After quickie adjustments, I went to Big Sur and shot a dozen rolls. It was way too slow for handheld use, and the blown-out sky and sand left me with almost nothing printable. To top it off, a sea lion roared-up in front of me, and as I turned away, I tripped and cut my chin open on the rocks.

Five stitches. Nothing printable. I never used TechPan again.

Ho, I don’t see what Chris is talking about. Logic full of holes?

We develop our notions about benchmarks, and defend the notions… not the results. Even using the wrong filter in a test is adequate, if you use it in all the tests; I think Chris was concerned because he knew that some filters were highly-optimized, but apparently not the ones chosen for some benchmarks. It’s a test of the platform, I/O speed, and a group of instructions. You could get a reasonably good idea of how Photoshop would run on a platform just by timing some giant block moves. (Disregard the preceding; I’m trying not get into the debate.)
LH
Lawrence Hudetz
Sep 21, 2003
I have several hundred rolls of 6×6 Cm Tech Pan I have printed over the years, some very spectacular. I have one image that is a test bed of sorts. Unfortunately, it is damaged, and no longer scans well. Tech Pan is tameable. I always use a tripod.

Sorry about your injury.

I also am backing out of the test bed argu….uh, discussion. I would expect that Chris Cox and company have excellent facilities to come up with repeatable results of any platform. What bothers me is the comtinuation of name calling, recriminations, casting aspersions, etc. So unbecoming a process that is essentially an objective one.

We probably should take this film reminence to another thread.
RH
r_harvey
Sep 21, 2003
We probably should take this film reminence to another thread.

It’s much more fun to redirect an existing thread. At least our version has more to do with using Photoshop.
Y
YrbkMgr
Sep 21, 2003
<chuckle>
LH
Lawrence Hudetz
Sep 21, 2003
Ahhh, there’s Tony. Hi,ya Tony!

Just came back from riding my bike <sweating and puffing>. Well, subplot it is.

r_harvey, if you want to give Tech Pan another try, go to Photohrapher’s Formulary and purchase TD-3. I have used it for years. Careful with the agitation, however. Too much get streaks, too little and huge Eberhard effects. Do a 2 minute presoak, about 12 min development agitating 10 sec/min, and don’t agitate at all for the last 2 min.

It’s a bit fussy because it is so dilute. The provider suggests you make a stock solution then a final. If you use it infrequently, I find it better to purchase a 10mL pipette, draw 6.5 mL of solution A and 6.5 mL B and mix with 500cc distilled water (important!). If you do several rolls in a tank, a 25 mL graduate is a good investment.

Rinse first with water, then with dilute acetic acid, then fix. It fixes in about 30 seconds.

It’s important that you use filtered water throughout, and distilled water for presoak and developer.

The zone system is just about useless with Tech Pan. In fact, if you want to control contrast, exposure is a good tool. I always do two exposures; one "normal" and one +1 over. Underexposure is a killer.

Speed under this procedure is ISO 25, no matter what the flyer in the TD-3 box says. ISO 100 is a joke!

Cheers.

PS: Is this enough of a subplot? 😉
ME
mike.engles
Sep 21, 2003
Hello sub-plot chaps

Who can blame you. These benchmark things make sad cases of us all. What is so annoying are the very knowing,but unbacked assertions from Chris. He above all could settle the argument using PSbench. It does not matter what the tests are as long as all computers do them .

It is really like a steeplechase(fences)horse race. The first pass the post with the jockey still riding wins. It is pretty useless for a loosing owner to whine that his horse, aka his computer does not like waterjumps aka watercolor or high fences aka what ever test you like.

Horses for courses, as we say in the UK. Thoroughbreds vs Thoroughbreds. No whining when you loose. Do better next time.

Mike Engles
Y
YrbkMgr
Sep 21, 2003
Well, no one will like my opinion but, well, heck who cares right?

Chris owes us nothing in terms of an explanation. He can disagree with tests and protocols because he has the credentials with which to do it. It is not his obligation to educate us on how a proper test should be conducted.

Chris is the sort of guy who may voice an opinion about this or that, and in a way, stimulate you, the reader, to seek out the information and learn. I’m not saying that it is a style that I personally like, but it’s not for me to like nor dislike – that’s what it is.

Further, it’s my opinion that there are MANY times Chris has to walk a fine line. Revealing information in a public forum is precarious. His visits here are, in my opinion, a courtesy. To engage in a debate about how benchmarking should be performed where photoshop is concerned is futile – he’s not going to reveal all he knows just to satisify the curiousity of the forum; I wouldn’t. That’s an internal discussion at Adobe, for Adobe and is their sole province.

Anyone can create a test; that doesn’t make it accurate. So in the end, expecting Chris to elaborate on the subtleties of why a benchmark is not accurate is futile; if we were all in the same physics class, then yes, the teacher has an obligation to walk the student through how to conduct a proper experiement; and when there are dissenters, the instructor shows them the flaw in their logic. We all come from different backgrounds with different levels of education and it’s an impossible task to ask an Adobe Engineer to bring us all on the "same page" just so that we can have a meaningful debate about benchmarking.

Finally, the exercise of benchmarking is a guideline – it is not a race where there is a clear winner and loser. There is only one point to benchmarking and that is as a guideline. Having said that, I can’t imagine what all the fuss is about anyway – my car is faster than yours?

<donning flame retardant suite>

Peace,
Tony
ME
mike.engles
Sep 21, 2003
Hello

You have made fair points.The real problem is that it is quite obvious that Chris has a bias. If he as a Adobe person is not neutral than anything he has to say has to be a little suspect.

I am happy to be educated, but not to be given assertions as facts. The problems with these Apple /IBM comparisons is that Apples are a known quantity. They are made by Apple and to a common specification.
IBMs are infinite in their variety.

In the end the benchmarks as you say can only be a guide, but I feel that fastest doing a set of tasks is the best way of judging,-the horse racing analogy. It is up to the protagonists to optimise their machine as best as they can, and not complain when the results are not to their liking. It has to be said that horse racing is very prone to fixing as are computer benchmarks, but if the race is honest, first past the post is the best. One really cannot even out all the variations.

In the Macaddict tests, they, Macaddict were sent a supposedly optimised Apple machine, from Apple. The tests were common tests, nothing exotic. I assume that MacAddict just loaded a default Photoshop setup and did the tests. That is what we all do.
Yet the tests were disputed, because they did not match tests Apple had done themselves, with the same supposedly optimised machine.

Are Apple doing benchmarks on special machines, or ordinary off the production line ones? Did they send MacAddict a crippled machine?
Does not make sense.

Mike Engles
P
Phosphor
Sep 21, 2003
Chris doesn’t have a bias for hardware other than liking the fastest (but does have a small bias for OSes — I hate viruses and like Unix). I work with all systems, I write code for all systems, and I test all systems.

MacAddict made mistakes – that is obvious from their results. They should have contacted Adobe or Apple to find and correct the problem(s), but they didn’t. So now they’ve published really inaccurate numbers and just fed the windows trolls.

Apple’s tests were with near production and final production G5 machines, and production (Dell) Windows machines configured for maximum performance. I personally reproduced their results on my own systems, and double checked the configuration and results of Apple’s systems. (because, yeah, it almost sounded too good to be true) Apple could have gotten even better scores if they let the operations hit the hard disk (because OS X is getting much better disk transfer rates than Windows), or if they cherry picked the results. But they didn’t. They went for something fair and representative of real users work.

I really don’t know what mistakes other people are making in benchmarking (other than trusting PSBench as-is — and I’ve already told you exactly what’s wrong with it), but I do wish they’d bother to ASK rather than publish and retract….
RH
r_harvey
Sep 21, 2003
… if you want to control contrast, exposure is a good tool. I always do two exposures; one "normal" and one +1 over. Underexposure is a killer.

So, you have to under-develop it to make it look anything like continuous tone film–and the scale is still a couple of zones shy. With normal development, it turns into Kodalith, but with pickier processing. This is pretty much how I remember TechPan… it sounds tough to zero-in if your water temperature varies by 30-degrees.

After it cools off here (106-degrees today), I’ll probably settle on color negative, and process-only at the local CostCo (sigh).

PS: Is this enough of a subplot?

It could use a spirit coyote.

but does have a small bias for OSes — I hate viruses and like Unix

That’s so subtle, I can’t imagine what’s missing. Could it be… evil empires?

After we have had the perspective of enough benchmark tests, the results will likely be much as predicted from architecture comparisons: faster busses and shorter pipelines will win–physics is an unforgiving master. Processors with regular instruction sets will have better compilers; others, like Itanium, will never reach their potentials.
Y
YrbkMgr
Sep 22, 2003
Mike,

I understand fully what you mean, but taking emotion out of it (where I’m concerned),

The real problem is that it is quite obvious that Chris has a bias.

He *may* have a bias, but he’s not stupid and would not intentionally share his bias in a public forum. It isn’t a stretch to think that Chris could get in a lot of trouble for what he does or does not say in the forum. He still has upper level management to contend with. Chris is in a pretty visible position, and as an engineer/programmer/grand poobah, whatever, IMO, he shares the facts as he knows them, and can back them up if questioned by management.

So my point is, we all have biases, but for me, and me only, if Chris says that results are suspect, I’m damn well not going to disagree with a "that’s your opinion". He doesn’t think like that – he’s a scientist first, works for Adobe second.

I think his point about companies sharing their results with Adobe for discussion/validation is a reasonable expection and those that *don’t* share them may be the ones with the bias.

Chris,

I may be all washed up in my synopsis here, and apologize if I’m putting words in your mouth or incorrectly characterizing things. I’ve been in big companies where high level technical folks (Ph.D’s) have an opportunity to interact with the market. In my experience it can be rather precarious.

Peace,
Tony
P
Phosphor
Sep 22, 2003
Tony – no, you’ve got most of it right.

There are a LOT of limits on what I can say, endorse, etc. (and I still get in trouble every once in a while)
LH
Lawrence Hudetz
Sep 22, 2003
Insults don’t become the scientific process. Windows trolls?

Look, a scientist first means that, given a choice between corporate management and the truth, the truth should out, without reservation. If corporate muzzles, it needs to be complete, not allowing half truths or partial truths. Those kinds are odious, if not down right dangerous.

Ooops, I wasn’t getting into this……………!

R_harvey, exactly right. Tech Pan is a Litho film, but one that EK recognized as having valuable attributes in "under-development" Under- development with respect to what? No all lithos will respond like Tech Pan. I consider the development recommended as optimum for myself, and is not underdeveloped.

Back to standards, I guess!
Y
YrbkMgr
Sep 22, 2003
Insults don’t become the scientific process. Windows trolls?

Then you obviously have never seen a Ph.D. whose work was criticized by those less than qualified to judge. But no one mentioned anything about the scientific process.

Look, a scientist first means that, given a choice between corporate management and the truth, the truth should out, without reservation. If corporate muzzles, it needs to be complete, not allowing for half truths or partial truths. Those kinds are odious, if not down right dangerous

Lawrence, I respect you and all, but I’m having a hard time seeing where you’re coming from. There has been no mention of corporate muzzles where the truth is concerned.

Consider this scenario that I am making up. Folks want to compare which platform is performing "photoshop better". Chris knows all of the reasonably credible tests that have been publishes, and studied their methods, both, on paper and in-house. Chris finds a handful of folks who obviously don’t have a fair appreciation for how photoshop works, and therefore how to set up a fair test.

One of these folks goes ahead and publishes information that Chris knows to be false because of the test criteria used, as well as interpretation of the data.

There is no corporate muzzle on saying that the test is bunk. But it’s not unreasonable to think that if he were to expand on the resons why, in an intelligent and plausable way, that he would be revealing more information to the public than is covered in his NDA. The corporate rule is: When in doubt – shut up. Otherwise, you have to check with Lawyer Tom from Adobe to find out if you can clarify a point about a stupid benchmark test.

Add to that the fact that really, when folks publish data that is directly opposite of what you know to be true, hell, I’d say it was bunk – and if those folks don’t listen to reason, I might even call them trolls with a hidden adgenda. Who’s to say?

No, I don’t think there’s been any issue of a corporate muzzle where the truth is concerned, rather, that Chris has chosen not to expand on his reasons, nor engage in a debate about the test – for whatever reasons, personal or corporate.

I’d bet there aren’t a lot of folks, visiting this forum, who have a significant amount of experience in scientific test methods, and interpretation and presentation of data. I know there are some, but not a lot. Why would he educate folks on that? That’s not his role. The only reason he visits the forum is to find out what’s happening with his baby, and help now and then when it doesn’t take too much typing.

I’m sorry if I sound like a snot, I’m not trying to be – it’s just that my view is Chris doesn’t owe us squat, and we get WAY more here than we do in most forums.

Peace,
Tony
LH
Lawrence Hudetz
Sep 22, 2003
Tony, yes, I have experienced PhD’s (and lesser degrees) getting uptight about criticism from a location which does not possess the credentials to criticize that particular thesis. Working for a nuclear facility exposes one to many PhD’s! Four letter expletives abound. However, in public response, the rhetoric is toned down, and anything from a simple dismissal to a pointed response to the paper at hand is usually seen. Personal attacks, even veiled ones are really frowned on, at least, that was true then (ca 1960).

Let me go on record to say that I completely agree with you on Chris’ position. I expect that he does have the best information available, simply because of the quality nature of Adobe products I have used. You don’t get there by sloppy, unexamined premises. So, when Chris speaks, I listen, and also will ask for more if what I read is insufficient. If I only knew 10% of what he does in his field of expertise!

So Far as corporate muzzle, this is his statement:

"There are a LOT of limits on what I can say, endorse, etc. (and I still get in trouble every once in a while)"

That constitutes muzzling in my book, and is certainly proper. So, my complaint is, besides name calling etc, the existence of partial or half truths is worse than no information at all, IMHO that is. Others may properly disagree.

You are right, Chris and others in Adobe do not owe us squat, unless they choose to respond. Then, they owe us the truth in the matter. Hidden agenda may abound in the PC world by folks who have an axe to grind, but partial truths suggest a similar situation so far as the responder’s bias is concerned. What’s being hidden?

Tony, you are not being snotty. I highly respect your views and I am closer than you may think to yours. Expertise in bench marking is probably not a strong suit for more than a few people, including moi. The word "benchmark" should give pause to anyone venturing into that realm. I have run two on my computer, mostly to detect any substandard devices on my machine. I ran two so that I have a point of departure to examine. Both suites gave similar enough answers to satisfy my reasons to look, but no way would I publish the results as any indication of my computer’s capability to run any program.

Ahh, well, here I am again…..

I got to go to a different thread. I got a strange thing going on in PS6 I can’t figure out…..
ME
mike.engles
Sep 22, 2003
Hello

I still refer to the original Macaddict review of the Apple G5 sent to them by Apple. Chris disputes these numbers. He says that they are wrong. How wrong? Several magnitudes? Why? Macaddict took a computer sent to them by Apple, loaded on Photoshop and did five common tests.

They say were trying to do the benchmarks someone would do having just opened the box. So they asked Apple send they 512 ram so that they could emulate a basic machine. Then they did tests on the same machine using the full 2 GB sent by Apple originally. It is this configuration that my modest Pentium seem to match pretty well.

How wrong can anyone get doing something so simple?
It is not in Macaddicts interest to misrepresent Apple.

That is all I am asking.

Mike Engles
H
Ho
Sep 22, 2003
This is basically a no win situation for any Windows user. If your Win machine performs comparably to a Mac, then there is something flawed about the comparison, or the Mac is flawed, or both.

At least this is the attitude expressed here by someone who is in the position to create a fair comparison, yet restricts himself to snide comments about the testing methods created by others, and about the users of those methods. Offering only critisisms but no solutions is not constructive and serves no purpose. It appears he wishes to trade on his credentials alone, offering up opinion that we are to accept as fact… because he said so. I don’t see the logic in this.

Others have tried to defend this stance, and I have to take issue with that defense as well. I do not see how helping the users of computers of competing platforms establish an accurate baseline can be a problem for the suits. Of course, one could speculate that Adobe and Apple are in cahoots to tarnish PCs at any opportunity, and certainly, rigged demos are not unknown. I think, rather, that this is a personal issue and I am quite sure it will not be resolved here; no meaningful answers will be forthcoming.

In this venue at least, this horse is dead and will continue to be beaten only for the sheer pleasure of it.
ED
ellen devito
Sep 22, 2003
look, I’m no scientist, i’m no computer engineer. but there’s one simple reason some of us like PS7 Bench.

This is ALL ABOUT CONTROL.

in my humble opinion, what Chris Cox et al dislike is the fact that PS7 Bench allows YOU AND ME, the humble user, to do benchmarks for OURSELVES. on our OWN SYSTEMS! heaven forbid.

see, this way, they can’t send a "perfectly tuned" system to MacAddict.

and its much harder to "Debunk" or claim "phony benchmarks" if someone says they did a benchmark at HOME on their OWN computer. We all know our own personal systems the best, yet we’re supposed to DEPEND on "gurus" and experts from Adobe, Apple, MacAddict, etc etc to TELL US what is "the best" on their own mysterious configurations. absurd.

this is popular revolt, pure and simple!

as customers, we want to do testing for ourselves, testing WE CAN ACTUALLY UNDERSTAND and DO OURSELVES on OUR OWN RIGS, and this SCARES the YOU KNOW WHAT of all these corprate types who have limits on what they can/can’t say.

Apple has EXCLUSIVELY tied its marketing campaigns to Adobe Photoshop and Adobe products to the exclusion of many OTHER software titles.

for years it has relied on a "tradition" of being "the best" in PS. for an Adobe engineer NOT to continue to keep this tradition up would be more suspicious. why else then was the "PC Preferred" section on the Adobe website removed so magically fast? Adobe can’t even dare to claim it prefers PC’s for Premiere and After Effects, so forget about "holy" Photoshop losing its crown.

If PS7 Bench is really so horribly flawed (using PS’s OWN timing feature, oh, can you smell the IRONY?) -then suggest ways we can make it better.

That’s being fair, and treating your public like an intelligent ADULT that can do things for themselves.
ME
mike.engles
Sep 22, 2003
Hello

It has to be said that the MacAddict results are really stupid. I also I feel that there is nothing wrong with PSbench, as long as it is fair. I really cannot get round all this normalised, statistical stuff, for something that is not rocket science.

If Apple just published those results, ordinary users would accept them. A computer is a box that runs processes, which can be timed.
Ordinary users will use those as a base, as long as certain basic settings are stated and adhered to.

Mike Engles
LH
Lawrence Hudetz
Sep 22, 2003
Normalization is SOP, and is not to be feared or obsessed over.

You are right Mike. As I said earlier, tests which establish standards are blind to labels. You have a black box. You put in a Standard input, measure the output and draw your conclusions.

That is what we really should do here. The boxes are in the next room. For each test, a platform is picked at random, out of sight, and the operating conditions established. All you know is A, B, C, etc for the Device Under Test. In fact, for most testing protocols this is exactly what you get. When looking at the setup, you see a little box on the schematic with wires running here and there, and the little box is labelled: "D.U.T". All you know is the device from a generic point of view. Run the test, then, Tada! we find out who did what.

Ellen, I know just where you are coming from, but I can tell you that testing is a quagmire. Be wary. It sometimes takes months to come up with the schematic with the little DUT box that you Know for Certain what the results will be like. And this being true for simple devices like laser diodes, how much more so for a complex device like a computer? But, hey, keep it up. It’s interesting to get tripped up by an amateur. Keeps the cognoscenti humble. 😉
RH
r_harvey
Sep 23, 2003
No all lithos will respond like Tech Pan.

Tech Pan feels like Kodalith. Nobody where I used to work liked Kodalith, because it was expensive and too controllable–you could get almost anything from it if you understood exposure and development; so much silver that it seemed infinitely black (I’ve used it for continuous tone). They liked some cheap stuff (I’ve forgotten the brand) that had almost no emulsion–you could develop it with the lights on, and it’d just barely get black–and it either had the image, or not; develop it until it goes black (or dark green). Fuji litho film was a little better, but you had to understand it… and it didn’t have the range of Kodalith. All of these films are around ASA 3-6.

Personal attacks, even veiled ones are really frowned on, at least, that was true then (ca 1960).

That was true of society in general, in 1960.

It looks like the new generation of processors (excepting P-4 derivatives) will, given similar configuration, perform within +/- 10% of one another; there is a lot of cross-licensing of technology, and they’ll all soon enough be hamstrung by memory, anyway. If you want to invest in the best components, you’ll be on the high-side, regardless of the CPU; or save a few bucks, and have a life.

I think the sub-plot had more mileage left, but it’s a few months early for the main plot to have any solid data to support it much longer.
Y
YrbkMgr
Sep 23, 2003
tests which establish standards are blind to labels. You have a black box. You put in a Standard input, measure the output and draw your conclusions.

Sorry – nope. You design a test based on the the system being tested and control for variables – What is the length of water?
LH
Lawrence Hudetz
Sep 23, 2003
Huh?? Mixing metaphors it seems. Or is this a Zen question?

Double blind, black box testing is the norm where I come from. You don’t think this applies to computers?
Y
YrbkMgr
Sep 23, 2003
What I mean to suggest is that when you design a test, you must understand first what you are testing and design a system that accounts for variables in order to control it. You cannot measure the length of water because water is not condusive to linear measure. So it goes that just because someone designed a test does not mean it accommodates the the system being tested. Would you use Volts to measure computer speed?

So then it begs the next question – what DOES one measure? What are the variables? Are the variables controlled for?

A real scientific test, say, in a chemical system, begins with running standards of known purity in varying dilutions and establishing a concentration curve. Then the unknown is measured in that system and the concentration is read off the curve. In some chemical systems the concentration is linear only to a point, then the curve plateaus. Then mathematics have to be applied in order to "linearize" the curve – but if you do that, what happens to the part that was linear at the lower concentrations?

Further, the system you are using to measure these concentrations must be calibrated according to adopted standard laboratory techniques. You must account for "drift" in the equipment testing – for example spectophotometric measurements.

My point is, unless you understand the system and variables, you cannot create an accurate test. You can create a test, sure; but does it measure what you’re trying to measure? How do you know?
RH
r_harvey
Sep 23, 2003
Like when trying to benchmark the Pentium Pro with 16-bit tests, we’re in a transition time. It’s a little early to call winners and losers.

No sense crying over spilt water.
H
Ho
Sep 23, 2003
My point is, unless you understand the system and variables, you cannot create an accurate test.

Precisely the point. The one person in this thread who has the authority to disparage the PS Bench (for his own reasons, as yet unsubstantiated with any data) and also the wherewithal to create a valid benchmark simply thumbs his nose at the proceedings and goes away. This leaves the benchmarkers to their own devices and I wish them well.

<<OOPS>> Tangential Thought Alert: you know, it’s the users (perhaps more so than the programmers) who have really spurred the development of Photoshop… the people who took the software and did things with it that the creators never imagined are the ones who, in my opinion, started the big ball rolling that eventually made PS into what it is today. I say give them their benchmark.
Y
YrbkMgr
Sep 23, 2003
The one person in this thread who has the authority to disparage the PS Bench (for his own reasons, as yet unsubstantiated with any data) and also the wherewithal to create a valid benchmark simply thumbs his nose at the proceedings and goes away

You make a fair point Ho, in as much as I too would like to hear what Chris has to say on the subject. But you DO realize that when or if he spills, 18 guys and an old lady with a fake fur are going to challenge his logic – it’s a "no win" for him. Note the conspicuous absence?

If I were in his shoes, I’d shut up about now too.

r_Harvey,

Like when trying to benchmark the Pentium Pro with 16-bit tests,

In your typical style, few words, but the point speaks volumes. Couldn’t agree more.
RH
r_harvey
Sep 23, 2003
My point is, unless you understand the system and variables, you cannot create an accurate test.

If you do understand, you cook the test to match any results the person who paid for the test would like. Benchmarks are adjustable, surprisingly so, by relatively small changes.

also the wherewithal to create a valid benchmark

Quite a different task than making Photoshop work as well as possible on disparate hardware. Why not test is to see who gets done first, feels better about the process, would like to do it again, doesn’t hate the computer because it’s full of spyware/viruses/bugs…
DM
dave milbut
Sep 23, 2003
I say give them their benchmark.

"Let them eat cake."

(sorry, couldn’t resist.)
Y
YrbkMgr
Sep 23, 2003
<encino man> Ohhhh Dave </encino man>

btw, happy birthday dave.
LH
Lawrence Hudetz
Sep 23, 2003
Tony as usual we agree more than disagree. I don’t understand how you misread me, or are your comments for others? Analog systems are notorious for nonlinearity. I know and have labored mightily with them, especially feedback systems. Here we have digital, and a different set of considerations, but the basics still hold true.

How amy folks out there are familiar with the Church-Turing Thesis concerning the computer stopping and giving an answer?

Anyway, off to a town meeting. I get to dabble in politics. 😉
DM
dave milbut
Sep 23, 2003
Thanks Tone!

Pax!
dave
LH
Lawrence Hudetz
Sep 23, 2003
Happy birthday from moi, as well, dave. I just celebrated my 66th last Wednesday. Fun times. 🙂
Y
YrbkMgr
Sep 23, 2003
Lawrence,

I don’t understand how you misread me, or are your comments for others?

Mostly I the latter. Earlier I resisted the temptation to try and explain how to set up an experiment because as my father once said: Never argue with an idiot in public – people may not be able to tell the difference. I’m not calling anyone an idiot, just that my frustration in explaining about the scientific process is kind of what Chris would have to go through.

You know, I think, how much I respect you, and your opinion. I took the opportunity to pontificate on setting up an experiment based on the fact that I knew you would know what I meant, and others could follow along. In fact, I kind of goaded you into it with the "how high is water" comment.

So, I apologize (to some extent) to have to make a point at your expense knowing that you are pretty aware of the scientific process.

Lastly, I would only add this. Most people think "what’s the big deal?" Run it on my system at home, run it on my system at work, which one is faster? Clear winner right? But in point of fact, that is a casual experiment, for fun. Not an experiment you would go ahead and submit to a trade journal for publication – it’d be torn to shreds. So I am making a distinction between the right way to conduct an experiment, and a casual way – IMO, this is really where Chris is coming from.

Btw, happy belated birthday to you too Lawrence.

Peace,
Tony
LH
Lawrence Hudetz
Sep 23, 2003
Tony, Tony, Tony,

Sure know how to set your hook in me! You would think after all the water under the bridge, I would actually learn something!

Oh, well, I enjoy it and maybe some day I’ll catch on.

But then how would we have fun? 😉

Thanks for the vote of confidence guy!

Peace always

Larry
DM
dave milbut
Sep 23, 2003
Thank you Larry. At 38 I feel like I’m approaching the dark side of the moon! 🙂
J
JasonSmith
Sep 23, 2003
LH
Lawrence Hudetz
Sep 23, 2003
38! Jeeze, you are a young un! The same age as my youngest son, who, by the way, works for Autodesk. You both give me computer advice. Sort of.

Let’s see. 38 is a pistol caliber. 66 is the old highway from Chicago to L.A.

I like 66. Get your kicks on US 66. It was my dream road as a kid. I lived at the start of US 66 in Lyons IL. where it joins US 34 into Chicago. 34 wasn’t bad either. It goes to Estes Park Colorado, in the Rockies.
Ah, youth. Dark side my foot!;-)

Have a good year, dave
PH
Photo Help
Sep 23, 2003
Jason,

Cute. What will it be when it grows up? 🙂
DM
dave milbut
Sep 23, 2003
ph, that’s an image from the terascale project… supercomputer out of g5s:

dave milbut "What 1,100 Macs look like – Terascale project photos" 9/23/03 9:25am </cgi-bin/webx?13/0>
K
Klaas
Sep 24, 2003
What’s with everyone having brthdays in late September? (Mine was last Friday.)

Could it be New Year’s celebrations? 😛
T
Terrat
Sep 24, 2003
When consumers talk about technology and controlling it…it’s usually time to meet up with the future again…like the boogieman under the bed. I know it’s superstitious to think this way.

The costs of matching and learning about new softwares and peripherals to match up with the different systems and new hardwares when Windows 3 went to Fats 32? Or when Mac classics went to the G4? Don’t talk about it; or it will happen.

This would mean that the basic Windows is now going to change for all Windows users….Say, "Fats 64". Yikes!
LH
Lawrence Hudetz
Sep 24, 2003
Well, the 64 bit AMD’s are being deliverded as we speak. Unlike Mac, it won’t run 32 bit programs.

It’s the domino effect, like "Fats Domino".
P
Phosphor
Sep 24, 2003
"it won’t run 32 bit programs."

You sure, I just saw an article on cnn.com about this. they said that it runs 32-bit programs beautifully.
DM
dave milbut
Sep 24, 2003
yes it will larry.

AMD’s Athlon steps up to 64 bits <http://zdnet.com.com/2100-1103-5080217.html>

"Intel’s current Pentium chips and those expected in the near future offer higher clock speeds but stick with 32 bits. In contrast, the Athlon 64 can run both 32-bit and 64-bit software. And it’s that ability to run both types of software at the same time that AMD is counting on to tempt buyers."
LH
Lawrence Hudetz
Sep 24, 2003
Right. I don’t know where I got the info, but I read another article which detailed the AMD.

Sorry for the mis-info!:-(

Now, here’s a link to a G5 test: <http://www.pcmag.com/article2/0,4149,1274138,00.asp>

They ran Photoshop 7 as one of the app. See what you all think of it.
P
Phosphor
Sep 25, 2003
Larry – they did something wrong with the Photoshop performance test.

The G5 should have run circles around x86 chips.
(because it does in all the valid tests I know of and all the ones I’ve done myself)
LH
Lawrence Hudetz
Sep 25, 2003
Chris, that amazed me also. I didn’t want to say anything at the outset, but I sure arched an eyebrow. Does the test have any actual hard information of value? I did a quick review and went to the conclusion and then set up the link, so I haven’t carefully looked.

???!!!
P
Phosphor
Sep 25, 2003
Lawrence – I didn’t find enough information in their article to determine what they did or how they did it wrong…
P
Phosphor
Sep 25, 2003
I like how they conclude that the new G5 is ‘just as good’ as the fastest PCs…

nothing more, nothing less.
LH
Lawrence Hudetz
Sep 25, 2003
I didn’t think so.
Here we go again………….
ME
mike.engles
Sep 25, 2003
Hello

I just did a experiment. I made a 200 MB file by resizing ‘peppers.jpg’ to 200 MB. I ran watercolour using the dialogue. The time to process was 94 secs. I undid and purged, then ran the filter as a repeat. The time was 70 secs.

The times for Ocean Ripple were 99 and 78 secs.
The times for lighting effects was 25 and 8 secs. I did some fiddling in the dialogue. that seems to be added to the process time.

This happens in Winxp pro with Pentium 2.6 and 1gb ram.
I don’t know if it happens on Apple. Also in the case of Ocean Ripple, some caching is done to the hard disk, before the process indicator shows.

Mike Engles
P
Phosphor
Sep 25, 2003
Without knowing the exact image dimensions, preferences settings, and filter settings you used, nobody can reproduce your results. They’re meaningless.

And why are you using obscure special effects filters to compare performance instead of commonly used functions? (resize, rotate, open, save, GBlur, USM, RGB<->CMYK, etc.)
ME
mike.engles
Sep 25, 2003
Hello Chris

I was not using them as benchmarks for anyone. I was asking if Apple Photoshop added the filter dialogue open time to the process time, like Windows Photoshop.
That is why I was quoting two times, one using the dialogue and the other as running the filter as repeat.
The PCmag review of G5s said that it took a long time for Windows Photoshop to load the ‘controls’. I was wondering if this is what they meant. The review also implied that Apple Photoshop did not take time to load the ‘controls’.

Also, why is the dialogue open time included in the process time, or to put it simply why two different times for a filter?

Mike Engles
P
Phosphor
Sep 26, 2003
Mike — my mistake.

Yes, it’s the same on both platforms.

Because the dialog can open during the processing time (some filters show the dialog in the parameters phase, some show it during the filter/Start&Continue phase).

The time is measured from the command execution (the filter is first called) until the filter is done and returns control to Photoshop.
SM
sam m brown
Sep 26, 2003
Ok a quick question regarding using an action for benchmarking/testing different machines – Don’t want to get into the windows vs mac debate that these questions always lead on to…

If I was to set up an action to time – lets ignore what it includes as in my case would be a real world action/group of actions that we use so therefore it would be process’ directly related to what we use.

The question is using the playback modes – accelerated vs. step by step. Am I correct in assuming if you were going to take into account the graphics card performance then you would have to run the action in the step by step mode and if you were to run the action as accelerated then your graphics card would have less of a baring on the test.

Sorry if this seems like a simple/irellevent question.

Cheers,

Sam.
P
Phosphor
Sep 26, 2003
Sam – yes, that is correct.
P
Phosphor
Sep 26, 2003
This discussion is more and more constructive, thanks Mike and Sam for the intersting questions, and Chris for the answers!

Chris, would it require lot of works to give more accuracy to Photoshop’s timing procedure? (record in 1/100ths of seconds, for instance.) (keeping in mind that benchmarks are not just about marketing, or bragging rights related, and are also a good way to check one’s system…)

Also, the playback options seems to be general, not action specific, thus I guess that you will include a change of Playback Options step to make sure they will go at full speed, Sam…
P
Phosphor
Sep 27, 2003
Pierre – not much work, we just haven’t had a great reason to do it.

And you can get a lot more accuracy using a scripting system (JavaScript).
RH
r_harvey
Sep 27, 2003
There’s so much uncontrollable chatter going on in Windows that any short timing test will vary wildly. Try the SI benchmark in Norton Utilities for DOS running in a DOS box, and watch the numbers jump all over; that’s a very short test. High-resolution event timing in Windows is actually documented! But it adds overhead. Or you can just keep track of the start and stop time–which has no overhead, but low-resolution.

Extending a test over larger data sets or more iterations, will ensure greater repeatability than a software-based timer.
P
Phosphor
Sep 27, 2003
Harvey – yes, I know.

That’s part of why I tend to repeat my tests at least 5 times. The other part is looking for trends: does the time decrease or increase with iterations (sometimes indicating a problem).
RH
r_harvey
Sep 27, 2003
yes, I know.

I was talkin’ to the crowd.
P
Phosphor
Sep 27, 2003
"SI benchmark in Norton Utilities"

Wow, you’re relying on Norton’s? Bad mistake. ( at least for the Mac side)
RH
r_harvey
Sep 27, 2003
Wow, you’re relying on Norton’s? Bad mistake.

Yes, I’m using Norton Utilities for DOS, from 1992. Why, is there a problem?
Y
YrbkMgr
Sep 27, 2003
I’m using Norton Utilities for DOS, from 1992

Those were the days… when it was Peter Norton and Central Point Software (PC Tools). Once Symantec got ahold of NU, well, it’s never been the same.
RH
r_harvey
Sep 27, 2003
Norton’s okay, I guess, but I like Mace Utilities with Vopt. It has that counter-cultural, bearded-hippie appeal, a command line interface, and their Swiss Army knife logo is so modern.
Y
YrbkMgr
Sep 27, 2003
Well, back in the day, I only used PC Tools (Central Point Software). Symantec bought both Norton and CPS. I ran NU for a while, but don’t run it now at all, except AV. Too much overhead.
RH
r_harvey
Sep 28, 2003
Peter Norton gave up his PC Magazine column, sold his company to Symmantec, and retired in his 40s to a giant antique wooden house in the northeast. Once every eighteen months, he stands there with his arms folded, and they take his picture for the new Norton Utilities boxes. I guess polishing all that antique oak keeps him busy.
Y
YrbkMgr
Sep 28, 2003
Lucky guy.
PH
Photo Help
Sep 28, 2003
Tony,

Never install NU, the best part about it is the stand alone utilities. The utilities haven’t changed much since the days of DOS. They still offer very good command line utilities. The big difference now is that everyone is using usb memory drives and CD’s now not to mention better hard drives so we don’t have as much of a need for disk utilities now. That is of course with the exception of Ghost. Anyone that has any doubts about how powerful Ghost is hasn’t used it or didn’t understand it if they did.
PC
Pierre_Courtejoie
Nov 1, 2003
Chris, Is there something wrong in Mr Schuette’s test protocol? : <http://www.lostcircuits.com/memory/2gb/8.shtml>

EDIT: He forgot to put the history to 1 step as we can see in his forum:

< http://www.lostcircuits.com/discus/messages/75/1781.html?106 7546741>

but still there is a difference in the stopwatch VS PS’s timer…
CC
Chris_Cox
Nov 3, 2003
Pierre – I don’t know. I’ll have to take a look at the stopwatch times and see what’s going on (I never use it – I use a stopwatch or a script with timers).
PC
Pierre_Courtejoie
Nov 5, 2003
Thank you! It might explain certain oddities, and throw some (more) discredit on the validity of PSBENCH as a valuable benchmarking tool…

Are those script under NDA? I don’t want to beat a dead horse, but having a valid tool to troubleshoot/compare machines could be very useful! (you could even log on studio Exchange under another name to post it <G,D&R> )
CC
Chris_Cox
Nov 9, 2003
The scripts (and tools) we use internally are not going to be released to end users. Sorry.

Yes, I’d like to have something better for comparisions and testing. But I have no idea when or if we might release something like that.

Master Retouching Hair

Learn how to rescue details, remove flyaways, add volume, and enhance the definition of hair in any photo. We break down every tool and technique in Photoshop to get picture-perfect hair, every time.

Related Discussion Topics

Nice and short text about related topics in discussion sections