PDA

View Full Version : Reviews seem inconsistent, what am I missing?



rhodeislandguy
12-14-2007, 09:50 PM
Maybe I'm misreading something somewhere, but the dcresource review comparisons of the a700 seem inconsistent as to the camera's low light performance.

The Sony a700 review says (in part):
At the highest ISO settings, I think the Canon EOS-40D does slightly better than the A700, though shooting in RAW could negate this.

The Nikon D300 review says (in part):
NIght shots are hard to compare (since conditions are never consistent), but I'm thinking that the D300 did better than the Canon EOS-40D in this test. Both the D300 and the 40D wiped the floor with the Sony Alpha DSLR-A700, which isn't so hot in low light at high sensitivities.

"Slightly better" and "wiped the floor" are not equivalent in my book. Which is it?

DonSchap
12-15-2007, 08:23 AM
The fact is that any of the three manufacturers are now a far cry from the earlier releases. The low light capability between the current three is nit-picking free-for-all, so you can get one or the other or the other ... and probably do just fine in your choice.

For argument sake ... you can probably assume its a level playing field, at the moment. They are all REAL good for what they can now do.

What you need to consider is what you are planning on doing with the system you select and what system you may already have legacy hardware to use with it.

Canon and Nikon are huge ... and have a quite a variety of lenses and items to select from.

The SONY has in-the-body-Image-Stability ... and that is the big attraction for low-light shooting, because you can possibly leave the shutter open a bit longer with ANY lens you put on the front of it and not get "shake." Otherwise, with the other two ... you would need a lens equipped with image-stability and that represents added cost.

Once ...


sTaBiLiTty

31620


... is built-in the camera, it is "FREE" from that day on, no matter what lens or whatever moves ya. ;) (Makes perfect sense to me.)

Flip a coin ... then flip again! :cool:

cdifoto
12-15-2007, 09:28 AM
The fact is that any of the three manufacturers are now a far cry from the earlier releases. The low light capability between the current three is nit-picking free-for-all, so you can get one or the other or the other ... and probably do just fine in your choice.

I can pretty much go along with this.

erichlund
12-15-2007, 10:01 AM
Maybe I'm misreading something somewhere, but the dcresource review comparisons of the a700 seem inconsistent as to the camera's low light performance.

The Sony a700 review says (in part):
At the highest ISO settings, I think the Canon EOS-40D does slightly better than the A700, though shooting in RAW could negate this.

The Nikon D300 review says (in part):
NIght shots are hard to compare (since conditions are never consistent), but I'm thinking that the D300 did better than the Canon EOS-40D in this test. Both the D300 and the 40D wiped the floor with the Sony Alpha DSLR-A700, which isn't so hot in low light at high sensitivities.

"Slightly better" and "wiped the floor" are not equivalent in my book. Which is it?

If you have the latest internet exploder (7) or another model with tabbed windows, open all three reviews side by side and look at the night photos at ISO 3200. I think Jeff was trying to be "diplomatic" during the A700 review. Frankly, there's a huge difference, both in noise and color fidelity.

This does not mean the A700 could not do better. It means that on this test, when you look at all three side by side, the A700 comes up well short.

Shooting RAW and with good exposure and post processing technique, I think you could still use the A700s ISO 6400, where Jeff says leave it alone. Shooting Nikon, I've never had to deal with the amount of chroma noise I see in Jeff's shots with the A700, so I don't know how difficult it is to deal with. OTOH, Canon's have traditionally had more chroma noise where they did have noise, and they clean up well, so I suspect you would be OK.

Jeff Keller
12-15-2007, 11:23 AM
I have reworded that paragraph a bit to make it a little less offensive to A700 owners :D

themitty
12-15-2007, 12:04 PM
I really have to question the validity of this "test". It seems you're trying to use it as a (at least relatively) scientific test, but there are way too many variables that are different from test to test.

For example, the level of light on a given night can vary greatly (you even mention this in your review), along with air conditions. Second, they are taken at slightly different angles, and finally, there's a huge difference in aperture/shutter speed on these shots.

For the a700, you used iso3200, 0.2s, f5.6, while the d300 was at iso3200, 0.76s, f/13. Can you explain why you'd do this?

What's the point of a test if you have so many variables - you're supposed to keep everything constant, and just change one parameter (the camera in this case).

Not to mention, that no one would actually shoot high ISO for this type of scene - you'd use a low ISO and a long shutter speed, along with a tripod, or something to steady the camera.

Given that the atmospheric conditions are never going to be constant, what's the point of this test? Why not use a controlled room with constant lighting, and photograph something with fine texture, text, etc to really compare low light levels and high ISO? Seems that would be much easier to keep consistent, and be a much more accurate reflection of high ISO capabilities.

cdifoto
12-15-2007, 12:23 PM
I really have to question the validity of this "test". It seems you're trying to use it as a (at least relatively) scientific test, but there are way too many variables that are different from test to test.

For example, the level of light on a given night can vary greatly (you even mention this in your review), along with air conditions. Second, they are taken at slightly different angles, and finally, there's a huge difference in aperture/shutter speed on these shots.

For the a700, you used iso3200, 0.2s, f5.6, while the d300 was at iso3200, 0.76s, f/13. Can you explain why you'd do this?

What's the point of a test if you have so many variables - you're supposed to keep everything constant, and just change one parameter (the camera in this case).

Not to mention, that no one would actually shoot high ISO for this type of scene - you'd use a low ISO and a long shutter speed, along with a tripod, or something to steady the camera.

Given that the atmospheric conditions are never going to be constant, what's the point of this test? Why not use a controlled room with constant lighting, and photograph something with fine texture, text, etc to really compare low light levels and high ISO? Seems that would be much easier to keep consistent, and be a much more accurate reflection of high ISO capabilities.

If you want hardcore measurbator test results, look at dpreview.com or start your own infallible website.

If you want some realistic samples, look at dcresource.com.

Both are valid.

themitty
12-15-2007, 12:51 PM
If you want hardcore measurbator test results, look at dpreview.com or start your own infallible website.

If you want some realistic samples, look at dcresource.com.

Both are valid.

No, they're not both valid. Jeff is clearly using this test to compare (or measurbate, as you call it) the same shot with different cameras - it's not a sample shot at all. It's passed off as a standardized test, and it's a very flawed test. What's valid about a test with so many variables? Do you understand that by using different apertures and shutter settings you're getting very different exposure values. This clearly has an impact on noise performance - I would hope you're not ignorant of that (I know Jeff isn't, which is why it is so disappointing to see these inconsistencies).

To your childish response about me starting my own infallible website, no thanks. But this is an open forum, and Jeff asks for comments on his reviews, so that's what I'm doing.

What's wrong with making a constructive suggestion that it would be a much better, more realistic, and more valuable test if the cameras used the same settings? We're not talking about point and shoots where the user would let the camera make the decisions. These are $1400 plus cameras, and should be tested accordingly.

cdifoto
12-15-2007, 01:09 PM
No, they're not both valid. Jeff is clearly using this test to compare (or measurbate, as you call it) the same shot with different cameras - it's not a sample shot at all. It's passed off as a standardized test, and it's a very flawed test. What's valid about a test with so many variables? Do you understand that by using different apertures and shutter settings you're getting very different exposure values. This clearly has an impact on noise performance - I would hope you're not ignorant of that (I know Jeff isn't, which is why it is so disappointing to see these inconsistencies).

To your childish response about me starting my own infallible website, no thanks. But this is an open forum, and Jeff asks for comments on his reviews, so that's what I'm doing.

What's wrong with making a constructive suggestion that it would be a much better, more realistic, and more valuable test if the cameras used the same settings? We're not talking about point and shoots where the user would let the camera make the decisions. These are $1400 plus cameras, and should be tested accordingly.

There are different types of reviews for a reason.

rhodeislandguy
12-15-2007, 01:14 PM
I have reworded that paragraph a bit to make it a little less offensive to A700 owners :D

Jeff--I was simply confused by the two reviews. Certainly most all of this is subjective and you absolutely have to write what YOU think and see. That's what makes your reviews a great resource.

themitty
12-15-2007, 02:59 PM
There are different types of reviews for a reason.

what's the point of that portion of the review then? If it's truly just a sample shot, as you claim, than why would the reviewer use it to directly compare it to other cameras? Do you honestly feel it makes sense to compare camera A to camera B when the shooting parameters (and conditions) are completely different?

I'm not trying to slam Jeff's whole review - I think he does a very good job in most respects, but this aspect of it just seems plain wrong to me, and I'm simply trying to illustrate why.

erichlund
12-15-2007, 05:18 PM
No, they're not both valid. Jeff is clearly using this test to compare (or measurbate, as you call it) the same shot with different cameras - it's not a sample shot at all. It's passed off as a standardized test, and it's a very flawed test. What's valid about a test with so many variables? Do you understand that by using different apertures and shutter settings you're getting very different exposure values. This clearly has an impact on noise performance - I would hope you're not ignorant of that (I know Jeff isn't, which is why it is so disappointing to see these inconsistencies).

To your childish response about me starting my own infallible website, no thanks. But this is an open forum, and Jeff asks for comments on his reviews, so that's what I'm doing.

What's wrong with making a constructive suggestion that it would be a much better, more realistic, and more valuable test if the cameras used the same settings? We're not talking about point and shoots where the user would let the camera make the decisions. These are $1400 plus cameras, and should be tested accordingly.

Actually, if you read ALL the words, he does specifically say that you cannot draw a lot of conclusions from the night scenes, because of the varying conditions. That is why he also does the studio shots. There, he controls the light. Even there, the differences are clear, and you cannot argue different conditions in identical studio shots.

themitty
12-15-2007, 05:31 PM
Actually, if you read ALL the words, he does specifically say that you cannot draw a lot of conclusions from the night scenes, because of the varying conditions.

Which makes his original statement all the more amazing and wrong.


That is why he also does the studio shots. There, he controls the light. Even there, the differences are clear, and you cannot argue different conditions in identical studio shots.

Well, the parameters are much closer, but it's still not identical. 1/100 for the a700, 1/80 for the d300, both at f8, with +1/3 EV comp. So the a700 has slightly less exposure than the d300.

Plus, if you look at them next to each other, as in this thread:

http://forums.dpreview.com/forums/read.asp?forum=1039&message=26030694

I don't see how you could possibly say one wipes the floor with the other. The a700 shot is clearly sharper, with better color and contrast, and more color noise, while the d300 has less noise, which seems to be a result of higher level of noise reduction, and therefore is more soft.

Rooz
12-15-2007, 05:34 PM
you'll also note that jeff says go into a store and try these for yourselfr b4 you but. its not necessarily about picking the best, its about whats picking the best for you.

if it was just about picking the very best of the best then there would be no point doing any reviews, the d300 is obviously the King, the 40d is the prince and the a700 is the court jester. :):D

themitty
12-15-2007, 06:11 PM
if it was just about picking the very best of the best then there would be no point doing any reviews, the d300 is obviously the King, the 40d is the prince and the a700 is the court jester. :):D

Well, that's one hell of a court jester than - it provides the brains for the king, steadies his shakes, and costs $500 less. :D

Jeff Keller
12-15-2007, 06:37 PM
I don't see how you could possibly say one wipes the floor with the other. The a700 shot is clearly sharper, with better color and contrast, and more color noise, while the d300 has less noise, which seems to be a result of higher level of noise reduction, and therefore is more soft.

Whoa, I never said anything about floor wiping in the studio test comparison. I said:
Once again, I think the D300 beats both the 40D and the A700 in terms of high ISO performance -- and I think you'll agree if you compare the test photos.

Also, I've never claimed that any of these tests are scientific, as a previous poster mentioned. Especially the night shot -- it's impossible to be consistent on that one, due to the conditions more than the cameras.

themitty
12-15-2007, 06:51 PM
Whoa, I never said anything about floor wiping in the studio test comparison. I said:
Once again, I think the D300 beats both the 40D and the A700 in terms of high ISO performance -- and I think you'll agree if you compare the test photos.

I didn't claim you said that about the studio comparison. You could certainly see how someone could read your quote and take it as such - "it was written as a blanket statement - the d300 adn 40D wipe the floor with the a700 at high ISOs". You've since corrected it, which is great.

But, how can you look at the two 100% crops I linked to and not see that the a700 image is sharper, has better color and more contrast. Is the end goal to really wipe out all the noise, sharpness, color and contrast be damned? If so, than yes, the d300 wins.

And for the record, I've owned both the a700 and 40D. I kept the a700, but I think the 40D is indeed slightly better at high ISOs in terms of retaining fine detail. I also feel it's better than the d300 in this regard, so I'm not just some blindly loyal fanboy.


Also, I've never claimed that any of these tests are scientific, as a previous poster mentioned. Especially the night shot -- it's impossible to be consistent on that one, due to the conditions more than the cameras.[/I]

Again, to my original point, why on earth would you go out of your way to compare the cameras based on these images, and make such an obviously inflammatory statement to boot? And why are you shooting them at such vastly differing exposures?

DonSchap
12-16-2007, 07:02 AM
I didn't claim you said that about the studio comparison. You could certainly see how someone could read your quote and take it as such - "it was written as a blanket statement - the d300 adn 40D wipe the floor with the a700 at high ISOs". You've since corrected it, which is great.

...

Again, to my original point, why on earth would you go out of your way to compare the cameras based on these images, and make such an obviously inflammatory statement to boot? And why are you shooting them at such vastly differing exposures?

"Hey little buddy ... what are you doing down here on the floor? (as he bends down and retrieves his α700) My, my ... seen some better days, haven't you? What do you think you are ... some kind of floor mop? Analysts and their ... evaluations, huh? C'mon, let's go back to work. Enough loungin' around. I mean, considering that you were Editor's Choice (http://www.popphoto.com/cameras/4801/editors-choice-sony-alpha-700.html) for November 2007 ... wow, this is rather annoying."

All kidding aside, the α700 delivers the goods, as far as I am concerned. It is a quantum leap over the α100 ... and that little number took Camera of the Year 2006 honors as it was. So, yeah ... I had an investment in Minolta glass and SONY came around and back-filled (literally ... a digital body behind my glass). Pretty cool, when you think all of that could have been completely lost to film-industry history, three years ago.

Something is in the works with the Full Frame design (α900? or "Flagship"), so I'd suggest they seem pretty serious about this little game we're seeing. Many have taken the road that SONY is a flash in pan ... I'm not betting that way. They seem to have come to play ... and perhaps they're just a tad bit behind, but with Canon's EOS 1Ds MKIII and Nikon's D3 already out ... they could and more than likely will slip in with something devastatingly better ... leave the others stuggling to catch up.

We shall see what we shall see ... 2008 may set the record straight! :cool: