The Chess Mind

Neon Qwerty writes, by email:

Hey Dennis!

[snip]

I also have a trivial question re: computer chess and faulty engine evaluations. Everyone agrees that computers will often evaluate a side as winning when a human can intuitively tell that it's a dead draw. The question is: what is the most [egregious] evaluation that a computer can give a drawing position? Two pawns up? Three? A rook?

Just wondering and thought that you might know,

Alan.

Alan,

That's a fun question, though one difficulty in answering is that new engines are coming out all the time and this is a problem programmers regularly work on. So the answer is that I don't know, but as an opening bid I offer the following position:

White is up two rooks and a bishop, but although the position is hopelessly drawn, most of my software programs don't get it.

Shredder 9 wins the palm on this one, awarding White only a very modest .35 edge (at depth 37), which perhaps reflects the abstract possibility that White could put a rook en prise and Black mistakenly capture it.

Fritz 9: White has a 4.5 pawn advantage.

Rybka 1.0, 32-bit: White +12.95.

Fruit 2.2.1: White +12.96.

Junior 9: White +15.61

Hiarcs 9: White +16.56!

Not a proud display for computerdom, but Shredder 9 shows that it's possible to write code for this. In any case, readers are invited to top this example, and to find situations Shredder doesn't handle as well.

Some time ago I ran my own little comparison of Fritz 9 and my previous favorite, Shredder 9; now Kevin Solcich has thoughtfully written in to share the results of his test of Fritz 9 against the brand-new Hiarcs 10. My comments are interspersed.

Dear Dennis,

I thought I'd contact you directly as I wasn't allowed by the system to comment on your 11/17/05 post.

[DM: The default deadline for comments is within 7 days of a post's publication; understandably, as readers are less and less likely to scan (now much) older posts for comments. I'm glad you persevered and used the contact link!]

In that post you gave four positions for Fritz 9 and Shredder 9 to analyze. You concluded that F9 was ultimately a little better.

[DM: True, but a little misleading. If one has to choose between the two engines, F9 seemed to me superior, but it would be more accurate to summarize view with these three comments:

1. F9 solves deep tactics more quickly than S9. 2. S9 finds subtle positional ideas more quickly than F9. 3. F9's speed advantage in tactics exceeds S9's speed advantage in positional play.]

I did the same thing today with F9 and the just released Hiarcs 10. My computer is very similar to yours. I have to give the verdict to F9 as well. It was a tiny bit faster (actually a good bit more then that in position one) in all four positions. Although I think I'd have to say that H10 worked out all the details a little bit quicker in position four.

I think I am a little sad that I spent my money on H10. As you wrote in your original post, this sort of test isn't conclusive but it at least made me wonder whether the average club or non-professional player (myself) really should have much use for more then one of these ultra-strong programs.

[DM: I think that depends on what you want to do with the software. If you use it for analysis, whether of GM games, your completed OTB and online blitz games, or (legally, I hope) in correspondence/email chess, then it's worth considering a second engine. Unless you're at least 1800-2000, though, that's probably not all that important.]

I hope my report was of some small interest to you. Keep up the good work.

[DM: Indeed it was! Thoughtful questions and contributions from my readers are welcomed, and I appreciate your taking the time to write.]