Simple cheating suspicion evidence detection for dummies

Simple cheating suspicion evidence detection for dummies

Postby CharlesRoberson » Wed Feb 21, 2018 2:10 am

Of course I don't literally mean dummies.

Due to my computer chess expertise, I've been asked to detect cheats on several occasions. Doing it properly is a complicated process, but a quick test to decide if you should involve an expert is simple and you only need a chess.com account - paid or free. Here is how:

1) login to chess.com
2) Slide down to Learn then click Analysis
3) a board will appear with options to the right
4) click on Load PGN
cut and paste the moves into the box or type them in
5) click the button that looks like a computer and magnifying glass
6) Now, click the tab that says quick

I did this for the following game
1. e4 c6 2. d4 d5 3. e5 Bf5 4. h4 h6 5. g4 Bh7 6. e6 fxe6 7. Bd3 c5
8. Bxh7 Rxh7 9. Qd3 Nf6 10. Nf3 Nc6 11. Qg6+ Kd7 12. Ne5+ Kc8 13. c3 cxd4
14. cxd4 Qc7 15. Bf4 Nxd4 16. Na3 Nf3+ 17. Nxf3 Qxf4 18. Ke2 Kd7 19. Rac1 a6
20. Rhd1 Qxg4 21. Qc2 Ke8 22. Rd4 Qf5 23. Qc7 e5 24. Nxe5 g5 25. Nac4 e6
26. Nd6+ Bxd6 27. Qxd6 Rd8 28. Nc4 Ne4 29. Qb6 Rf7 30. Rxe4 Qxe4+ 31. Ne3 d4
32. Rd1 Qf3+

7) you want to look at the statistical analysis- scroll back up to top of analysis (example from above game)

Code: Select all
Strength   White   Black
Excellent   20       27
Good        5        2
Inaccuracy (?!)   6   0
Mistake (?)   1   1
Blunder (??)   0   0
Forced     0      2
Best Move     42.9%   85.2%
Avg. Diff     0.80    0.08


So, here we see white had 6 inaccuracies, 1 mistake, 0 blunders and a best move percentage of 42.9%.
I would say insufficient info to suspect cheating for white.
On the other hand, Black has 27 excellent moves, 2 good moves, 0 inaccuracies, 1 Mistake, 0 blunders and a best move percentage of 85.2%.
After looking at the mistake, it is the kind of move that could have been easily confused with a computer suggested move. I'd say this shows sufficient evidence for suspicion. More analysis from an expert and possibly more data is needed to confirm the suspicion.

I post this because bringing such things to a TD is a timely manner is very important. If you wait until after the tournament to complain, it is too late.
CharlesRoberson
 
Posts: 69
Joined: Wed Mar 21, 2012 2:38 pm
First Name: Charles
Last Name: Roberson

Re: Simple cheating suspicion evidence detection for dummies

Postby sspencer » Mon Mar 12, 2018 4:24 pm

Ok so this peaked my interest and I played a game against a know and popular computer program. I am playing white, program is black, game set to 24 hours a move. Such as one can play on many online sites such as Chess.com.



1.d4 Nf6 2.f4 d5 3.e3 c5 4.Nf3 Nc6 5.Nc3 cxd4 6.Nxd4 Bg4 7.Be2 Bxe2 8.Qxe2 e6 9.Bd2 Bc5 10.Nf3 O-O 11.O-O-O Qe7 12.Ng5 h6 13.h4 b5 14.g4 b4 15.Na4 Bd6 16.Qg2 Nd7 17.Nf3 Nb6 18.b3 Nxa4 19.bxa4 a5 20.Kb1 Rfc8 21.h5 f6 22.g5 fxg5 23.fxg5 hxg5 24.Nxg5 Ne5 25.Rdf1 Rf8 26.h6 Nc4 27.Bc1 Be5 28.Rfg1 g6 29.Nf3 Qh7 30.Nh4 Rf6 31.Nxg6 Rxg6 32.Qxg6+ Qxg6 33.Rxg6+ Kf7 34.Rg5 Bf6 35.Rf1 Ke7 36.Rg6 Be5 37.e4 dxe4 38.h7 Rh8 39.Bg5+ Kd6 40.Rd1+ Kc7 41.Rh1 e3 42.Rg8 e2 43.Rxh8 Bxh8 44.Re1 Nd6 45.Rxe2 Kd7 46.Rd2 e5 47.Rd5 Ke6 48.Rxa5 1-0

Strength White Black
Excellent 40 43
Good 8 4
Inaccuracy (?!) 0 0
Mistake (?) 0 0
Blunder (??) 0 0
Forced 0 0
Best Move 74.5% 78.3%
Avg. Diff 0.08 0.06

To the best of my knowledge set this program up to play its best and did not handicap it. Neither I nor it made any Inaccuracy, Mistake or Blunders according to the analysis, yet the score at the end of that game was plus 7 for white. I actually made more good moves than it did. Knowing I was playing a program utilized an anti-program strategy.

Yet at the bottom shows me almost matching it move for move on best move percentage. I read and scoffed at the article in Chess life last month about cheating on Chess.com. This analysis proves I was right in doing so ha ha. So you are saying someone can look at these numbers and determine cheating by it? With all do respect I find that comically absurd.
sspencer
 
Posts: 44
Joined: Tue May 03, 2011 2:22 pm
First Name: Shawn
Last Name: Spencer

Re: Simple cheating suspicion evidence detection for dummies

Postby ThrillerFan » Fri Apr 27, 2018 10:42 am

This is the first time I've checked the board in quite a while, and when I saw the subject line of this one, I had to look and see what was said, and I have to 100% agree with Shawn.

If you take a 2000 player, and you compare him to a 2700 player, obviously he is going to make far more errors. If you compare him to a 1300 player, he is of course going to make far fewer errors.

However, to make such as assessment about computer cheating on a single game is completely and utterly asinine! Any player is capable of playing a single brilliant game! You need a valid sample size before you can say anything, like 30 or more (30 is typically the "magic number" in statistics for valid sample sizes).

And to be dumb enough to rely on a single bot and not using human intervention to figure this out? PA-LEEZ!


SMH!
ThrillerFan
 
Posts: 144
Joined: Thu Sep 09, 2010 11:47 am
First Name: P
Last Name: McCartney

Re: Simple cheating suspicion evidence detection for dummies

Postby CharlesRoberson » Tue May 08, 2018 1:20 pm

I think you both miss understood the article.

Patrick: The title says "suspicion" so it means one game is enough to cast suspicion. The article goes on to say that once you have suspicion you seek an expert. It doesn't say what the expert will do. The expert can then use multiple engines and other techniques. So, your statements completely miss my point.

Shawn: There is a difference between comparing human vs human and comparing human vs engine. Your one example which is a special case that doesn't meet the topic (human vs human) is inadequate to disprove the point. Anticomputer techniques are well known and would be taken into consideration by an expert.

Again, the article is about deciding if there exists sufficient reason to be suspicious.
CharlesRoberson
 
Posts: 69
Joined: Wed Mar 21, 2012 2:38 pm
First Name: Charles
Last Name: Roberson

Re: Simple cheating suspicion evidence detection for dummies

Postby ThrillerFan » Thu May 10, 2018 10:11 am

No Charles, I didn't miss the point, I think YOU missed the point.

All you want to do is add another layer of analysis via an expert. That still doesn't confirm squat. Whoopie! I had a 2400 player run it through a few computers and it confirms brilliancy by Black. Who cares? It's one game!

I've bounced between 2000 and 2180 for the last 5 years. Does this mean that I can't have a single game where I make moves that a 2500+ player would typically make? Chess is not a linear game, and we are not all "Steady Eddie's" that perform at our level at all times. I've had entire tournaments where my performance rating is well over 2300 and I've had others where I literally felt like I couldn't find my way out of a paper bag and perform at a 1700 level!

So once again, adding some "Expert" to analyze a single game cannot in any way, shape, or form, confirm the suspicion. That expert could be Kasparov or Carlsen himself. Still doesn't mean diddly squat when your sample size is 1 game.

Show me 30 consecutive games by a 1900 player with a collective performance rating of 2650 and now you are talking! Until then, this whole thread is a farce! Almost as bad as the McDonald's coffee lawsuit in 1996! (You're a complete idiot if you wedge a cup of hot liquid between your legs in a moving car!)
ThrillerFan
 
Posts: 144
Joined: Thu Sep 09, 2010 11:47 am
First Name: P
Last Name: McCartney

Re: Simple cheating suspicion evidence detection for dummies

Postby sspencer » Fri May 11, 2018 8:43 pm

I have to agree with Patrick here a large sample size is required or else this approach is farcical. I picked up playing postal recently again. I recently defeated the number one seed in my section a 2400 plus player I was 1840 at the time. He played loose and fast despite having 30 days for 10 moves and got beat bad (crushed actually ha ha.) That solitary win is not a big enough sample size to indicate someone may be cheating, should not even be a red flag nor should it be a recourse for a higher player who got humiliated to come to an official and point to the results that his opponent who butt hurt him cheated.
sspencer
 
Posts: 44
Joined: Tue May 03, 2011 2:22 pm
First Name: Shawn
Last Name: Spencer

Re: Simple cheating suspicion evidence detection for dummies

Postby CharlesRoberson » Thu May 17, 2018 6:13 pm

I did not say that one game was enough to prove cheating. I don't know how the two of you imagined that.

Proving cheating is a multistage test and the stated test is a first simple/fast one. It is hypothesis testing. The hypothesis being tested is was it POSSIBLE that cheating happened in this game. If you fail the test, then cheating did not happen. However, if you pass the test then MAYBE cheating happened. You can't guarantee it because this is just the first simple filter of a bigger process. So, passing it means more elaborate analysis is needed.
CharlesRoberson
 
Posts: 69
Joined: Wed Mar 21, 2012 2:38 pm
First Name: Charles
Last Name: Roberson

Re: Simple cheating suspicion evidence detection for dummies

Postby dcremisi » Sat Jun 02, 2018 1:31 pm

By whom? Who is licensing these experts? And even worse- What p value is sufficent to to “prove” cheating? Also-a condition for a T test is independence, and moves in a chess game aren’t independent. And you also have the problem that strength of opponent matters. A lot. I’ve had 3 minute blitz games with higher quality of play computer findings on Chess.com than the highest rated win of my career. We think in plans, strategies- only computers think one move abead at a time. And claiming you have to complain before the tourney ends is irresponsible, and will lead to “jumping the gun” on accusations. And we all know how rumors and destroyed reputations work, even after the accusation has been officially disproved. I have seen the impact both of dubious accusations and of computer cheating before in my Chess career, and both pose an equally serious threat to Chess. This plan is in my view at best over-correction to a real problem, and at worst gonna lead to false accusations right and left. If someone is repeatedly cheating, they’re rating will skyrocket. When that happens, maybe take a quick look to see if it’s a bit too perfect. Maybe. But one game, or even one tourney? Just... no.
dcremisi
 
Posts: 52
Joined: Fri Aug 03, 2012 6:56 pm
First Name: daniel
Last Name: Cremisi

Re: Simple cheating suspicion evidence detection for dummies

Postby CharlesRoberson » Mon Jun 04, 2018 10:49 pm

So, you are saying that fear of false accusations and all it leads to is sufficient reason to stick our heads in the sand?

Before there is any more elementary stats incorrectly quoted pertaining to this domain, I suggest people read up on the subject. There are people that do real research on the topic and FIDE uses them. If you read the research you'll see that the data points are moves instead of games and not all moves are usable data points. So, one game may not be sufficient and one game might. Secondly, the idea of needing 30 or more games is quite incorrect and doing so would allow people to cheat once every 5 or so games and not get caught.
No need to argue here just look up the research and read it yourselves.
CharlesRoberson
 
Posts: 69
Joined: Wed Mar 21, 2012 2:38 pm
First Name: Charles
Last Name: Roberson

Re: Simple cheating suspicion evidence detection for dummies

Postby dcremisi » Tue Jun 05, 2018 1:50 pm

No, we shouldn’t “stick our heads in the sand”. But we can’t be hasty. When the USCF/ FIDE releases an OFFICIAL process, that'd be reasonable. I’ve played in hundreds of USCF and FIDE tourneys, and never encountered such a thing. Self licensed “experts” and accusations based on one game- that is unacceptable. Your “every 5 games” point is sadly true. Even more effective would be cheating a little bit every other game, just a few moves. Sadly, we need to accept cheating in chess is FAR easier than catching cheaters without accusing innocents. "Better that ten guilty escape than one innocent suffer". The burden of proof is always on the accuser, which is as it should be. Proof beyond reasonable doubt.

If you want to help, lobby the USCF to standardize a process. Posts like this just cause paranoia. The only way this works is uploading all rated games to a database (kinda like chessstream) and the USCF automatically evaluating. You referenced some FIDE research. My question is: why isn't fide using it? I've played in enough FIDE events to know it isn't regularly used. Until they stand behind their work enough to use it, I don't buy it. Remember, you need a REALLY high level of certainty. A game in the top 0.1% of someone at your level sounds unusual... until you realize the average master plays 1000 game in their career. And that's leaving aside that an 1800 player who cheats moderately and routinely will rise to 1900, and his play appears normal. It's more likely people would "peek" a couple times a game, which Anand has estimated could mean a 150 "bump" in rating, then cheat a statistically obvious amount one game and none the next five. That's an idiotic method of avoiding suspicion, statistically and physically at the tourney. The challenges in applying the current research are well laid out here: https://cse.buffalo.edu/~regan/personal ... cleKWR.pdf. I'm not saying habitual cheaters can't be statistically caught, just that as Mccartney says it takes a large number of games to cancel out the numerous confounding variables and create very high statistical certainty. These processes havn't been rolled out by official organizations... and for a damn good reason.

To avoid seeming too pessimistic, I'll add statistics aren’t the only possible proof. If I noticed my opponent used the bathroom every move, had just jumped up 300 points, and took the same amount of time every move, that would work. I don’t even see how you could repeatedly cheat more than a few moves a game without attracting suspicion. This statistics business is fun, but wanding people for electronics before entering the playing hall and designated bathrooms would work better. Not that I think it’s needed. I don’t believe I’ve ever been cheated against thus far, and I’ve only seen two serious allegations involving others. There’s not enough money in Chess to make I worth it. :cry: If your looking for cheating: https://en.wikipedia.org/wiki/Cheating_in_poker.
dcremisi
 
Posts: 52
Joined: Fri Aug 03, 2012 6:56 pm
First Name: daniel
Last Name: Cremisi

Next

Return to North Carolina Chess Discussion

Who is online

Users browsing this forum: No registered users and 5 guests

cron