Crossword SNITCH

A quantitative guide to the degree of difficulty of the Times Cryptic Crossword

Solver Types and a Neutrino Detector

The SNITCH has always relied on a set of Reference solvers to calculate the difficulty of the crossword. Updating this list over time has led me to look at different types of solver. In particular, it has been important to detect and remove "Neutrinos" from the data (see further below). Once this is done we can look at other types of solver more clearly.

What's a Neutrino?

Neutrino – A person who manually solves the puzzle on paper, and then goes to the Crossword Club and types in the answers as rapidly as possible, either in order to give a high position on the leaderboard or because they are entering a typing competition. (Times for the Times glossary definition)

Have you noticed that the top of the Times Cryptic Crossword Club leaderboard is occupied by solvers who have some remarkable completion times? How does someone solve the puzzle in around 2 minutes, when it probably takes that long to just write in the answers?

On the Times for the Times blog site, we've come to know these solvers as "Neutrinos", a term coined by Tony Sever some years back to describe those people who like to occupy the top spots on the leaderboard without actually being top-notch solvers. The theory is that these people do the crossword on paper, then transcribe the answers on the Crossword Club interface.

And the big question is: how can we detect these? Some of them are pretty obvious: if you look at the results at the table on the right, the top solver in recent months is 2 Down with an average solving time of 02:02 (min:sec). Given that this is faster than average times of the known best solvers, we're sure that this is not a real solving time. But it's much harder to tell when you're talking about average times that are in the realms of possibility. Who are the Neutrinos and who are the real solvers? This page tries to answer that question.

Clusters of solvers

The chart below shows a way to separate Neutrinos from other solvers. It plots each solver's average solving time (in seconds) on the x-axis against an "alignment to SNITCH" factor on the y-axis. This alignment factor is 1 when, on average, a solver's personal NITCH rating for a crossword is proportional to the SNITCH rating for that crossword. It is 0 when there is no alignment. (For those interested, more detail on this is provided below).

The blue dots on this chart represent the Reference solvers for the SNITCH as of Oct 2018. (Note that you can hover over each point to see the name of the individual solver.) As you'd expect, these are clustered around an alignment factor of 1 (generally between 0.5 and 1.5) regardless of their average solving time.

The red dots represent solvers who have been around for some time but are not Reference solvers. A group of these is spread across the range of average solving times and also spread about the SNITCH alignment factor of 1. But there's a cluster of red dots in the lower left corner that has low solving times and is spread around an alignment factor of 0. These are the classic-style Neutrinos - those who are very fast on average, regardless of how hard the puzzles are.

This data suggests that any solver with a solving time of less than, say, 8.0 minutes and a SNITCH alignment factor of, say, less than 0.5 could be safely classified as a Neutrino.

The grey dots represent new solvers since the Reference solver list was baselined in Oct 2018>. We can now see that some of these are clearly Neutrinos (the ones in the cluster at the lower left, for example). Others with low solving times need further investigation, and we do this by checking the details of their alignment with the SNITCH.

Remaining Questions

We see that some solvers have alignment to the SNITCH figures well above 1, which would suggest that they seems to find harder puzzles especially hard. These solvers often have small data sets, including those with relatively high average solving times that don't always make the top 100 on the leaderboard. With the smaller amount of data, the alignment is likely to be less reflective of their true performance.

Blog Solvers: adding solving times from the blog

On the Times for the Times blog site, solvers regularly report their solving times. Where they have allowed it, we now add them to the list of solvers and track their progress. This is a new category of solver, the Blog Solver, for those who've reported solving times regularly over the last six months.

Notes to Blog Solvers:

  • the most reliable format for reporting times is with format "mm:ss" or as "x minutes" in the title or at the start of the entry
  • only the first main comment for each blogger is used; responses to other comments or a second comment is ignored.
  • to report errors, include "n errors" just after the time
  • If you just have a time it is assumed you have no errors unless you mention DNF or pink squares or giving up, etc, in the text
  • Examples of good time notifications are "38:30, 0 errors" and "39 minutes, 0 errors" in the comment title or as the first part of your comment
  • This info is also available using the "Blog Solver" button on the right hand side of the detailed crossword solving times for each crossword

Updating solver status

This analysis allows me to reclassify solvers, as shown in the chart below. One of my main motivations is to be able to update the list of Reference solvers reliably. I'd like to be able to include new solvers (especially good ones) without accidentally including Neutrinos. And I'd like to track other non-Reference and non-Neutrino solvers, even if their data isn't used for the SNITCH.

So I can now classify solvers as follows:

Neutrino
Those with fast solving times and low or inconsistent alignment to the SNITCH (shown in orange)
Reference Solvers
Those aligned to the SNITCH and having an average solving time of less than 45 mins (shown in blue)
Tracked Solvers
Those that don't meet the conditions for Reference Solvers but I can be fairly sure are not Neutrinos (shown in red)
Blog Solvers
Those that report their solving times in the Times for the Times site (shown in purple)

New solvers will come into the list over time and can be reclassified on my next update.

I've shown the old and new status in the full table to the right so that you can track the changes for each user.

So what's the SNITCH alignment factor?

This is actually pretty straightforward if you know a little maths. I plot the solver's individual NITCH scores agains the corresponding SNITCH for the same puzzle and measure the slope of the line of best fit. A slope of 1 indicates that the results are proportional, a slope of 0 means that there's no correlation.

For example, verlaine has results as shown in the chart below. Easier puzzles (according to the SNITCH) are on the left of the x-axis, harder ones on the right. verlaine's NITCH results vary, but are generally higher as we progress from left to right. The overall slope of the line of best fit is 1.05, showing good alignment with the SNITCH on average.

In contrast, benedick has results as shown in the chart below. There is no correlation with the hardness of the puzzle according to the SNITCH. In fact, the slope is negative, meaning that on average the harder puzzles are easier for benedick than the easier ones. This lack of correlation is exactly what you'd expect if the solver is just typing in known answers.

Nothing is simple and we have some solvers with a mix of results like the ones shown in the chart below. The correlation with the SNITCH is inconsistent - good on some solutions but not on others. On the latter, the solver looks just like a Neutrino, with short response times regardless of the SNITCH value. Presumably they type the answers in directly on these shorter puzzles. For this reason, where we detect this sort of solver, we will mark it manually as a Neutrino.

Solving times for club members

The table below shows crossword club solvers who have submitted more than 20 solutions in the last six months, with an average solving time of less than 60 minutes for an error-free crossword. They are ranked by that solving time. It also indicates their status, including whether they are Reference solvers for calculation of the daily SNITCH.

Rank Solver Status - Oct 2018 Current Status Average Time SNITCH Alignment
1 2 Down tracked neutrino 02:02 -0.07
2 Southern Cross new_solver neutrino 02:23 0.03
3 niq.dow new_solver neutrino 02:40 -0.02
4 TimA tracked neutrino 02:50 -0.02
5 Stefan Szulc tracked neutrino 03:03 0.01
6 sarahsolomon new_solver neutrino 03:05 -0.0
7 bmag new_solver neutrino 03:26 0.12
8 Margaut tracked neutrino 03:32 0.1
9 Henry Wyatt tracked neutrino 03:33 0.09
10 willperf new_solver neutrino 03:35 0.13
11 seamus.butler new_solver neutrino 03:38 -0.03
12 childshouse new_solver neutrino 03:41 -0.01
13 robertsonjon new_solver neutrino 03:45 0.27
14 BrianS tracked neutrino 03:46 -0.0
15 robertscottf new_solver neutrino 03:50 0.35
16 Petinny new_solver neutrino 03:52 0.05
17 accessit new_solver neutrino 03:58 -0.05
18 GolfDocAndy tracked neutrino 04:17 -0.09
19 C M Gilles new_solver neutrino 04:26 -0.21
20 ray lewis tracked neutrino 04:31 -0.02
21 mohn reference reference 04:46 1.06
22 chaswil tracked neutrino 05:03 0.19
23 antigone new_solver neutrino 05:23 0.02
24 Pontius tracked neutrino 05:28 0.1
25 Nathan Panning new_solver reference 05:40 0.94
26 kenhors new_solver neutrino 05:41 0.09
27 Diskus tracked reference 05:43 0.75
28 bettyw65 tracked neutrino 05:54 0.1
29 Pnmcg new_solver neutrino 06:04 0.02
30 alf3056 tracked neutrino 06:05 0.1
31 Magoo reference reference 06:11 1.37
32 Jason reference reference 06:24 0.95
33 lili tracked reference 06:24 1.22
34 hazeltearne new_solver tracked 06:30 2.1
35 verlaine reference reference 06:30 1.05
36 mauefw new_solver tracked 06:40 1.38
37 mistigris reference reference 06:59 1.09
38 Paul Gilbert tracked reference 07:02 1.2
39 cmjhutton new_solver reference 07:16 1.02
40 hilarym tracked reference 07:24 0.97
41 aphis99 reference reference 07:29 1.11
42 greggybell tracked neutrino 07:37 0.98
43 benedick reference tracked 08:17 -0.53
44 Strober new_solver tracked 08:28 0.44
45 richard reference reference 08:31 0.84
46 Contrarian3 new_solver reference 08:43 1.08
47 samgdavies new_solver tracked 08:57 0.34
48 boulemiche tracked reference 08:57 0.52
49 asaldanha reference reference 09:03 1.15
50 markdobell new_solver reference 09:34 1.19
51 Umpire new_solver reference 09:55 0.89
52 glheard reference reference 10:00 1.04
53 tiggis tracked reference 10:04 1.29
54 Kinghedger tracked reference 10:12 1.1
55 chindit reference reference 10:14 0.93
56 Deane Short new_solver reference 10:37 1.01
57 AlanS reference reference 10:41 1.15
58 KenD51 tracked tracked 10:41 -0.05
59 Topical Tim reference reference 10:42 1.07
60 lickert reference reference 10:44 0.95
61 HoneyBadger new_solver reference 10:49 1.05
62 Jim Willis new_solver reference 10:52 1.25
63 christopherclark4945 new_solver tracked 11:08 0.2
64 philjordan1147 new_solver reference 11:09 0.82
65 keriothe_ reference reference 11:22 1.21
66 miked1961 tracked reference 11:29 1.24
67 fshephe reference reference 11:34 0.63
68 SimonAnthony reference reference 11:35 1.54
69 neilr reference reference 11:38 1.15
70 Heyesey tracked reference 11:39 1.42
71 Tom Stubbs reference reference 11:40 1.15
72 David Crooks reference reference 11:41 0.83
73 Morphosis new_solver reference 11:47 1.04
74 chriss new_solver reference 11:49 1.26
75 toots new_solver reference 11:49 0.87
76 scrabbler new_solver reference 11:50 0.95
77 Henry White new_solver reference 11:51 0.97
78 Sheapey reference reference 12:00 1.35
79 chriswords reference reference 12:04 0.72
80 andy tracked reference 12:05 1.04
81 fuzz tracked tracked 12:07 0.38
82 edwards.robert1 tracked tracked 12:07 0.07
83 canismiser new_solver reference 12:15 1.34
84 tilbee new_solver reference 12:20 1.01
85 d.middleton new_solver reference 12:35 0.88
86 Stavros reference reference 12:36 1.18
87 John Walsh tracked reference 12:38 0.79
88 robertL tracked reference 12:45 1.24
89 aphasic tracked reference 12:46 1.11
90 Golvellius tracked reference 12:47 0.93
91 ChuckB tracked reference 12:52 1.1
92 Ed freshpot reference reference 12:57 0.97
93 MikeOsborne reference reference 12:58 1.18
94 March new_solver reference 13:00 1.07
95 Silva reference reference 13:08 1.17
96 michaelbell53 new_solver tracked 13:16 0.3
97 JimboFS new_solver reference 13:18 1.19
98 Paul Bacon new_solver reference 13:20 0.94
99 SimonB new_solver tracked 13:29 0.3
100 LouWeed reference reference 13:38 1.02
101 Penfold_61 new_solver reference 13:39 1.01
102 Eddddddd tracked reference 13:40 1.17
103 njnclarke tracked reference 13:42 1.09
104 7dPenguin tracked reference 13:48 1.24
105 Freemers new_solver tracked 13:58 0.4
106 howardwilde1 new_solver reference 13:59 1.06
107 Groucho reference reference 13:59 0.98
108 gabrojak new_solver reference 14:02 1.17
109 Pootle new_solver reference 14:04 1.4
110 fishernick79 tracked reference 14:12 0.91
111 john.brassey new_solver tracked 14:15 0.32
112 Peponi13 new_solver reference 14:17 0.87
113 Crucifer tracked reference 14:17 1.12
114 Semillon tracked reference 14:17 1.18
115 Tibthorpe reference reference 14:20 1.05
116 franceslc11 new_solver reference 14:20 0.97
117 Ben Staveley reference reference 14:21 1.32
118 Secondbass new_solver reference 14:24 0.66
119 alexh tracked reference 14:31 0.76
120 cdb_100 tracked reference 14:38 1.04
121 myles tracked reference 14:42 0.97
122 GeorgeTz new_solver reference 14:45 0.93
123 zinzan new_solver reference 14:51 0.99
124 CentralLine tracked reference 14:53 1.31
125 grape tracked reference 15:00 1.39
126 JezWoolfrey reference reference 15:04 1.07
127 hatterboy new_solver tracked 15:04 -0.01
128 phthisis reference reference 15:09 1.01
129 Johninterred new_solver blog_solver 15:10 1.18
130 Finzi new_solver reference 15:10 0.69
131 BlackburnRover new_solver reference 15:19 0.48
132 pitcaithlie new_solver reference 15:25 1.85
133 andrewlake tracked reference 15:35 0.92
134 Looch reference reference 15:38 1.03
135 daulbs tracked reference 15:41 1.22
136 RobR reference reference 15:42 1.08
137 MK76 tracked reference 15:46 1.15
138 tapbarker tracked reference 15:48 1.19
139 LJCN new_solver reference 15:56 1.11
140 Paul Hammond new_solver reference 15:56 0.78
141 chriswestontour new_solver reference 16:00 0.81
142 Achernar tracked reference 16:01 0.72
143 AndyR tracked reference 16:08 1.03
144 Mike Mansbridge new_solver tracked 16:14 2.58
145 Squallaby reference reference 16:14 1.5
146 robinh reference reference 16:23 0.57
147 Jeremy S new_solver reference 16:33 0.82
148 Galspray tracked reference 16:41 1.43
149 Philip_ new_solver reference 16:45 1.32
150 DunPangolin new_solver reference 16:48 0.77
151 ilan new_solver reference 16:53 0.63
152 cmcnsmith new_solver reference 16:55 0.64
153 Encota new_solver reference 16:57 1.29
154 jimwetzler tracked reference 16:58 1.0
155 ajcauk1 new_solver reference 16:59 0.88
156 MrChumley tracked tracked 17:07 0.37
157 Swanny tracked reference 17:08 1.65
158 earlylight tracked reference 17:12 0.57
159 GMcD reference reference 17:17 1.05
160 Crossiant new_solver reference 17:20 1.46
161 Obrador new_solver reference 17:25 1.29
162 PGooden reference reference 17:25 1.1
163 Colin2355 new_solver reference 17:31 1.15
164 npweston new_solver reference 17:32 0.83
165 Craven reference reference 17:34 0.78
166 Riche new_solver reference 17:35 0.88
167 Grestyman tracked reference 17:44 1.2
168 Dr Dolittle new_solver reference 17:47 1.88
169 pterodactyl reference reference 17:59 0.9
170 LuckOfLucien tracked reference 18:08 1.13
171 FraserH new_solver reference 18:10 1.25
172 DanSolo new_solver reference 18:10 1.27
173 Foozler new_solver tracked 18:14 0.43
174 rrricharrrd tracked reference 18:19 1.11
175 Dvynys new_solver reference 18:33 1.57
176 SeamusX new_solver reference 18:35 1.02
177 Dzhigit tracked reference 18:39 1.07
178 tds new_solver reference 18:41 1.03
179 hopkinb new_solver reference 18:43 1.17
180 McBeakLives new_solver reference 18:44 1.31
181 Jeremy Kenyon new_solver reference 18:50 0.63
182 julian_w tracked reference 18:53 1.12
183 martinede new_solver reference 18:53 0.79
184 Curarist tracked tracked 18:54 1.1
185 Gallion Fish new_solver reference 18:56 1.06
186 Zabadak3 new_solver reference 18:58 0.91
187 endafk tracked reference 18:59 1.07
188 KET tracked reference 19:00 1.14
189 SBQuick new_solver reference 19:02 1.43
190 John Arnold reference reference 19:02 1.23
191 valerian1967 tracked reference 19:06 1.09
192 Cookincats tracked reference 19:07 1.57
193 Mike C new_solver reference 19:12 0.91
194 TB22 new_solver reference 19:12 1.05
195 jeremy.kenyon new_solver reference 19:15 0.79
196 doublecross tracked reference 19:20 1.2
197 pc8 tracked reference 19:24 1.11
198 Solver52 reference tracked 19:24 0.4
199 James P tracked reference 19:36 1.43
200 Zander tracked reference 19:40 1.17
201 rt51 new_solver reference 19:43 1.08
202 uvofiz new_solver reference 19:54 0.73
203 M Bennett new_solver reference 19:58 1.2
204 Andrew Whiffin new_solver reference 20:00 1.44
205 daniel.faith new_solver reference 20:03 1.07
206 Hamish.S new_solver reference 20:07 1.42
207 kath tracked reference 20:08 0.96
208 dec reference reference 20:08 1.07
209 squark new_solver reference 20:10 1.2
210 DamienKnapma tracked reference 20:22 0.94
211 sue.cresswell new_solver reference 20:29 1.2
212 UisceTe new_solver reference 20:30 1.1
213 Pxspxs new_solver reference 20:41 0.93
214 hugh schofield new_solver reference 20:42 1.37
215 drginacoco new_solver reference 20:47 0.62
216 gsriordan52 new_solver reference 20:47 1.04
217 DianeRS new_solver reference 20:49 1.29
218 Upfisher new_solver reference 20:50 1.45
219 Nangle tracked reference 20:51 0.95
220 Kevin Gregg reference reference 20:59 1.32
221 Onelug new_solver reference 21:03 1.2
222 david_ch new_solver blog_solver 21:09 1.44
223 Waterloo new_solver reference 21:13 1.14
224 David Bourne tracked reference 21:16 1.58
225 crosswordian reference reference 21:25 0.92
226 andrewfeist96 new_solver reference 21:28 0.66
227 george.garlick new_solver reference 21:36 0.93
228 Corymbia new_solver reference 21:37 1.25
229 frost tracked reference 21:42 1.33
230 Rod B new_solver reference 21:52 0.97
231 luxtonj61 new_solver reference 21:57 1.28
232 Leegoods new_solver reference 22:05 1.32
233 Tudds reference reference 22:23 1.24
234 The Caffeine Kid new_solver reference 22:46 0.96
235 richardCV22 tracked reference 22:46 1.16
236 cognominative new_solver reference 22:48 1.25
237 JohnDun reference reference 22:51 1.36
238 McBeak new_solver reference 22:52 0.82
239 stainl665 new_solver reference 22:58 1.27
240 monkfish new_solver reference 23:00 1.05
241 Jagsfan1971 tracked reference 23:04 1.91
242 goblin tracked reference 23:08 0.81
243 janesa new_solver reference 23:26 1.18
244 6098mr tracked tracked 23:30 0.28
245 medodd new_solver reference 23:35 0.94
246 JoeyC tracked tracked 23:42 2.28
247 Jeremy Weissmann new_solver tracked 23:54 2.35
248 1234 reference reference 23:59 1.14
249 tony the butcher montana new_solver reference 24:01 1.54
250 jimancliffe new_solver reference 24:12 1.06
251 klamery4 new_solver reference 24:21 0.67
252 Crippens1 reference reference 24:24 0.86
253 Sheen new_solver reference 24:24 1.11
254 puzzled1 tracked neutrino 24:26 0.89
255 JE59 tracked reference 24:35 0.97
256 rjcassidy reference reference 24:41 1.16
257 MichelinPoitiers new_solver reference 24:42 0.83
258 brian.mcnamara new_solver reference 24:45 1.27
259 Dr_Bob new_solver reference 24:57 0.84
260 Fidelio new_solver reference 24:58 1.02
261 Edprof reference reference 25:21 1.01
262 Rowlie26 new_solver reference 25:30 1.16
263 chris661554 new_solver reference 25:31 1.21
264 Mkgnao reference reference 25:34 0.9
265 Yatton50 tracked reference 25:39 0.77
266 uffindellbirtjn new_solver reference 25:44 1.83
267 Brenk tracked reference 25:49 1.17
268 SlumdogEngineer tracked reference 25:56 1.51
269 NAPASAI reference reference 25:57 1.44
270 JMbrow29 new_solver reference 26:01 1.02
271 eniamretrauq reference reference 26:20 1.1
272 hugapug new_solver reference 26:25 0.97
273 crucihacker new_solver reference 26:27 1.32
274 Gussie new_solver reference 26:31 1.56
275 bobbb tracked reference 26:36 1.02
276 timbo tracked reference 26:42 1.11
277 JayEm new_solver reference 26:43 0.72
278 infinitejest new_solver reference 26:45 1.27
279 Deezzaa new_solver reference 27:05 1.07
280 ulaca reference reference 27:14 1.15
281 barracuda reference reference 27:22 1.14
282 Bert Dove tracked reference 27:37 0.97
283 Mudge new_solver reference 27:42 1.86
284 pvoller new_solver reference 28:13 1.1
285 SK42 tracked reference 28:52 0.95
286 Ekacekabi new_solver tracked 28:59 2.33
287 symbenall2 tracked reference 29:07 1.18
288 Mike Harper tracked reference 29:15 1.48
289 nekabue reference reference 29:18 1.16
290 Winchelt new_solver tracked 29:23 0.24
291 phcbarnes new_solver tracked 29:29 -0.36
292 joe.mcgrane new_solver reference 29:46 1.02
293 Grumpole new_solver reference 29:56 1.3
294 gbestanley reference reference 30:07 0.57
295 BertieVII new_solver reference 30:16 0.9
296 Cryptique new_solver reference 30:40 0.73
297 hipponym new_solver reference 30:50 1.66
298 Manveru reference reference 30:56 1.04
299 Jeeves reference reference 31:07 1.16
300 vinyl12345 new_solver reference 31:07 1.57
301 rosselliot reference reference 31:22 1.11
302 RogerC new_solver reference 31:31 0.92
303 annikins new_solver reference 31:50 1.72
304 Rookie new_solver reference 32:18 1.07
305 VictoriaW tracked neutrino 33:08 0.16
306 simjt new_solver reference 33:08 1.17
307 alistair.boere new_solver reference 33:18 0.65
308 Lloyd new_solver reference 33:33 0.93
309 WordPlodder new_solver reference 33:41 1.23
310 koro reference reference 34:04 1.15
311 bruce.johnston22 new_solver reference 34:09 0.88
312 Birdingbob new_solver reference 34:34 1.38
313 Maggot tracked tracked 34:34 0.54
314 starstruck tracked reference 34:42 1.34
315 CoxBurglar new_solver reference 34:50 1.05
316 dosbarth new_solver reference 35:02 1.48
317 boltonwanderer new_solver blog_solver 35:21 1.15
318 Pleasuredome tracked reference 35:29 1.16
319 kapietro new_solver reference 35:36 1.19
320 j.stanton734 new_solver reference 35:53 1.14
321 tyrotim new_solver reference 36:15 0.75
322 Shushannah tracked reference 36:31 1.03
323 clivejharlow new_solver reference 37:03 0.92
324 RaghuS tracked neutrino 37:17 0.29
325 E Bradford new_solver blog_solver 37:56 1.48
326 Word Psmith tracked reference 38:08 1.23
327 Dalai Farmer new_solver tracked 38:18 1.21
328 piltdown man new_solver reference 38:38 1.49
329 rv_1 new_solver reference 38:45 1.1
330 HanS tracked tracked 38:54 0.24
331 hydrochoos tracked reference 39:15 0.64
332 joss henry tracked reference 39:40 1.26
333 gothick new_solver reference 39:40 0.88
334 jackk.larg new_solver reference 39:56 1.37
335 maryht tracked tracked 40:50 0.44
336 baronrob tracked reference 44:35 1.57
337 empty reference reference 44:42 0.92
338 MartinP tracked tracked 47:55 1.24
339 Steve Wood tracked tracked 48:12 0.86
340 intentional rounding new_solver tracked 48:21 1.04
341 The Real Steve Snackman new_solver tracked 49:33 0.72
342 kjbr tracked tracked 50:20 1.3
343 profsjpayne new_solver tracked 54:56 1.12
344 The Mighty Salmon new_solver tracked 56:44 1.03
345 Masha tracked tracked 57:34 0.88
346 paulmcl tracked tracked 57:47 1.24