The games would isolate and test specific abilities. Let me give some examples:
1. The first example game I played on the cambridgebrainsciences site: take N cards, one side is blank, the other contains a number (1 to N, unique). Scatter them on the table, numbers side up. Allow a few seconds for memorising. Now flip them, and the user has to click them in ascending order. If success, onto level N+1. If failure, back to level N-1 and lose a life. Start with three lives. Record the maximum score before lives hits zero.
2. "Simon Says" type game: 4 coloured buttons each making their own note. A length-N sequence is played through them and the user must key it back.
3. Reaction test
4. Solving a maze
5. Kim's game: N random objects presented on screen (sunflower, vacuum cleaner, budgerigar, etc). Screen is cleared, one object is taken away, at the remaining N-1 get presented. User has to remember which one was missing.
6. Recognising faces
7. Solving Sodoku
8. Classic IQ test type problems, spatial manipulation, pattern recognition
9. WhackAMole style concentration test.
10. Classical autism tests, like an animation with two triangles and multiple-choice to guess the scene they are enacting (mother and child playing, prisoner and guard, stalking, etc)
Etc etc. I'm pulling this out of the top of my head. But I'm sure cognitive scientists have devised and categorised such tests very thoroughly.
Be important thing is that each game targets specific skill, and produces a single metric/score.
My thought would be to simply create a website with the tests, where someone can login, take tests, and have their result data recorded. They could also fill in a paragraph containing pertinent information, like what compound/stack they are experimenting with. And this data would be publicly available.
This way processing the data would be completely separated from acquiring the data.
So say a researcher (someone on this forum maybe) wishes to perform an experiment (say the effect of piracetam on short term memory retention) with 20 subjects (other users of the forum who have access to piracetam, this could be coordinated in the forum). He can just get them to take tests numbers 12, 17 and 25 (which test short term memory retention, let's say) putting in "XYZ123" into their "test reference" textbox field. Then the next day he could just pull out all records matching "XYZ123" and do his analysis.
Alternatively one user could track their own performance on a couple of games as they try different compounds.
Community members could contribute new games, maybe.
