Sub-tests and test rules
The g-game comprises five sub-tests, presented in a fixed order as shown in this table:
| Sub-test order | Activity name shown to twins | Sub-test name | Sub-test type | Number of test items | QC item included? |
|---|---|---|---|---|---|
| 1 | Mountain | Vocabulary (Mill Hill) | verbal | 8 | yes |
| 2 | Space | ISTO/NVRA/NVRB | non-verbal | 9 | yes |
| 3 | Tower | Missing Letter | verbal | 6 | no |
| 4 | Ocean | Ravens Matrices | non-verbal | 11 | yes |
| 5 | Woodland | Verbal Reasoning | verbal | 6 | yes |
Hence, there are 3 verbal tests and 2 non-verbal tests. In total, there are 20 verbal items and 20 non-verbal items, allowing all the scores to be summed to give a total (for 'g') that is equally weighted for verbal and non-verbal abilities.
Each sub-test except for Missing Letter includes a QC item. The QC item does not contribute to the sub-test score, but is included with the aim of identifying twins who may be responding randomly. The QC item is designed to be trivially easy, hence an incorrect response may suggest a random or careless pattern of responding (when considered alongside other evidence). The identification of random responders in web tests, including the g-game, is described in the web data cleaning page.
The rules for all five sub-tests are very simple and can be summarised as follows. Logical rules were imposed by the programming within the web implementation.
- Scoring
For each item in each sub-test, the correct response is given a score of 1; any other response (including timeout) is treated as incorrect and is given a score of 0. The total score for each sub-test is the sum of the item scores (omitting QC items), hence ranging from 0 to the number of items in the sub-test. - Ordering of items
The ordering of the sub-tests, and of the items within them, is predetermined and fixed for all twins. For details, see the g-game items page. - Timeout rule
A timeout applies to each item of each sub-test (see g-game items for details). If the time limit is reached in an item without a response, the item is forfeited (treated as incorrect, with score 0); the twin is then automatically taken to a timeout screen (see screen shots below for an example); the twin must then click on a "Next" button in order to proceed with the following item. The timeout screen itself does not have any time limit, so a twin may pause for any length of time before proceeding. - Completion of items
Within each sub-test, each item was compulsory and could not be skipped. An item missed due to timeout is treated as an incorrect response. Each sub-test cannot be left incomplete: if a twin closes the browser in mid-sub-test, then no data for that sub-test is recorded. - Break points
A break point occurs after the end of every sub-test. At a break point a twin may log out of the g-game without loss of data, and may later log in again and resume by starting the next sub-test. This is explained to twins in the welcome screen (see below). Break points are not shown explicitly to twins, but are implied whenever the home screen is shown.
Before attempting any sub-tests, twins must log in, read an information sheet and complete a consent form. Before and after every sub-test, twins are taken to the "home screen". After completing all of the sub-tests, twins are given feedback. All these elements of the g-game are described in more detail below.
Welcome, login and consent
The link provided to each twin, in the email invitation, took them to the welcome and login screen shown below. Note that, for the benefit of twins, the g-game was referred to throughout as the Pathfinder study.
Note that there is a link on the welcome screen, and on the consent screen below, to the g-game information sheet (pdf). In the g-game invitation email, each twin was provided with a unique (hence identifying) login code. In order to proceed, the twin had to enter the login code in the box and click on the "Begin" button. They were then taken to the consent screen as shown below:
By the consent stage, the twins have logged in and identified themselves, so the welcome message at the top of the consent screen can greet them by name (forename only). In this screen, twins can give consent and proceed to the activities by ticking the three boxes and clicking on the "Next" button.
After completion of this consent, twins are taken to the home screen.
The home screen
The home screen is displayed before and after every sub-test. The visual elements of the home screen show the twin which sub-tests have already been completed and provide a means to start the next sub-test; or, when all sub-tests are completed, it provides a means to proceed to the feedback screen (see below).
Immediately after giving consent, this initial message is superimposed on the home screen:
After clicking OK, the full initial home screen becomes visible as shown here:
The visual elements of the home screen include five images framed by octagonal windows, representing the five sub-tests to be completed; above each window is a lamp, which will be lit to show when the respective sub-test has been completed (none are lit in this image); and at the front is a dais, on which will be placed a stone for each sub-test completed. These elements will be further illustrated in other images below.
The next sub-test to be completed (in this case the first, represented by the window furthest-left) is signalled by a pulsing glow in the respective window - this is not visible in the image above. The twin then proceeds by clicking on that window. Other windows, for sub-tests already completed and for sub-tests not yet available, are not clickable.
The screen shot below illustrates the home screen after completion of three of the five sub-tests. The lamps above the first three windows are lit, and three stones have been placed on the dais at the front. The fourth sub-test is now available, and the fourth window would be seen with a pulsing glow (not visible in this image).
The screen shot below shows the home screen with a superimposed message that appears directly after completion of the fifth and final sub-test. Note that all five lamps are lit and there are five stones on the dais at the front. The g-game is now finished, with all the data recorded on the server. The twin is invited to click on the dais which will take them to the feedback screen.
Feedback
After completing the final sub-test, clicking on the dais in the home screen will take the twin to the feedback screen as illustrated in the screen shot below. The star rating at the top (here showing 3 stars) is based on a calculation of the total g-game score from all five sub-tests. 1, 2, 3, 4 or 5 stars may be shown.
Beneath the overall rating is a visual display of the twin's performance in each of the five sub-tests. As with the overall rating, each sub-test is rated from 1 to 5 using stars and a bar chart display. In this example, the twin has done best in the Tower and Woodland sub-tests (with 4 stars each), and has done least well in the Mountain sub-test (2 stars).
The relationship between the precise scoring and these star ratings is hidden from the twins in the web interface as the star ratings are designed only to give a general idea of performance. The overall star rating relates to the total 'g' score out of 40, with 5, 4, 3, 2 and 1 stars respectively given for scores greater than or equal to 33, 26, 19, 12 and 0. In each sub-test, the star rating for the bar chart is based on a percentage of the maximum available sub-test score: 5, 4, 3, 2 and 1 stars respectively are given for scores greater than or equal to 80%, 60%, 40%, 20% and 0% of the maximum sub-test score.
After clicking OK in the feedback screen above, the end screen is shown as illustrated by the screen shot below. This states that the study is now ended for the twin and explains what they may do next. It also repeats the feedback display (in the example below, 5 stars have been awarded in all five sub-tests, denoting scores of at least 80% throughout).
This same end screen is shown if a twin decides to log in again at a later date, after having completed the g-game.
Example screen shots
Some examples of screen shots from the sub-tests themselves are shown below. These examples include the introductory screens with sub-test instructions, at least one item from each sub-test, and one or two examples of timeout screens and sub-test end screens. Note that a full specification of all test items is given in the g-game items page, so it is not necessary to show screen shots from every item below. The background images varied from item to item, even within the same sub-test, and for reference a complete gallery of the background images is shown under another heading below.
Each image below is captioned to identify the item shown.
Features common to all items of all sub-tests are (1) the count-down timer, top-right, showing how many seconds of the time limit remain and (2) the "full screen" control, which is the small square icon top-left.
The instructions here include example items for two of the three types of question (Sequences and Groups respectively) that are included in this sub-test.
As in the other ISTO/NVRA/NVRB items below, the stimulus image(s) are at the top of the screen and the five response options are at the bottom of the screen.
This is typical of the screen shown after the time limit has expired, without response, in any item of any sub-test.
The item itself (in this case, Space item 2, a Sequences example) is greyed out, and the message "out of time, click 'next' to continue" is superimposed over it.
Note that the time counter circle, top-right, is now blank.
The twin can no longer click on any of the five responses. Instead, the twin must click on the "Next" button in order to continue to the next item. In this case, the "Next" button appears bottom-right within the image of a yellow spaceship.
The stimulus, with "(?)" in place of the missing letter, is on the left.
The response options are the 26 letters of the alphabet, which are clickable on the right of the screen.
This is typical of the screen shown at the end of each of the five sub-tests.
The "stone" to be collected is the image bottom-left of the screen - clicking on this "collects" the stone and takes the twin back to the home screen, where the new stone will be shown on the dais.
The stimulus image, containing a blank where the missing pattern must go, is at the top of the screen. The eight clickable response options are at the bottom.
This is another typical example of the screen shown after the time limit has expired without response.
The item itself (in this case, Ocean item 3) is greyed out, and the message "out of time, click next to continue" is superimposed over it; the time counter circle is now blank; the "Next" button appears bottom-right within the image of a starfish.
This illustrates the question and response layout used in questions 1, 2, 3 and the QC item of this sub-test.
The stimulus statement, with "?" marking the missing word, is shown at the top of the screen.
The five clickable response options are shown at the bottom.
This illustrates the question and response layout used in questions 4, 5 and 6 of this sub-test.
The stimulus statement is shown on the left of the screen.
The eight clickable responses are shown on the right, in two groups of four. In order to respond, the twin must click on one of the top four responses (to complete the statement "heart is to ...") plus one of the bottom four responses (to complete the statement "as lungs is to ...").
This is another typical example of what a twin sees immediately after completing a sub-test. On clicking the green stone (bottom-left) the twin is taken back to the home screen where the stone will be added to the other stones on the dais.
Background images
The images reproduced below are background images for the items of the five sub-tests. They follow the themes of Mountain, Space, Tower, Ocean and Woodland which are the names of the sub-tests in the twin web interface. Some of these background images can be seen behind examples of sub-test items in the screen shots above.
Each background image was used for the introduction screen, for 1, 2 or 3 sub-test items, or for the end screen. The order in which the background images below are listed is the same as the order in which the backgrounds appeared in the test battery.
Also reproduced below are images of interface elements that were used as containers for the item stimuli and responses and which were superimposed on the backgrounds. In the images below, these have a white background but in the item images they were transparent so as not to obscure the background image.
Mountain (Vocabulary) sub-test
Space (ISTO/NVRA/NVRB) sub-test
Tower (Missing Letter) sub-test
Ocean (Ravens Matrices) sub-test
Woodland (Verbal Reasoning) sub-test
