Project - Developing and evaluating novel, Web-based, gamified cognitive assessments for older adults
Research Project Abstract
Emerging evidence highlights the potential benefits of assessing cognition digitally rather than using traditional, paper and pencil neuropsychological tests. The reliability of a single, lengthy (>4 hours), face-to-face assessment can vary as a result of stress, sleep quality, and mood as well as the variable ecological validity of the tests themselves. In contrast, it has been shown that brief and frequent cognitive assessments (i.e. burst testing) delivered via a smartphone demonstrate excellent reliability and validity in community-dwelling older adults, suggesting they may be sensitive to neurodegeneration in preclinical dementia. Therefore, in this study, we are developing gamified mobile cognitive tests to asses and track cognition in older adults, with the aim being able to detect the earliest signs of cognitive decline, thus facilitating therapeutic intervention.
Challenges within the field
Preclinical dementia occurs 10-20 years before symptoms first present clinically, preceding subjective cognitive impairment, objective evidence of cognitive decline, and functional changes. The gold standard neuropsychological tests typically used in clinical practice lack sensitivity to the subtle cognitive changes associated with preclinical dementia. Thus, there is an urgent need to develop more sensitive cognitive assessments, particularly in memory and executive functioning, to detect the earliest signs of cognitive decline and thus facilitate therapeutic intervention.
Research Project Description
Emerging evidence highlights the potential benefits of assessing cognition digitally rather than using traditional, paper and pencil neuropsychological tests. The reliability of a single, lengthy (>4 hours), face-to-face assessment can vary as a result of stress, sleep quality, and mood as well as the variable ecological validity of the tests themselves (Chaytor et al., 2003). In contrast, it has been shown that brief and frequent cognitive assessments (i.e. burst testing) delivered via a smartphone demonstrate excellent reliability and validity in community-dwelling older adults, suggesting they may be sensitive to neurodegeneration in preclinical dementia (Hassenstab et al., 2018). Digital tools, such as a digitizing pen, extend data capture to include features of an individual’s approach to a given task (e.g. pen strokes, organisational strategies, and hesitations) as well as deficits in efficiency and information processing (Alzforum, 2017). While digitizing tests may facilitate reliable assessments outside of the clinic setting, web-based tests remain flawed as they are often boring and repetitive, thus leading to participant disengagement and discontinuation (Lumsden et al., 2016).
Applying Gaming Principles to Personal Health and Wellbeing
The ability of mobile digital games to achieve player bases numbering millions of people (Coutrot et al., 2018) points to the engaging properties of digital games and their usefulness for research and assessment. As older adults aged ≥65 are the fastest growing segment of digital game players in Australia (Brand et al., 2018), applying game design principles to cognitive tasks seems practical, particularly for brief, frequent cognitive assessments. A systematic review of gamified cognitive assessment and training paradigms also found evidence suggestive of associated improvements in engagement, intrinsic motivation, and training outcomes (when relevant) (Lumsden et al., 2016; Savulich et al., 2019). In addition to promoting repeated engagement, gamifying cognitive tasks can improve usability, decrease test anxiety, and increase ecological validity. If done carefully, beneficial outcomes of applying game design principles, can be achieved without invalidating the gamified cognitive tasks.
Methodology and Timelines
Development of cognitive assessment tasks (in process): Novel digital assessment measures are currently under development, including tests of processing speed and cognitive inhibition, using a traditional Stroop paradigm, as well as measures of verbal and visual learning and memory.
User testing (mid-to-late 2020): Three iterative cycles of one-on-one, 90-minute, ‘think-aloud’ user testing sessions will be conducted face-to-face with 15 community-dwelling adults aged 50-80 years, with five from each decade, to gather qualitative data for collation and knowledge translation in collaboration with software designers. Each cycle will test successive iterations of the paradigms that will be refined following end user feedback. Participants will also be asked to answer brief self-report questionnaires about usability and meaningfulness when completing the cognitive assessments. Furthermore, analytic data relating to the visual behaviour of users (i.e. swipes, clicks, pages viewed, etc.) will also be collected during this phase.
Research translation and future projects:
If preliminary data shows these assessments to be an effective means of assessing and tracking cognition in older adults, we will leverage additional funding for more rigorous assessment of their potential effectiveness (reliability, validity, sensitivity, specificity, etc.) relative to both gold standard neuropsychological tests and well-established web-based cognitive assessments, via the Healthy Brain Ageing Clinic, which recruits approximately 120 older adults (≥50 years) annually for a comprehensive neuropsychological and medical assessment. Additionally, the applicability of the tests across other potential user groups (e.g. testing for suicidal thoughts and behaviours, and common mental illnesses such as anxiety and depression, thus aligning with expertise of the Black Dog Institute) will be explored.
We aim to develop three, Web-based, gamified versions of cognitive tests, with the intention of building lightweight and flexible paradigms that can be deployed across multiple platforms. Specifically, we aim to:
- develop gamified mobile cognitive tests to assess and track cognition in older adults;
- pilot, evaluate and refine these tests according to user feedback collected via three iterative cycles of user testing with the target end user population (i.e. older adults); and
- conduct preliminary analyses of user behaviour via heatmaps (e.g. clicks, taps, scrolling behaviour collected via online analytics tools) to investigate factors associated with user motivation, engagement, and compliance.
- the delivery of easily accessible, cost-effective, and engaging tests of cognition that can be completed routinely at home via the Internet, and
- qualitiative and quantitiative feedback from user testing sessions to inform ongoing development and refinement of the gamified cognitive tests to detect the earliest signs of cognitive decline.
Key Publications from this project
Infographic / Medical Diagram / Scientific Diagram / Picture
Development is underway.