Validating an Online Learner Readiness Instrument: From Readiness to Self-reflection
Brief Abstract
This session will feature researchers from two universities collaborating on the development and evolution of a tool to measure online learner readiness. Panelists will share an analysis of the readiness scale, as well as a reconceptualization and implementation of this tool to support 21st century learners.
Presenters
Extended Abstract
As different learning modalities have become more common during the pandemic (e.g., online, hi-flex, hybrid), higher education personnel are increasingly interested in factors that prepare students for learning. While several measures specifically assessing online learner readiness are currently available, most were developed over 5 years ago, and include outdated questions about technology. Our study sought to fill this gap by adapting Dray and colleagues’ (2011) online learner readiness scale in combination with consultation from student success professionals and academic advisors.
This panel will feature researchers from the two collaborating universities. They will share ways in which this instrument has evolved from past research, the results of their validation study, and how this instrument is being reconceptualized as a tool to support undergraduate learners. Panelists will discuss the application of this instrument as a tool for: 1) learner self-assessment; 2) connecting students with resources and support services; and 3) helping student support coaches identify opportunities for students to further develop learning skills.
This presentation will include three sections: 1) online learner readiness scale development and key findings; 2) continuity and change in the readiness scale over time; and 3) a reconceptualization and application of the readiness scale. At the end of all three sections, participants will first be asked to respond to questions on google slides. In part 1, they will be asked to write down their thoughts and ideas about the readiness scale, and the ways in which it can be applied at their institutions to promote learner success across a range of modalities. Key themes of their responses will be discussed among the panelists. In part 2, participants will be asked to identify barriers in using this readiness scale at their own institution and key themes will be discussed among the presenters. Finally, in part 3, participants will be asked to write a few ideas about how they might use a readiness scale for student support in their current role. The google slides will be available to the participants after the presentation. If there is a capacity in the online platform, participants will be asked to verbally share their thoughts during these activities.
By attending this session, attendees will be able to:
- Describe how online learner readiness applies to learners more broadly
- Describe the types of questions used in a readiness measure.
- Discuss the application of learner readiness tools to promote self-reflection and support seeking among undergraduate students.
About the Study
Two universities collaborated to revise and update the online learning readiness instrument, focusing specifically on measuring students’: 1) self-efficacy; 2) locus of control; 3) skills within the context of course work; 4) communication with students and instructors; 5) challenges with time, coursework, and outside commitments (work, family); and 6) technology efficacy.
The scale was tested at institution A with 853 newly enrolled online students, and then cross validated at institution B with all enrolled online students (n = 9408). Exploratory factor analysis was used to identify and remove items that were not reliably related to each subscale. Confirmatory factor analysis was used to validate the reduced instrument, a 25-item scale. Results of the confirmatory factor analysis revealed four subscales with adequate reliability scores for both institutions.
Newly developed items for this measure fit well into the communication efficacy and self-regulation efficacy subscales, suggesting that these dimensions continue to play an important role in learner readiness (e.g., Jansen et al., 2016; Sun & Rodgers, 2020). In addition, the continued emergence of locus of control and technology efficacy suggests the dimensions of online learner readiness have remained remarkably stable over time (e.g., Zimmerman et al. 1992; Hung et al., 2010).
As the lines between face-to-face and distance learning continue to blur, the results suggest that “online learner readiness” might be more broadly applicable to all learners in higher education. Consultations with key stakeholders (e.g., student success coaches, academic advisers, senior administrators) further suggest that- beyond a diagnostic tool to evaluate student “readiness”- our instrument might best serve students as an opportunity for self-reflection throughout their academic tenure. In collaboration with academic advisors and student success coaches, this instrument could further be used to provide students with personalized support services and resources as they move through their learning journey.
Beyond “readiness”, the reconceptualization of this instrument as a “learning self-reflection tool” might best serve 21st century students in higher education, as well as the stakeholders who support them.
Therefore, the revised instrument was re-named the Learning Self-reflection Tool (LSRT). At institution A, the LSRT is being built into the online orientation for online students. Upon completion of each subscale, students will be directed to specific resources related to the types of skills. Success coaches and advisors are planning to use the instrument as a tool to talk about skills for successful online learning.
At institution B, the LSRT is being built into a website. Upon completion of the four subscales, students will receive an individualized report with a sub score for each subscale and will be directed to resources specific to each subscale. Students will also have the choice to schedule time with a student success coach, a new program under development.
References
Dray, B. J., Lowenthal, P. R., Miszkiewicz, M. J., Ruiz-Primo, M. A. and Marczynski, K. (2011). Developing an instrument to assess student readiness for online learning: a validation study. Distance Education, 32(1), 29-47.
Hung, M-L., Chou, C., Chen, C-H., & Own, Z-Y. (2010). Learner readiness for online learning: Scale development and student perceptions. Computers and Education 55 (3), 1080-1090. http://dx.doi.org/10.1016/j.compedu.2010.05.004
Jansen, R. S., van Leeuwen, A., Janssen, J., Kester, L., & Kalz, M. (2016) Validation of the self-regulated online learning questionnaire. Journal of Computing in Higher Education. https//doi.org/10.1007/s12528-016-9125-x.
Sun, Y., & Rogers, R. (2021). Development and validation of the Online Learning Self-efficacy Scale (OLSS): A structural equation modeling approach. American Journal of Distance Education, 35(3), 184-199.
Zimmerman, B. J., Bandura, A., & Martinez-Pons, M. (1992). Self-Motivation for Academic Attainment: The Role of Self-Efficacy Beliefs and Personal Goal Setting. American Educational Research Journal, 29(3), 663-676.