Is Anyone Watching? Using Analytics to Inform Course Design

Concurrent Session 1

Session Materials

Brief Abstract

Many of us create personalized videos and use module overviews and wrap-ups in our courses. However, are students even viewing them? If not, how can we increase these views? In this session, we will discuss how to use analytics to determine how and where to share these resources.

Presenters

Lead Presenter: Sara Ombres, Embry-Riddle Aeronautical University

Sara has over ten years of teaching, training, and instructional design experience in the online, blended, and face-to-face modalities. This includes two Blackboard Exemplary Course Awards for excellence in instructional design, and extensive experience teaching and working with college students as well as facilitating professional development offerings to faculty. She holds an M.Ed. in Educational Technology and an M.S. in Management and currently works as an instructional designer and adjunct online instructor.

Extended Abstract

Instructors and instructional designers are often seeking to humanize and personalize the online course environment through lots of instructor presence and customized course videos. While these multimedia assets are often time-consuming and labor-intensive, they are created with the goal of increasing student engagement and sense of community in the course.

It is also a common design best practice to provide module overviews and wrap-ups in online courses to introduce the module topics and objectives and then to summarize and conclude the module at the end. Another recommended best practice is to follow a similar format across courses in a program so students are familiar with the design and can focus on the content. However, is this consistency leading to complacency?

While working with a faculty member on an initiative to increase the instructor presence in the online classroom, we looked at ways to create more placeholders and opportunities for instructors to create and share their own personalized videos within the structure of the course design template. There were already some videos that existed in the course template on the module overview pages and other pages throughout the course, so we began by investigating the analytics of those video and page views. The results were surprising – student views of the module overview and module wrap-up pages had dropped substantially over the course of the 9 weeks of the term. Even fewer of those students were viewing the videos within the pages. While these initial data points were disappointing, we used them as an opportunity for change. Clearly, the design answer was not MORE videos, but instead more strategically placed videos as well as ensuring the video content was relevant and customized.

In our first pilot, we removed the standard module overview videos and added an “Instructor’s Corner” placeholder for the instructor to add his or her own customized videos specific to that course and that term. We also reduced the number of videos shared on other pages to emphasize the importance of these videos and the value in viewing them. However, the results of this pilot still were not as desired. The major challenge was still encouraging students to view the overview page itself where the videos were held.

In the second round of the pilot, we set up module pre-requisites where students could not view the module content until the module overview page was at least viewed. While this did not require students to view the instructor-created video on the page, it still required them to open and view the page. The results from this pilot were very encouraging. There was a dramatic increase in the number of video views by students, thus increasing their engagement with the instructor in the course content.

In this session, we will cover the process of this pilot, as well as the rationale behind it and the analytics utilized throughout to help drive the design decisions made. We will also discuss and address the research-based best practices around instructor-created videos to ensure students are receiving quality content. Finally, we will discuss how instructional designers can and should use data and analytics as part of design discussions with course developers to ensure informed design decisions are made.

By the end of this session, participants will be able to:

  • Discuss how to use analytics to inform course design.
  • Explain research-based best practices for instructor-created videos.

In addition to the presenter-led presentation component of this session, there will also be audience interaction and participation through small group discussion and the use of Padlet or a related tool to organize participant thoughts and ideas.