Instructional Design Summit - Part 1: Active Learning in Large Enrollment Course: How peer learning and a flipped classroom engages 400+ students in authentic assessment
Concurrent Session 8
Brief Abstract
While large enrollment courses are a necessity in many large public universities, they are often seen as a challenge for faculty and students. This presentation will focus on one instructor at a large public university who reimagined the high enrollment course (400+ students) switching from exams to authentic projects and assessments.
Presenters
Additional Authors
Extended Abstract
Introduction
Faced with the obstacle of teaching at scale, faculty often rely on traditional methods and practices like lectures and multiple-choice exams to deliver content and measure student understanding. The shift from a traditional lecture/exam design to an authentic project-based pedagogy required multiple changes in strategy and a crucial piece of innovative educational technology. The result was a more engaged classroom where students not only learn but apply real-world knowledge and skills that are crucial to their future success. Audience members will leave this presentation with multiple strategies for employing peer learning and authentic assessment in large enrollment environments.
Explanation of presentation strategies
Presenters will use a number of different strategies to engage the audience and improve participation. It is estimated that the presentation will be 60% content delivery and 40% active learning participation. Audience members will participate in a simulated peer learning activity where they will have an opportunity to reinforce best practices that are outlined in the presentation. They will then have a chance to debrief the experience with a small group of other attendees before sharing out takeaways with the whole group.
Overview of course design
The course, Principles of Marketing, is designed to introduce students to concepts and language used by marketing professionals and to explore how marketing is employed to create value for customers and profit for firms. Students learn marketing terms and frameworks and evaluate how they are applied in current business examples. They further apply learned concepts by managing a simulated business, and, importantly, by describing how marketing is managed by a company of their choice in a series of written assignments.
Flipped classroom
The course leverages online ‘core lecture’ videos to create time in the classroom for in-class discussions, extensions of ideas, and guest speaker visits, among other things. In-class, real-time ‘quizzes’ are used both to motivate watching the online videos before class and to generate engagement in the class via opinion questions and other feedback. While the size of the class precludes some of the benefits of a fully flipped classroom, students are regularly provided with class time to begin developing their ideas for each written assignment.
Assignment types and sequence
An interwoven sequence of three major assignment types run through the course: (1) In-class lectures, supported by online video lectures, cover the course content. Students are motivated to engage and keep up via the use of daily in-class quizzes, (2) A business simulation game is played six times throughout the term. Each game focuses on managing a different input to the profitability of a coffee shop, and (3) Six writing assignments guide students to apply key concepts, as they are covered in class, to a company and product of their choice. These assignments culminate in students producing a marketing plan proposing how ‘their’ company could to do something new to improve its profitability.
Peer assessment technology
One of the biggest hurdles when teaching to this many students is determining how to assess their gain in knowledge and skills. Often instructors default to a scalable assessment method they are familiar with like multiple choice exams because grading authentic assessment requires time and resources that are not available in these larger courses. For this problem, the instructor turned to research-based peer assessment technology Peerceptiv. Peerceptiv leverages peer assessment to provide valid and reliable grades on rubric-based assignments without the need for instructor or teaching assistants to hand grade each student submission. A growing body of research supports the use of peer assessment not only as a valid and reliable method of assessment (Hongli Li et al 2016), but also as a teaching strategy that improves learning outcomes and skill development (Wu & Schunn 2020).
Peerceptiv employs a multi-step process of peer reviewing activities intended to ensure high-quality student engagement and consistent scoring outcomes. Six times throughout the semester students submit different formative phases of their projects to be evaluated by their peers based on well-defined quantitative and qualitative rubric prompts. At the end of each peer assessment assignment, students receive detailed scores on each rubric criteria and actionable feedback that helps them improve upon that portion of the project.
The Peerceptiv system also generates learning data and grades for the instructor to analyze at the end of each assignment. This data helps to inform where there are misunderstandings or knowledge gaps across the whole course or with individual students. The grades generated provide a measure of how well each submission performed according to the rubric as well as how each student performed as a reviewer. This type of grading incentivizes students to collaborate with one another at a high academic level and holds them accountable for the quality and specificity of their reviewing. Without this type of technology, the project-based pedagogy would be impossible to facilitate in a class of this size.
Student perceptions
Research indicates that students are more likely to be successful in courses where active learning methods are chosen in lieu of traditional lecture/exam pedagogies (Freeman 2014), but how do students perceive this shift? Results from students participating in this marketing course indicate that there is a strong preference for authentic assessments rather than exams. Students also tend to agree that the authentic assessments and the use of peer learning helped them to understand how to apply marketing concepts to a real-world context that was interesting to them. Other findings from student surveys suggest that most students feel that the grading is fair, although some mentioned that they would prefer instructor grading. Students also felt strongly that teamwork and team learning was an overall benefit to their learning.
Qualitative student responses about their experience were positive overall with comments like, “Having to do research from Peerceptiv assignments are definitely way more helpful for learning in this class than exams. Exams you just memorize a ton of information to forget later and these assignments I actually retain the information. It’s also nice we get to choose our products on things we are interested in!!!”. The few comments that were less positive tended to focus on specific aspects of the peer review process that a particular student did not like. “I really dislike the reviewing process, I think that it is a fine system. I just don't like leaving loads of comments for things that seem closely related. Most of the time I feel as if I am repeating myself in the like criticism/ reviews for people.” In general student comments focused on the benefits of participating in authentic assessments rather than exams.
Learning outcomes
Tracking and analyzing learning outcomes is often a challenge with authentic assessment. The types of work that students submit require human scoring and thus are not often utilized in large enrollment courses. One of the benefits of using a peer assessment technology is the data output from each assignment. For each of the six assignments in this course, learning data was generated about the quality of the student submissions by percentage grade and by rubric criteria. Peerceptiv also generates a rubric reliability metric that informs instructors how reliable the results are from each rubric criteria scored by peers.
Citations
Freeman S, Eddy SL, McDonough M, Smith MK, Okoroafor N, Jordt H, Wenderoth MP. Active learning increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci U S A. 2014 Jun 10;111(23):8410-5. doi: 10.1073/pnas.1319030111. Epub 2014 May 12. PMID: 24821756; PMCID: PMC4060654.
Hongli Li, Yao Xiong, Xiaojiao Zang, Mindy L. Kornhaber, Youngsun Lyu, Kyung Sun Chung & Hoi K. Suen (2016) Peer assessment in the digital age: a meta-analysis comparing peer and teacher ratings, Assessment & Evaluation in Higher Education, 41:2, 245-264.
Wu, Yong & Schunn, Christian. (2020). When peers agree, do students listen? The central role of feedback quality and feedback frequency in determining uptake of feedback. Contemporary Educational Psychology. 101897. 10.1016/j.cedpsych.2020.101897.