Cognitive Presence In Online Learning: A Systematic Review Of Empirical Research From 2000 To 2019

Concurrent Session 3

Brief Abstract

This systematic review synthesized research on cognitive presence in online learning from two decades (2000 to 2019) of scholarship. From initial search of 181 studies, a total of 30 articles were selected for review. Results show that the majority of the studies were carried out in higher education in the United States and Canada within the field of Education. Almost half of the studies used mixed-methods research design, of which discussion transcripts were the prominent method for data collection and content analysis was used the most to analyze the data. Research focus of these studies was mainly on instructional strategies and educational outcomes. In addition, the majority of discussions fell into exploration and integration phases of cognitive presence.

Presenters

Extended Abstract

The last decade has seen an increase in the number of online courses (Seaman, Allen, & Seaman, 2018). However, there are still challenges with online learning and quality in online courses is still of concern (Xu & Xu, 2019). Cognitive presence is an important indicator of quality of an online educational experience since it consists of deep and meaningful approaches to learning. In search of creating cognitive presence to maximize the quality of online learning, scholars have been studying cognitive presence from a variety of aspects. These include effective instructional strategies (Chen et al., 2019; Costley, 2016; Sadaf & Olesova, 2017), learning environments (Alavi & Teghizadeh, 2013; Archibald, 2010), student learning outcome (Galikyan & Admiraal, 2019; Redmond, 2014), and relationship of cognitive presence with other presences (Lee, 2014; Rolim et al., 2019). As such, these research studies provide guidance in creating and assessing the quality of cognitive presence in online learning. Due to the growth in number of studies exploring different aspects of research on cognitive presence, there is a need to synthesize this existing research.

Since cognitive presence has shown to support higher-level learning in online courses and has been examined from various perspectives, it is important for researchers and educators to reflect on cognitive presence research topics, methods, and trends. Thus, the purpose of this review is to gain a deeper understanding of the current research trends of cognitive presence in online learning. In this review, the following questions guided our inquiry:

  1. What are the publication trends of cognitive presence research in online learning? (Journals and years of publication, number of articles published, journals that publish cognitive presence research)
  2. What is the context of published cognitive presence research? (Instructional setting, countries represented, and subject areas represented)
  3. What research design, data collection, and data analysis methods are used in the studies reviewed?
  4. What is the focus of research on cognitive presence studies in online learning?
  5. What is the distribution of cognitive presence phases in online courses?

Theoretical Framework
Garrison, Anderson, and Archer’s (2001) Community of Inquiry (CoI) was used as the theoretical framework. The CoI model explains the concept of critical thinking in online collaborated learning environments, through cognitive presence, social presence, and teaching presence. Garrison, Anderson, and Archer (2001) defined cognitive presence as “the extent to which learners are able to construct and confirm meaning through sustained reflection and discourse in a critical community of inquiry” (p.11). CoI is grounded in PIM that provides a framework to assess the levels of cognitive presence. The PIM is based on the four phases of cognitive presence: (1) Triggering- become aware of a problem; (2) Exploration- explore a problem by searching/offering information; (3) Integration- interpretations/construction of possible solution, and (4) Resolution- providing potential solutions.

Methods
In this study, we used the five-step systematic review process described in the U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse Procedures and Standards Handbook, Version 4.0 (2017). The five steps included (a) developing the review protocol, (b) identifying relevant literature, (c) screening studies, (d) reviewing articles, and (e) reporting findings.

Data Sources and Search Strategies
Six databases were searched using the search terms “Cognitive Presence’ and “Online” for published articles between the years 2000 and 2019 using both the Title, Keyword and Abstract search function. The six databases searched included Academic Search Complete, ERIC, Library, Information Science & Technology Abstracts with Full Text, PsycINFO and Science Direct.

Inclusion and Exclusion Criteria
From the initial search, 181 articles resulted. These articles were screened both at the abstract level and full-text level based on the inclusion and exclusion criteria. This resulted in 30 articles, which were coded for the systematic review. The PRISMA flowchart created by the Ottawa Methods Center for conducting systematic (Moher et al., 2009), was used to document the process flow of identifying the studies to include in this systematic review.

Data Coding and Analysis
A review protocol for coding was developed in Microsoft Excel. Items coded for article description, research design, data collection methods, data analysis methods, research topic focus, and cognitive presence phase included in the review protocol. Three researchers coded the articles independently using the review protocol. One of the researchers was an instructional technology faculty and the other two researchers were doctoral students in the department of Educational Leadership. The researchers met weekly to discuss the codes. Interrater reliability was calculated and the agreement among the three researchers was 90%. The areas of disagreement were discussed before further coding. Descriptive statistics were generated to show the patterns and frequency of the variables of interest. Narrative data for research focus and instructional strategy were analyzed using content analysis to identify categorical themes.

Results
RQ 1. Publication Patterns
Since 2000 to 2019, cognitive presence research studies were published in 17 journals. The largest percentage of articles on cognitive presence was published in The Internet and Higher Education (n = 8, 26.7%). This was followed by American Journal of Distance Education (n = 3, 10%), Journal of Computing in Higher Education (n = 3, 10%), British Journal of Educational Technology (n = 2, 6%), and Online Learning (n = 2, 6%). There were twelve other journals that published only one article on cognitive presence. Overall, the number of the articles on cognitive presence has increased every five years with 40% (n = 12) articles published from 2015-2019, followed by 30% (n = 9) articles published during 2010-2014 and 26% (n = 8) during 2005-2009.

Context of Published Research
Of the 30 articles, the majority of the studies were conducted in Higher education (n = 27, 90%). Only two studies were conducted in K-12 and one in Healthcare. Of the ten countries represented in the 30 articles, most of them were carried out in the US (n = 10, 33.3%) and Canada (n = 9, 30%). About one to two articles were published in other countries, including Korea, Spain, UK, Australia, China, Iran, Netherlands, and Taiwan. The top five most common content areas included Education (n = 9, 30%), Engineering (n = 5, 16.7%), English (n = 4, 13.3%), Medical/Health (n = 3, 10%), and Multiple subjects (n = 3, 10%). There were five other content areas, including extended studies program, human resource development, human sciences, research methods, and technology, that published only one article on cognitive presence, and one did not report the content area.

Research Methodology
For research design, we categorized studies into three types, including quantitative, qualitative and mixed-method studies. Out of 30 studies, most (n = 16, 53.3%) studies used mixed-methods research design followed by quantitative (n = 9, 30%) and then qualitative (n = 5, 16.7%).

Among the various data collection methods, online discussion forum was employed in 25 studies (83%). Among these studies, 17 studies used online discussion posts as the sole data collection method while 8 studies combined online posts with other data collection methods. The second frequently used data collection method was survey/questionnaire (n = 5, 17%). Four out of the five studies used survey/questionnaire together with other data collection methods. Regarding the studies that employed interview as the data collection method (n = 5, 17%), about 4 studies combined interview with other data collection methods. Relatively less used data collection methods include observation, course grade, course materials, instructor reflection, and the process data collected from the online platform.

For data analysis, content analysis was used the most (n = 25, 83%), followed by inferential statistics (n = 15, 50%), descriptive statistics (n = 11, 37%), network analysis (n = 5, 17%), thematic analysis (n = 3, 10%), constant comparative analysis (n = 2, 7%), and behavior sequential analysis (n = 1, 3%). It should be pointed out that six studies (20%) employed the descriptive statistics, inferential statistics, and content analysis simultaneously.

Research Focus
The research focus was coded using inductive coding and was divided into four categories. About half of the studies focused on instructional strategies (n = 15, 50%) and some on educational outcomes (n = 7, 23%) in online courses. A few studies examined learning environments (n = 4, 13%) such as exploring aspects of an online course or virtual platform, and some studies focused on the relationship between cognitive presence and teaching/social presence (n = 4, 13%) such as testing relationship between the elements of social and cognitive presences in a community of inquiry.

The authors of the primary studies used a variety of research methods to examine various research topics. Among studies on instructional strategies, 7 of 30 studies used mixed methods, 15 used quantitative research and only 3 used qualitative research methods. In studies about educational outcomes, 4 studies used mixed methods, 2 studies used qualitative and only one study used qualitative research method. Among studies focused on learning environments, two used mixed methods, only one used qualitative and one quantitative research methods. However, in studies focused on cognitive presence relationships, three studies used mixed methods and only one employed quantitative method.

In terms of specific research focus, the most published research topics within instructional strategies were reflection on practice (n = 7), case-based learning (n = 5), inquiry-based learning (n = 5), and peer-facilitation (n = 5). On the other hand, least researched instructional strategies were article critique (n = 2), instructor facilitation (n = 2), and invited expert (n = 2). Within educational outcomes, most published research topics were cognitive presence levels (n = 6), critical thinking (n = 5), interaction (n = 5) and the least focus was on knowledge construction (n = 1).

Distribution of Cognitive Presence Phases
Out of 30 studies, 25 studies used PIM with four phases of cognitive presence for content analysis of students' online discussions transcripts. However, two studies did not specify the percentage distribution of cognitive presence phases. When examining the range of cognitive presence in the studies, the majority of students' messages were classified as exploration (19 - 63%) and integration (9 - 74%), and smaller percentage as triggering (3- 31%) and resolution (0 - 36%). Eleven of 25 studies coded messages that didn’t fit any of the four phases of cognitive presence as “other.”

Scholarly Significance
The review revealed several interesting trends that can inform scholarship on cognitive presence and promote reflection about where we are and where we need to go. First, despite the evidence that research on cognitive presence is growing, the number of articles published each year is relatively small. The underscores the need for more research on cognitive presence for effective online learning. Second, most of the studies were carried out in higher education in the U.S. and Canada within the field of Education. Third, almost half of the research studies used mixed-methods research design, of which online discussion transcript was the most prominent method for data collection and content analysis was most common method of analysis. Finally, instructional strategies (reflection on practice, case-based, inquiry-based, and peer-facilitation) and educational outcomes (cognitive presence levels and critical thinking, and interaction) were the most researched topics.

This review has implications for future researchers. By understanding the research gaps in existing research, researchers will be equipped to build upon and extend the research on cognitive presence to date addressing novel instructional strategies. In addition, the findings have implications for designers and instructors of online courses. They can better understand topics that are more extensively investigated as well as the results of those that are insufficiently researched to create cognitive presence capable of supporting high-level online learning. The findings reinforce the importance of including various strategies such as reflection on practice, case-based learning, inquiry-based learning, peer facilitation, debate, project-based learning, collaborative learning, role play, instructor facilitation, and roles assignment as instructional strategies to enhance cognitive presence in online courses.