Student Perception of Content Master and Engagement in Using an e-Authoring Tool

,


Introduction
The demand for online courses continues to increase. Murray, Perez, Geist, and Hedrick (2012) predicted that the majority of student college students would be taking at least some of their courses online by 2014. Allen and Seaman (2010) investigated enrollment in online courses. The response data from their study of 2,500 universities indicated that the universities were experiencing a greater increased demand for online courses than for the traditional face to face courses they offered. Providing financial support for the development and implementation of online courses was an important consideration in universities' long-term planning. Data from these universities indicated that during the fall semester of 2004 approximately 2.3 million students were taking online courses; however, by 2008 that number had doubled to 4.6 million students.
Shrinking college budgets, increased competition, and the availability of Massive Open Online Courses (MOOC) have resulted in university concerns about obtaining and retaining student tuition dollars. However, while students are exposed to online material with extensive multimedia components, budget constrains at many universities have limited the availability of web course designers to construct courses with these same components. Consequently, greater demands are being placed on instructors and subject area experts, who do not necessarily have these advanced technical skills, to develop effective and engaging online courses that appeal to adult learners.
Through their study of 124 students, Ward, Peters, and Shelley (2010) identified adult learners' perception of the characteristics they felt were necessary for quality learning experiences. Adult learners identified ease of access and minimizing course costs as major considerations impacting their desire to pursue online learning. These considerations seemed to be fueled by concerns about the economic recession, rising fuel prices, job demands, childcare concerns, and other responsibilities (Cercone, 2008). Although adult learners enjoyed the convenience online courses provided, they indicated concern about the quality of interaction, learning, and instruction in the online environment.
Constructing well-designed online courses has become a major institutional focus, but without additional technical support instructors and subject area experts are faced with many hurdles in constructing these courses. However, as e-authoring tools have become more readily available, instructors can more easily design courses that provide opportunities to interact with, manipulate, and apply content concepts in a variety of settings. Instructors are no longer limited to attaching Word documents to a Learning Management System.
In this paper, researchers at a small, private, southern university attempted to improve the quality of their online programs by incorporating an e-authoring tool that allowed for easier navigation, purposeful addition of graphics, and interactive content activities instead of the previously used linear text-based documents. The study focused on student perception of the effectiveness of the e-authoring tool in expanding content and increasing student mastery.

Review of Literature
Literature has identified three major components online course developers need to consider when constructing online graduate courses: the characteristics of adult learners, the arrangement of course content, and the design elements that result in effective e-learning.
In the graduate education programs at the private university, most of the learners are adults between 25 and 50. Cercone (2008) stressed the importance of understanding the nature of adult learning, so that designers can better understand how to construct distance learning. L. Carver and C. Todd Cercone (2008) examined the needs of adult learners. She noted that, as learners mature, they experience age related changes in the ability to form links between new and previously learned information, to retain information in short-term memory, to chunk information for organizing learning, and to process visual information. Based upon these issues, Cercone stressed the importance of online designers using "large, easy to read fonts and clear, bold colors, a variety of graphics, images, and tables" (p. 140). She indicated the importance of chunking new information into groups of 5 to 9 pieces of information. She also encouraged the use of activities that enable learners to practice new material using self-tests that provide immediate feedback. She identified that adult learners need instructor scaffolding through completed examples and audio or video files modeling the instructional concepts. All of these adult learning concepts were missing from the previous Word document format used in the online courses at this private university. So subject matter experts began looking for a tool that would better meet the needs of adult learners.
In addition, designers needed to understand the components of quality e-learning. Much research has been done in this area. Siragusa, Dixon, and Dixon (2007) examined the components of quality e-learning in higher education. Their findings indicated the importance of online courses containing extensive content, learning activities, required assignments, and the materials needed to support learning. However, this information needs to be organized in ways to address a variety of learning styles while allowing for interaction and collaboration. Murray et al. (2012) analyzed through a survey format the responses of 67 students who were enrolled in online asynchronous courses. The data indicated that students were not satisfied with online courses that used "flat resources, in the form of static text documents" (p. 126). The researchers used frequency counts and access rates to determine access patterns for the different sections within online courses. They compared student access to four types of material: core material, direct support, indirect support, and ancillary materials. Data indicated that students were more likely to access the syllabus (core material) and the modules (direct support) rather than the indirect support or ancillary information located outside of the weekly module. Students tended to "retrieve only content that they perceive to be necessary to complete course deliverables directly tied to course assessment" (p. 137). Murray et al.'s findings also indicated that "interactive content that gives instantaneous and frequent feedback will be most likely to be perceived as relevant and useful to students" (p. 137). Interestingly, the reason given for not accessing other materials was time constraints. Consequently, these findings indicated that those materials subject area experts feel are important for building content knowledge need to be embedded within the module rather than placed in other sections of the course. Brown and Voltz (2005) specifically analyzed effective elements in e-learning. They indicated that online instruction needed to be more than document based distance learning. Instead, online instruction should include multiple types of text, images, and sound media for presenting the course content while addressing various learning styles. Two elements that Brown and Voltz (2005) pointed out as particularly important considerations in constructing online learning are: activity and feedback. They stressed the importance of providing opportunities for learners to be involved with the content in multiple ways. In addition, they mentioned that effective e-learning needs to include immediate feedback to enable learners to increase and expand their understanding and application of the course content. Both of these design components were missing from the original versions of the graduate online courses. Their research showed there were important components that needed to be considered when redesigning the online learning courses. As Gee (2003) specified, in our current technology driven environment "images, symbols, graphs, diagrams, artifacts, and other visual symbols are particularly significant" (p. 13).

Methodology
Online graduate courses at a small, private university in the southern United States had previously been constructed by placing within the Learning Management System a Word document for each module. The document in each module was composed of a narrative containing the content, the indicated outside readings, a description of the assignments, and links to additional materials the student might need to complete the assignments. Additional resources were placed within the course, but in ancillary locations outside of the course module.
Examination of student comments and end of course surveys from previous courses revealed two content design themes: request for additional graphics and visuals to support content and improved quality and opportunities to master content. Researchers wanted to investigate if incorporating SoftChalk, an e-authoring tool, would effectively address these expressed design concerns.

Participants
The contents of three different online graduate courses were redesigned using an online e-authoring tool, SoftChalk. The course designers for each of the three courses were subject area experts, rather than web designers. Although the e-authoring tool was not used following a prescribed method, each subject area expert intentionally chunked information in smaller units supported by visuals, graphics, audio content, and interactive practices that provided immediate feedback. In addition, they purposely located as much content as possible within the course module, rather than in other locations throughout the course.
The goal of each course designer was to increase learner interactivity, reflexivity, and critical thinking while expanding the course content without changing the course learning objectives. Each course was arraigned in 8 modules formatted using the e-authoring tool. These modules contained the majority of the course content. Information about the textbook readings, websites, online discussions, individual and group projects, papers, and/or exams were included within the module. In addition to the modules, PowerPoint presentations and links to additional videos and related materials were included in Doc. Sharing, an ancillary location outside of the course module. The redesigned courses were then attached to the university Learning Management System (LMS).
All students enrolled in five sections of the three online graduate education courses (N=81) were eligible to participate in an anonymous online survey. Seven students were enrolled in the Educational Specialist program and 74 were in the Master's programs in Reading or Exceptional Student Education. Responses were not disaggregated by program, gender, or age.
Responses were anonymous, but the sample was predominately female since most of the students enrolled in the chosen programs were female. University data indicated that the majority of the sample was employed full time as teachers in the K-12 system while they were working on their graduate degrees. These sections of students were chosen to be included in the study because this was the students' first exposure to the e-authoring tool. Consequently, they were more likely to be able to compare the new format with the Word document previously used with the Learning Management System.
The purpose of the study was to explore the students' perceptions of the effectiveness of the newly implemented e-authoring tool. A mixed-method survey design, which included Likert scales and qualitative responses, was utilized. Response data were collected at the beginning of the term through an external link outside of the course website to allow for voluntary participation in the study. Implied consent was demonstrated through completion of the survey. Student participation or lack of participation did not impact student grades.
Two research questions were developed to explore students' perception of the effectiveness of the e-authoring tool.
1. How effective will students perceive the e-authoring tool to be in helping them to better understand the content? 2. Which of the functions of the e-authoring tool will students perceive to be most effective in helping them to understand the course content?

Data Collection and Evaluation Framework
Data was collected using a mixed method: quantitative then qualitative approach. A quantitative Likert scale data-collection technique was used to analyze student perception of the impact of the e-authoring tool on their engagement with and master of the course content. Two stages of data collection were employed in this study. The first stage was the administration of an online survey assessing students' perception of the e-authoring tool as compared to the previous method used for constructing the online courses. The primary data collection was the student surveys. In the second stage, students' and course designers' qualitative comments were coded and analyzed for themes. Themes were identified based on the qualitative data from the comments on the surveys, course evaluations, and focus groups. These were used to confirm the survey data.
Two types of questions were included in the Likert survey. The first group of questions asked the students about their general impression of the e-authoring tool and its impact on their understanding of the course content. Three questions focused on student engagement and two questions asked students to evaluate their content mastery. The three questions relating to engagement focused on visual appearance, ease of navigation, and visuals and graphics. The content mastery questions asked about the effectiveness of chunking the information and expanding the course content. The second group of questions asked the students to rate the importance of each of the five components of the e-authoring tool. These components included the web-based visual appearance, pictures and graphics, interactive exercises and practice quizzes, table of contents, and navigational links within the module pages.
The results from the first group of questions about engagement and content mastery (Table 1) revealed that a large majority of the participants felt the expanded course content (85%) and the chunking of information within the module followed by the interactive reviews (80%) were beneficial in developing their understanding of the learning objectives. The responses to the engagement questions were not as strongly positive, but were still favorable. Three-fourths (75%) of the participants indicated that the visuals and graphics enhanced their engagement with the course content, while approximately two thirds (60%) of the participants found the visual appearance of the e-authoring tool more engaging than the previous Word documents. Slightly fewer than half of the respondents (45%) found the navigation within the e-authoring tool easier than the previous format, while slightly less than half (40%) were neutral in this area. It is interesting to note that the respondents who disagreed in any category did not exceed 10%. This would seem to indicate a generally positive response to the implementation of the new eauthoring system. Students were asked to rate on a three-point Likert scale the five components of the eauthoring tool (web-based appearance, pictures/graphics, interactive/practice quizzes, table of contents, and navigational links) in their perceived importance for increasing engagement and mastery of course content. The results (Table 2) revealed that most respondents (90%) indicated that interactive and/or practice quizzes were very important components in helping to understand the course content. Interestingly, although in the last set of questions respondents did not indicate that the navigational link was easier to use, the majority of the respondents (85%) rated the Table of Contents and the additional navigational links within the module as very important. Most of the respondents (80%) indicated that the addition of pictures and graphics to support content understanding was important. Also interesting, the component that was viewed as least important, although still viewed as important, by slightly more than two thirds of the respondents (70%) was the web-based visual appearance. Overall the quantitative data confirmed the researchers' supposition that the tool would be effective in helping students to better understand the course material/content. Second, quantitative data analysis revealed most students found the interactive practices and /or quizzes to be the most important component of the tool. While not conclusive or relational, this could indicate the use of the interactive practices and/or quizzes may contribute to students' enhanced understanding of the content and requires exploration. Student comments also provided valuable insight into the effectiveness of the online eauthoring tool. Analysis of student comments identified four themes: (a) the tool provided a format that was much more user friendly than the previous format, (b) the e-authoring tool was easier to navigate than the previous format, even though the same LMS was used, (c) the tool provided a way to chunk the information to create a more engaging learning environment for students, and (d) the interactive practices and quizzes helped students to evaluate their own understanding of the material. These themes supported the findings from the survey data. The qualitative data supported the work of Cerone (2008) that adult learners indicated a concern regarding the quality of online education and of Siragusa, Dixon, and Dixon (2007) whose findings revealed the importance of online courses containing extensive content and learning activities, among other indicators, organized to address a variety of learning styles allowing for interaction and collaboration. This is supported by the study results; 85% of the survey respondents reported the Table of Contents as an important component. Furthermore, the qualitative comments supported survey responses that tool makes the class easier to navigate and engaging. Open coding of student comments supported the survey responses; 90% of the respondents indicated interactive practices were important and 85% agreed the expanded module content provided enhanced understanding of the content. Both these items were rated highest by the survey respondents.
One student summed up the difference between the two formats with the following comment, "I wanted to let you know that I am in love with the way the information is presented in each module. I've not seen it before. The SoftChalk format is easy to use and navigate and I love that some humor is included. The little activities after each section are neat. This particular course seems quite interactive and I've not experienced that in an online course before!" Qualitative comments from subject area experts who were also course designers revealed some important observations. Their comments indicated the e-authoring tool was easy to learn and manipulate. They were pleased that they did not need any specialized training to manipulate the software or attach the modules to the LMS. This was particularly important because it was the first time any of the subject area experts had used the e-authoring tool. The subject area experts indicated that they appreciated that course content could be created in Microsoft Word, which was the format they had previously used. They were then able to copy and paste the information into the e-authoring tool which created the more engaging user friendly format. The designers also indicated that they appreciated the e-authoring tool's easy formatting and incorporation of visuals, videos, interactive exercises, and practice quizzes directly within the modules. The fact that the software created the Table of Contents and the interactive links within the sections of the module simplified the work for the subject area experts. As with student comments, open coding of instructor comments supported the student survey data indicating the interactive practices were effective for engaging students with the course material and course content mastery. Both faculty and students indicated the ease of using the tool as an important component. Faculty identified the ease of creating and uploading the course content as a positive while students identified easy of course navigation as a benefit.
The designers and students commented that the e-authoring tool allowed for expanded course content and learning activities presented using a variety of formats rather than just the "flat format" (Murray et al., 2012) previously used. The tool allowed for opportunities for the learners to interact with the content in various ways (Brown & Voltz, 2005). The use of this tool allowed the course designers to include "images, symbols, graphs, diagrams, artifacts, and other visual symbols" (Gee, 2003, p. 13) which are particularly important in the current technology-based environment. Based on the findings of this study, use of SoftChalk will be expanded to include other course designers, courses, and programs within the university in an attempt to improve course delivery and student learning.
The institution in which the study was conducted is a teaching college whose mission is to "offer a practical, effective model for life and leadership in a challenging world" and to accomplish its mission, the University community creates a student-centered environment in which the love of learning is of prime importance (Saint Leo Catalog, 2013, p. 1). Preliminary data analysis appears to confirm the e-authoring tool supports a student centered environment and may play a role in helping the University achieve its mission and foster students' love of learning. As the ESE, Reading and Education Specialist programs are only offered to students online, it is critical the online format is instructive, engaging and connects students with the course content. Results appear to do so. Further analysis is needed to explore the relationship between of student perceptions of the tool and student achievement, as measured through the mastery of standards, course grades and end of course evaluations.

Further Research
The qualitative and quantitative data support the addition of an e-authoring tool to an online module as an effective method for increasing students' perception of their engagement with and mastery of the course content as compared to the previous more static Word format. The students identified visuals, graphics, interactive exercises, and practice quizzes as particularly important components for helping them master the concepts presented in the online courses. The data indicated that all components of the e-authoring tool should be used when constructing future online courses as about two-thirds or more of the respondents indicated each component was important. The findings from this study will help to guide further course development.
Although the findings supported the addition of the e-authoring tool, this study has many limitations. The sample population of the current study was limited to graduate students in three disciplines at a small university. When discussing construction of online courses, a much broader population needs to be considered. A larger study focusing on a sample of graduate students from other fields of study and undergraduate students from a variety of different disciplines would offer results that could be more easily generalized to a larger population. Additional research should be conducted with a wider sample of students from differing demographics. A different sample population might yield different results. A study that disaggregated the data by program, gender, and age might provide additional information that would be helpful in improving and refining online course construction. In addition, it would be interesting to know if there is a correlation between students' technology proficiency and student satisfaction with the e-authoring tool. This study focused on students' perception of their content knowledge rather than attempting to quantify knowledge growth. Further study could also focus on measuring growth in student content knowledge in online courses using the e-authoring tool rather than just students' perception of their learning or of the tool itself.
Although informative, this study has only examined a few of the many possible factors that may impact online course construction and student content mastery. Many other factors need to be considered to develop a more thorough understanding of this topic.