Carmen Canvas is the name for the Canvas LMS (learning management system) instance used for academic courses at The Ohio State University. They generically call the learning management platform ‘Carmen’ for conistency accross the years. So that one LMS can be swapped out with another and still retain the name ‘Carmen’.

Camen first started in 2005 when the university switched from using the WebCT learning management system to using the Desire to Learn platform. Later, in 2016, the LMS was swapped out with the Canvas learning platform but the ‘Carmen’ brand was retained. Carmen Canvas is the regular version of the Canvas LMS, while Scarlet Canvas is the Canvas Catalog product version used for public-facing enrichment courses.

Educators can work with an instructional designer or a Canvas admin to develop online courses in Scarlet Canvas. Some online course development steps for Canvas courses are:
Course planning and development:

• Fill out the course development planning worksheet to better define your course audience, content, and objectives. There are Qualtrics and MS Word versions of the course planning worksheet available.
• Use the backwards design methodology thinking of learner outcomes first. Outline learning objectives for the course. Learning objectives are typically listed in the introductory pages of the first module.

Course building in Canvas:

• Setup course modules and pages. Embed video content from MediaSite checking that closed captions are working. A good practice is to include a ‘meet your instructors’ page in the first module with educator bios.
• Quizzing – Decide on a quizzing / learning checks strategy. Many courses include a short quiz at the end of every module that unlocks the next module on a passing score. Some courses have longer pre and post course assessments in the first and last modules. Questions can be stored in question banks and imported into various module quizzes. Studies have shown that taking quizzes helps participants better retain what they learned.
• Decide if including learning objects from H5P would enhance learning in the course. Participants have different learning styles. Some types of learning objects might help them retain more knowledge.
• Course evaluation – Setup course evaluation surveys in Qualtrics and add links in the Canvas course. Typically a link to a course evaluation is added in the last module, the course completion module.
• Decide on how to issue course completion certificates to participants. If using automated certificates from Canvas catalog, turn them on in the course listing settings and paste in the certificate HTML template. Tweak the HTML template to fit the specific course.

Reviewing course pages for accessibility, branding compliance, and educational experience quality assurance.

• Review course page accessibility using resources like the WAVE tool, the internal Canvas accessibility checker, and/or Lighthouse tool in Chrome.
• Review course for branding compliance and educational experience quality assurance.
• Add instructors to the course and give them the Teacher role in Canvas.
• Share the course shell with a small cohort of educators and do a peer review.
• Analyze feedback from the peer review and decide what changes to make. Keep technical changes in a changelog.txt text file in the course files.
• Test changes made with a test student account. Check that course registration and payment are working as expected. Setup any special promo codes and test that they work.

Course launch, marketing, and post-launch monitoring:

• Make final changes, open the course for enrollment, and promote the course in social media and other outlets.
• Monitor how the course is working out for participants. Review feedback from participants and instructors for ideas to improve the course. Note small technical improvements and fixes in the changelog.txt text file.

Scarlet Canvas is Ohio State’s public-facing LMS platform for non-credit, professional development, continuing education, and enrichment courses. This learning management platform is powered by the Canvas Catalog product from Instructure which can provide automated course completion certificate issuing and public course listings in a course catalog. Canvas courses in the system may be unlisted or publicly listed in the catalog. Ohio State Extension uses the Scarlet Canvas LMS for many of its enrichment  and public outreach courses. Instructors and learners can login into Scarlet Canvas at: https://scarlet.instructure.com/.

See these Canvas Links & Resources for more information about working with the Canvas LMS.

Cognitively the act of taking a quiz, calling up knowledge from memory, makes that memory stronger and easier to access. So students who are more frequently quizzed retain more knowledge of the subject they are studying. Retrieval practice, such as with flash cards and practice quizzes has been found to be a powerful study method. Quizzes are beneficial learning check exercises to include in online courses as well. In academic courses it’s a common expectation that some of the assignments will be quizzes. Drafting quiz questions is a common checklist item on a course quality QA guide.

Contents:

• Benefits of using quizzes in online courses
• Enhancing quizzes as a learning opportunity with feedback and reflection
• Using quizzes as a method to ensure video lessons are fully watched by students
• Recommendations for use of quizzes in Extension courses
• Canvas LMS quizzing resources

Here are some of the benefits of using quizzes in online courses:

• Retrieval practice occurring during quizzes can greatly enhance retention of the retrieved information. An even higher level of retention than from restudying or rereading the course material.
• Quizzes permit students to discover gaps in their knowledge and focus study efforts on difficult material.
• An indirect effect of quizzes was found that if quizzed frequently, students tended to study more and with more regularity.
• Quizzing has been found to enable better metacognitive monitoring for both students and teachers because it provides feedback as to how well learning is progressing. Quizzes can be a beneficial self-learning check for students.
• Every time a student calls up knowledge from memory like when taking a quiz, that memory solidifies becoming more stable and more accessible.

A Scientific American article noted that “research in cognitive science and psychology shows that testing, done right, can be an effective way to learn. Taking tests can produce better recall of facts and a deeper understanding than an education devoid of exams².” In the academic world more frequent quizzing leads to better grades and academic performance. Practice quizzing in the academic realm can reduce high-stakes test-taking anxiety in students. In the continuing education and enrichment space the key benefit is better retention of useful knowledge and quizzes become lower stakes learning opportunities.

Enhancing quizzes as a learning opportunity with feedback and reflection

In order for a quiz to effectively function as a learning opportunity students need to receive feedback: information about what they got right and what they got wrong. So it’s important to thoughtfully setup feedback in online course quizzes. Learning management systems, like Canvas, have the ability to show feedback on both correct and incorrect answers. See this what options can I set in a quiz article for more information about setting up automatic feedback on question responses.

Getting students to engage in reflection after a test also helps reinforce what they learned. Some instructors use a quiz evaluation/reflection exercise called a ‘test wrapper’². In it students are asked how think they could have studied better and if there’s any areas the instructor could help with.

Using quizzes as a method to ensure video lessons are fully watched by students

Having a quiz about a video is my instructional design colleague Amy’s recommended way of encouraging participants to watch full video lessons. Here’s some more info about Using Canvas quizzes as learning checks to assess that videos are watched. After the video in a course module the instructor can place a quiz to act as a learning check. Then assign a passing score that’s required to open the next module.

Recommendations for use of quizzes in Extension courses

Low-stakes quizzes are useful in Extension courses for ensuring video lessons are fully watched and getting participants to think about the material they just viewed (thereby retaining more knowledge from the course content). Some participants occasionally choose to not watch whole videos lessons missing out on key knowledge. A quiz acts as a self-learning check for them to reflect on what they learned and a method to assess they are ready to move on to the next lesson. Many Canvas LMS courses use a passing score on an end-of-module quiz to unlock the next module. It provides a time to pause and reflect on what was learned to wrap up that module lesson before jumping into the next lesson.

Since an online Extension course is lower stakes than an academic course, it’s a good idea to kept quizzes less formal, with a lighter tone. Quizzes don’t have to tedious, they can include questions designed to be fun and crafted for the intended age group. For example questions written for a 4-H age group might be different in tone than those written for an adult class. Using fun question types can make a quiz more like edutainment than an academic assessment. Both Canvas LMS quizzes and H5P learning objects have question types than can be used to create entertaining and engaging exercises. Taking a lighter, edutainment approach to quizzing also helps reduce traditional test-taking anxiety in participants putting them at ease.

Canvas LMS quizzing resources

Here are some additional resources related to using quizzes in the Canvas learning management system:

• Using Canvas quizzes as learning checks to assess that videos are watched.
• Information about editing questions in a Canvas quiz or in a question bank.
• Canvas Quizzes Overview video with more information about quizzing in Canvas.
• Adding requirements to a module Canvas Community article with information about adding prerequisites, like passing score on quizzes, for unlocking a module.

Written by Jon Gladden, FCS eLearning Developer

References:

1) Roediger, Henry & Putnam, Adam & Sumeracki, Megan. (2011). Ten Benefits of Testing and Their Applications to Educational Practice. 10.1016/B978-0-12-387691-1.00001-6. Retrieved from: https://www.researchgate.net/publication/291166244_Ten_Benefits_of_Testing_and_Their_Applications_to_Educational_Practice . Accessed on 14 Sep 21.

2) Paul, Annie Murphy. Researchers Find That Frequent Tests Can Boost Learning. Scientific American 313, 2, 54-61 (August 2015). Retrieved from: https://www.scientificamerican.com/article/researchers-find-that-frequent-tests-can-boost-learning/ Accessed on 14 Sep 21.