Project Title: Analyzing Various Scoring Methods for Fill-In Concept Maps
Project Mentor: Krista Kecskemety – Engineering Education
True conceptual understanding is predicated on recognition of interconnection between concepts within a topic as opposed to the ability of information recall because of the importance of relationships between concepts in an overarching topic. Concept mapping is a tool that has been introduced to meet the failure of traditional testing methods in assessing this interconnection of information. Concept mapping is a graphical representation of a central topic and the way related concepts connect to that central topic. This style of assessment is supported by semantic memory theory in that semantic memory is structured as a network of conceptual understandings and links between related concepts. By assessing the structure of semantic memory, concept maps attempt to better encapsulate conceptual understanding.
There are three prominent methods for assessing proficiency of a concept map; the traditional, categorical, and holistic scoring methods. These three scoring methods attempt to assess a concept map’s breadth, depth, and connectivity to varying degrees with varying levels of involvement by a scorer. These scoring methods have been designed for unstructured concept maps, but in this study, have been adapted to assess a fill-in concept map. Fill-in concept mapping is a successful concept map variant that may be more appealing to instructors due to a perceived decrease in cognitive load for the student and workload for the instructor in scoring, but little research has been done looking into how a fill-in concept map should be assessed.
Using the altered methods, a set of scored concept maps were analyzed for takeaways on how these methods can be best adapted, and in what circumstances they may be most effectively used. It was generally found that there is poor reliability across the methods in resulting scores, but with iteration, the methods may be more appropriately catered to specific circumstances. The traditional method struggled to allow for a gradient of correctness, but was particularly easy to implement; the categorical method introduced punishment for breadth of categorization, but is well suited for assessing highly interconnected topics; and the holistic method was resource intensive, but may be a more generalizable scoring method.