The UMBC Cyber Defense Lab presents
Creating Educational Cybersecurity Assessment Tools
Alan T. Sherman
Department of Computer Science and Electrical Engineering
University of Maryland, Baltimore County
12:00–1:00pm Friday, March 9, 2018, ITE 229, UMBC
The Cybersecurity Assessment Tools (CATS) Project provides rigorous evidence-based instruments for assessing and evaluating educational practices. The first CAT will be a Cybersecurity Concept Inventory (CCI) that measures how well students understand basic concepts in cybersecurity (especially adversarial thinking) after a first course in the field. The second CAT will be a Cybersecurity Curriculum Assessment (CCA) that measures how well students understand core concepts after completing a full cybersecurity curriculum. These tools can help identify pedagogies and content that are effective in teaching cybersecurity.
In fall 2014, we carried out a Delphi process that identified core concepts of cybersecurity. In spring 2016, we interviewed twenty-six students to uncover their understandings and misconceptions about these concepts. In fall 2016, we generated our first assessment tool—-a draft CCI, comprising approximately thirty multiple-choice questions. Each question targets a concept; incorrect answers are based on observed misconceptions from the interviews. In fall 2017, we began drafting CCA questions. This year we are validating the draft CCI using cognitive interviews, expert reviews, and psychometric testing. In this talk, I highlight our progress to date in developing the CCI and CCA. Audience members will be given an opportunity to answer sample questions.
Presently there is no rigorous, research-based method for measuring the quality of cybersecurity instruction. Validated assessment tools are needed so that cybersecurity educators have trusted methods for discerning whether efforts to improve student preparation are successful.
Joint work with Linda Oliva, David DeLatte, Enis Golaszewski, Geet Parekh, Konstantinos Patsourakos, Dhananjay Phatak, Travis Scheponik (UMBC); Geoffrey Herman, Dong San Choi, Julia Thompson (University of Illinois at Urbana-Champaign)
Alan T. Sherman is a professor of computer science at UMBC in the CSEE Department and Director of UMBC’s Center for Information Security and Assurance. His main research interest is high-integrity voting systems. He has carried out research in election systems, algorithm design, cryptanalysis, theoretical foundations for cryptography, applications of cryptography, and cybersecurity education. Dr. Sherman is also an editor for Cryptologia and a private consultant performing security analyses. Sherman earned the PhD degree in computer science at MIT in 1987 studying under Ronald L. Rivest. www.csee.umbc.edu/~sherman
Support for this research was provided in part by the National Security Agency under grants H98230-15-1-0294 and H98230-15-1-0273 and by the National Science Foundation under SFS grant 1241576.