New Rules For Assessment Design

October 31, 2014

New Rules of Assessment Design


The learning and development community is undergoing a massive shift in the ways we deliver and assess learning. In years past, we all thought of learning as unidirectional. We, the teachers, would offer a course. Learners would take the course. We would assess their learning.

Today, we’re beginning to see that learning is not so linear. It’s social. It’s informal. It’s self-directed.

So are great assessments. In this article, we’ll explore some “new rules” for creating effective assessments (as well as revisiting some old ones) so you can get more mileage out of these valuable learning tools.


What makes a great assessment?


In a corporate L&D setting, we often think of assessments as having one primary goal: to ensure our learners meet some minimum prescribed requirement. For example, that they are OSHA compliant or understand our corporate rules and policies.

While in some cases this may be the only purpose they serve, often assessments can carry a lot more weight in our educational arsenal. In addition to ensuring regulatory compliance, they are also extremely effective tools to teach and reinforce lessons, as well as to evaluate the effectiveness of your courses.

The University of Connecticut has outlined a number of ways to use assessments within an educational setting.


Image excerpted from the University of Connecticut website


While not all of these goals will be relevant in a professional setting, this image drives home a key point: assessments do more than just test basic knowledge of a topic. They are useful in a wide variety of ways.


When we think about what makes a great professional assessment, our simple definitions is: a tool that improves students’ learning and teachers’ teaching.


If an assessment accomplishes both of these goals, it’s pulling its weight.


A Traditional Approach to Assessment Design


Before we dive into the new rules of great assessments, it’s important to review some of the tried-and-true best practices. Because you can’t change the rules of a game without understanding them first.

Linda Suskie, Director of Assessment at Towson University, recommends the following 8-step process for designing effective assessments:

(As excerpted from the University of Connecticut website)


1. Start with clear statements of the most important things you want students to learn from the course, program, or curriculum.


2. Make sure you build those key concepts into your curriculums. Because great assessment design begins with the course itself.


3. Collect more than one kind of evidence of what students have learned. No assessment vehicle is perfect, and students have different learning styles. Collecting different kinds of evidence will provide a more holistic understanding of students’ learning.


4. Ensure your test questions are crystal clear. If the questions aren’t clear, they aren’t helping your students learn or helping you teach. 


5. Ask students and colleagues to review drafts of assessments and learning modules to ensure they are clear and effectively teach important topics.


6. Try out assessments on small groups before deploying them across the enterprise.


7. Gauge your assessments based on the usefulness of the information they provide both to instructors and students.


8. Use your assessment to improve teaching and learning activities. This important final step is often forgotten in the corporate L&D world.


The New Rules of Assessment Design


What Linda Suskie has outlined above is an effective methodology for thinking about assessments. What we propose is to expand your tactics and mediums (great assessments are more than simple multiple choice) to get more insight from your assessments and better meet the needs of the modern learner.


New Rule #1: Use Many Different Question and Answer Formats


Traditionally in L&D, we rely on one question type: the (yaaawn!) multiple choice question. We believe it’s high time to shake things up.

Look around. We live in a media-rich world. Snackable audio and video content are the entertainment mediums of choice, and nearly everyone is a content producer (think social media).

As an L&D professional, you can capitalize on this to make your assessments more effective by using audio and video assessments.

Rather than essays or boring multiple choice, have your students respond in a video or audio format. Not only will learners be more engaged, but you’ll get more natural and thoughtful responses—especially from workforces that may have lower levels of writing competency.

If you don’t have access to audio and video tools, you can also consider using free-form questions, such as fill-in-the-blanks. These types of questions require the student to have a more substantive understanding of concepts to be successful (there is less opportunity for guessing), giving you better insights into the value of your instruction.


New Rule #2: Implement Question-Level Analytics


Among Linda Suskie’s recommendations for designing effective assessments is to “assure your questions are crystal clear.” She also recommends seeking student review of all assessments.

While student and colleague feedback can be valuable, empirical evidence will tell a story that objective opinions cannot. For example, when examining the responses to your learners’ questions, you may find that a majority of your learners got one question wrong. This is a sign you either need to strengthen that lesson or consider rewording the question.

The good news is, with modern technology, getting this type of insight is easier than ever.


New Rule #3: Use Peer Grading and Comments


As we mentioned earlier, assessments are opportunities not only to assess, but also to reinforce. Providing opportunities for “peer-grading” is an excellent way to do this. To implement this, we recommend that rather than assessments always culminating in an automatic grade, students’ responses are passed off to their peers.

Peers are then able to evaluate responses, reinforcing their own learning, as well as providing feedback on the quality of the instruction. This also delivers an additional form of evidence that students understand the topics you are teaching.


New Rule #4: Allow for Self-paced Assessments


Not everyone learns—or is prepared to answer questions—at the same pace. This has been widely recognized in the traditional educational community for many years. The simple solution is to allow your students to take their examinations at their own pace. With self-paced assessments, students will have more time to digest the questions in a meaningful way, and their learning outcomes will improve in the process.