How you ever watched a YouTube demonstration video and said: “How hard can it be?” Later to find out it is a lot harder than expected. I have.

Training needs to be more than just listening to a lecture or watching someone else complete a task. We should train in ways that allow us to practice our skills. Good training should also assess whether or not we can successfully execute these skills. This is where evaluation comes in.

You might recall in our previous blog post “Measuring Threat Assessment Skills – Evaluating Learning,” we took a deeper dive into Evaluation Level 2 (Learning) and discussed trainee self-reports as an evaluation measure. For self-reports, trainees report their understanding of specific course material (tied to training objectives, of course) on a 1 to 5 scale before and after the training. It is an indicator of learning, but not a great one. It relies on the trainee to report AFTER the training how much they thought they knew BEFORE the training.  It is also subjective; it measures what trainees think they learned, not what they actually learned.

 A better alternative is to test the trainee before and after training, to measure what they knew before the training then measure again afterwards. We can then assess change and improvement. This also more objective.   

Pre and Post-Tests

You can test knowledge or skills. A simple approach to assessing knowledge is testing comprehension and understanding of the delivered material. This can be nothing more than true-false or multiple-choice questions on course content; an easy process in online learning. Testing can be harder in instructor-led courses where time is limited and when class sizes are large, but that does not mean it shouldn’t be attempted.

The second and better way to evaluate learning is to test skill improvement; can trainees perform the task that you trained them on better after the training. It is best if you have trainees complete a pre-test to make sure that skill improvement is actually attributable to the training.

If you spend thousands of dollars on training and your officers already knew what they were doing before the training (as revealed in pre-testing) or trainees didn’t increase their skills (based on pre and post testing), then you might not want to pay for more for that training in the future.  

Skills Testing in Active Threat Assessment

Skills tests should be a valid representation of the skill you want your trainees to perform. Skill testing can be a bit harder than other forms of evaluation, especially with many tasks related to law enforcement. It can also be time-intensive.

For our instructor-led courses, we use a video-based observation task which takes 25 to 30 minutes to complete. In this task, trainees are asked to identify a potential person of interest (POI) who may pose a threat to themselves or others. Trainees watch a video and answer questions about what they observe. There is a set procedure and pre and post-tests are similarly formatted. The assessment is designed so that someone who is NOT trained in active threat assessment methods can complete it.

Take a quick look at this video clip of the shopping area.

We have trainees observe 3 – 4-minute video a scene like this video (at normal speed) and answer questions about:

1. patterns of consistent behavior in the location;

2. the extent behavior of specific subjects is deviating from others and the extent these behaviors are deemed as suspicious. 

3. from those specific individuals, choose if there is a potential person of interest who would require observation or further interdiction based on their behavior.

After the pre-test test, trainees receive 8-hours of training in systematic observation and active threat assessment techniques.

After the training, they take a post-test. From this pre and post-tests, we can measure not only if trainees are better at spotting the bad guy, but also their ability to systematically observe an environment, identify deviating and suspicious behavior, and dismiss people who are NOT a threat. Important, huh?  

In our online Threat Awareness program, we repeatedly test trainees in the active threat assessment skills they learn through a series of practice exercises and practical exams. You can learn more about different threat assessment approaches or active threat assessment in blog posts in our blog (or you can take our training too).

Some Test Results

We have done pre and post-testing with military personnel, police officers, and police recruits. In a sample of 40 police recruits, 10% successfully identified a person of interest in the pre-test  After training, 71% of recruits successfully identified a person of interest.  

In our post-tests, even observers who did not correctly identify the person of interest were more likely to ignore individuals who were not a threat and focus on those who were showing suspicious behavior.

This improvement is impressive and shows that observation is a learned skill that improves with a systematic approach and practice.  

Wrapping it up.

Training is expensive. It costs you money and time. Evaluate the training your officers receive in order to maximize your training dollars. Don’t hesitate to reach out if you want to talk to us about training evaluation and making sure your officers learn what they should.

If you want to see how good you are at assessing threats, take our free Intro to Active Threat assessment training and find out.