Mass. pledges to create new firefighter promotional test by March

A firefighter promotional test that was canceled just days before it was scheduled to be administered could be reworked by March, the Massachusetts Human Resources Division has assured the Professional Fire Fighters of Massachusetts and the chairman of the state Civil Service Commission.

“We’re better off than we were a week ago,” said Rich MacKinnon, president of the state firefighters union.

In a closed-door hearing with the Human Resources Division and Civil Service Chairman Christopher Bowman, the union was assured the Civil Service Commission would retain oversight of the work being done to craft a new exam.

The firefighter promotional exam was canceled based on the finding last month by Suffolk Superior Court Judge Douglas H. Wilkins that promotional examinations for sergeant given to police had an adverse impact on minority candidates.
The firefighter promotional exam was canceled based on the finding last month by Suffolk Superior Court Judge Douglas H. Wilkins that promotional examinations for sergeant given to police had an adverse impact on minority candidates.

Bowman instructed the Human Resources Division to meet with the vendor used to develop examinations and devise a timeline of achievable goals and benchmarks by the end of the day Tuesday. Massachusetts contracts with PSI to formulate its examinations.

The firefighter promotional exam was canceled based on the finding last month by Suffolk Superior Court Judge Douglas H. Wilkins that promotional examinations for sergeant given to police had an adverse impact on minority candidates.

The class-action suit, Tatum et al v. The Commonwealth of Massachusetts, was filed in 2009.

In his ruling, Wilkins found that an 80-question, multiple choice exam extrapolated from a reading list provided to applicants does not adequately reflect the rigors and reality of the job, and that the best test takers don’t necessarily make the best field officers.

The Tatum class action lawsuit is scheduled to enter a penalty phase March 20.

The firefighter promotional exam was canceled, according to the state, because it was formulated in the same fashion. Firefighters looking to advance their careers were assigned a reading list and a similar 80-question, multiple choice test.

“It’s not what we wanted, but it’s better than where we were,” MacKinnon said, explaining that when the exam was canceled, the Human Resources Division said it could take anywhere between eight months and five years to develop a new exam.

Bowman promised to keep the affected groups informed on the progress in reworking the test.

“But he also suggested that all the stakeholders, including the firefighter unions, form an advisory group that meets regularly to oversee the process,” MacKinnon said.

Police officers remain in limbo

The Human Resources Division opted not to grade the police promotional exam administered in September based on the Tatum case findings by Wilkins. The move leaves more than 800 officers who had taken the exam in limbo.

The Massachusetts Police Association, an umbrella group representing many of the state’s municipal departments, sent a letter to Gov. Charlie Baker Wednesday, urging him to counteract the Human Resources Division.

The organization said the state should "initiate the 'remedial plan' process to remind the court that innocent, third parties impacted by the decision who reasonably relied on prior law should not be harmed as an incident of the plaintiff’s remedy.”

The organization urged the Human Resources Division to continue to score exams already administered and schedule additional exams until a final judgment is reached in the Tatum case.

In his findings Wilkins, determined that the state knew the examinations had an adverse impact on candidates of color and had alternate means of evaluating candidates seeking promotion but failed to use those means.

The lawsuit embraces sergeant’s exams administered by the state between 2005 and 2012. It was filed by officers in Boston, Brockton, Lawrence, Lowell, Methuen, Springfield and Worcester, as well as officers working for the MBTA.

'No perfect test'

Jim Harris, of J. Harris Academy of Police Training based in New Jersey, said there is no “perfect test,” but there are ways to make evaluations as fair as possible.

In New Jersey, the state Civil Service promotional test for sergeant marries a multiple-choice question section, based on assigned reading and a video response exercise assessment. The multiple-choice questions pertain to laws and the published state’s Attorney General Guidelines that regulate police response.

The video response measures how a candidate reacts to a specific scenario: Does the candidate take charge of the scene and act decisively, or are they hesitant or unsure of what their response should be?

There’s a difference, Harris said, in the response of a candidate for sergeant who shows up at a scene and sets a perimeter, delegates authority and gives deadlines for meeting tasks, as opposed to a candidate who hesitates or wonders what they should actually do.

“The video situation is one of the best options for gauging candidate response,” Harris said.

Some communities create their own exams, while others hire testing companies to create and administer them. Invariably, the exams are reviewed and scored by an outside firm.

To create an exam, Harris said, the agency must analyze the tasks undertaken by the officers in that rank: If it’s a promotional exam to the rank of sergeant, the agency analyzes what a sergeant does on any given day. The resulting test should encompass knowledge of criminal and case law, how to deal with citizen complaints and interpersonal skills relating to managing a group of people.

Assessment centers, where candidates respond to a scenario, can be problematic. They are more expensive, more time-consuming and the last candidate to take the assessment has an advantage as they most likely know what the scenario encompasses and the expected response. Scenarios must remain constant for each candidate to be assessed consistently.

“If an assessment center is done correctly, the assessors will look for the traits a candidate needs to be a good leader,” Harris said.

Finding the best candidates, those with interpersonal skills, management skills and knowledge of laws and restrictions, is the best way to protect the profession, Harris said.

He acknowledged that the multiple-choice exam extrapolated from an assigned reading list favors certain candidates.

“The more standardized tests people take, the better they get at taking tests,” Harris said. He suggests that candidates be encouraged to study for the exams and actively learn what is encompassed by the test.

“That’s where I come in,” Harris said. “I teach so that candidates can understand the material, make it relatable for the real world.”

Harris recognized that many departments believe test coaches give candidates the test answers; however, Harris said, it’s a question of teaching the material to ensure candidate success.

This article originally appeared on Telegram & Gazette: Human Resources sets March for new promotional test for firefighters