Changing the five-star rating scale can eliminate effects of racism on digital platform workers

Highlights

• Customers on digital labour platforms often discriminate against racial minority workers in evaluations by rating them lower than similar white workers.
• This study found that changing rating scales from five-star to two-point scales (thumbs up or thumbs down) reduced discrimination against racial minority workers because customers could only choose between “good” and “bad” performance. This change also eliminated the pay gap for these workers.
• A simple adjustment in rating methods offers a promising organizational intervention to create fairer work conditions for racial minorities.

Today, digital labour platforms are ubiquitous, used for services such as food delivery, ridesharing, and home services. These digital labour platforms often ask customers to rate workers’ performance, and ratings can affect workers’ pay and future work. But what happens when customers have racist beliefs? Research shows that negative stereotypes and biases can cause customers to unfairly lower ratings for racial minorities compared to similar white workers, and these biases are hard to address.

Types of rating scales affect ratings

Researchers Tristan L. Botelho, Sora Jun, Demetrius Humes, and Katherine A. DeCelles investigated how platforms can mitigate customers’ racial discrimination in evaluations. Prior research shows that modern racism often emerges in subtle rather than blatant ways. The authors proposed that five-point rating scales can encourage discrimination to emerge because these scales give customers the option to slightly downgrade evaluations while still rationalizing that it is a good rating (e.g., downgrading from five to four stars). A two-point scale removes this option, forcing customers to focus solely on good or bad performance and making them less likely to score a racial minority worker as bad if their performance was good.

The authors studied an online home services platform that changed its customer evaluation system from five-stars to thumbs up/thumbs down. Through analyzing job and rating scale data, they discovered that racial minority workers prior to the change had an average rating of 4.72 stars, compared to 4.79 stars for white workers. Although this difference may seem small, it affected workers’ income rate and translated to racial minority workers making 91 cents for each dollar earned by white workers.

After the change to a thumbs up/thumbs down scale, the difference in ratings between white and racial minority workers was eliminated, with both groups receiving the highest ratings around 95% of the time. The change also closed the gap in pay.

After the change to a thumbs up/thumbs down scale, the difference in ratings between white and racial minority workers was eliminated, with both groups receiving the highest ratings around 95% of the time.

The authors confirmed their findings with further experiments, asking online participants to rate groups of workers with equivalent performance and different races. They found that those who showed stronger racist beliefs were less likely to rate racial minority workers poorly with a thumbs up/thumbs down scale than with a five-star scale. They also found that encouraging evaluators to rate solely based on good or bad performance, even with a five-star scale, helped reduce racial bias in ratings.

Structural change can reduce effects of discrimination

Thus far, there has been little success in finding ways to mitigate racial discrimination from customers towards workers in evaluations, which is especially important in a gig economy. This study shows that changing the way customers evaluate can reduce the effects of their racist beliefs on worker income. Organizations can shift away from the common five-star rating scale to help create a fairer work environment.

Katherine DeCelles, one of the authors, notes that their research “highlights the importance of structural solutions to help reduce the impact of customers’ bias on workers, given that they cannot be trained or selected the way that managers can. This emphasizes the important role that an organization–here, the design of an evaluation system on a platform—can play in mitigating societal inequality.”

“This emphasizes the important role that an organization—here, the design of an evaluation system on a platform—can play in mitigating societal inequality.”

This change may also have similar impacts for discrimination based on gender, age, and disability, although further research is needed. Future research can also explore whether these effects hold outside of online evaluations, such as in in-person workplaces.

______

Research brief prepared by:

Carmina Ravanera