Client:
Research Project
UX Designer & UX Researcher
View Live Version

Mitigating Toxicity in League of Legends

Client

Carnegie Mellon University Research Project

Solution

In-Game Interface

Direct Contribution

UI Design & User Research

Year

2023

Scope of Work

12 Months

Location

Pittsburgh, PA

League of Legends is a popular online video game that’s known for its strategic depth, fast-paced action, and large cast of characters . It also has a massive esports scene, with professional players competing in tournaments worldwide. This competitive environment brings about toxic behavior amongst players.My team’s goal was to use cognitive psychological means to target intrinsic reward within players of League of Legends to lessen their tendency towards toxic behavior.

Hypothesis

“By implementing a value system that creates cognitive dissonance and reflection via priming among players, this will reduce those player’s toxic behaviors due to intrinsic reward.”

Intervention

This was the final system that our team tested on multiple users over the course of the semester. A player chooses their value, observes their teammate’s values, play the game, endorse their teammates, and then progress in the value system.

When a player first starts their game, they are prompted to pick 3 values they wish to grow in and reflect on why. This is an example of self-awareness theory which suggests that paying attention to ourselves plays a major role in self-control and how we regulate our behavior. By forcing players to reflect on the value they’ve chosen, it promotes self-reflection which reinforces the value that player has chosen.

While waiting for the match to start, the loading screen displays every player’s values above their avatar.

This is an example of priming which is the idea that something you encounter subconsciously influences how you react in the future. Within the context of our intervention, seeing these values will subtly nudge your subconscious towards those chosen values. Primed with these values, it will lead to value-driven behavior during the match.

During gameplay, each player has their values displayed on screen next to their avatar. In addition, players can see their teammates’ values displayed near their avatars throughout the game.

By placing each player’s values next to their avatar, it encourages social proof as each player will try to act in accordance to their values due to it being publicly displayed in association with their identity. In addition, a player acting against their values would create cognitive dissonance as a deterrent. Lastly, all players are constantly primed with their values on display during the game.

When the game ends, players are given the option to endorse their teammates on their values. Through our research and testing, we found it was best to allow players only the option of positive endorsement and no negative feedback.

Implementation of this feature causes self-reflection among players. Through our research and testing we found that evaluation aligned in a positive framework causes players to find at least one positive trait. This encourages social proof as all players know that they will be evaluated for their game play as well as forces acknowledgement that other players are behaving well and they should reciprocate that. In addition, this primes players for their next game as they see other players’ values and reflect on their own.

Once players have received all of their endorsements, those ratings are aggregated and converted into points that contribute towards an extrinsic reward system. As players collect points from endorsements, they reach tiers in which they unlock small non-performance-enhancing rewards like skins or hats. Once a player reached the third tier for their value, they are allowed to pick a new value to work on and restart the process.

Research

Our Team consulted leading experts in the game design field as well as conducted secondary research on similar interventions.

With the lectures in Cognitive Psychology mechanisms that we received in class from Professor Kaufman, our team then dove into secondary research by performing a literature review on existing research in virtual gaming environments, online toxicity, impulsive behavior, and cyberbullying. From this, we learned that the industry primarily uses extrinsic reward (points, items, bonuses) to mitigate toxicity but it’s largely ineffective. This is because, from a psychological perspective, it only addresses the action and not the cause of the action. A player can easily act within accordance of an extrinsic reward system to simply reap the benefits provided. But down the line players begin to disregard about the reward system because either the system is not providing adequate reward to warrant good behavior. Thus causing players to go back to their old toxic behaviors and thus not creating permanent change in the community. Our team recognized that to create lasting change, we would have to approach this problem with intrinsic reward to change player’s behaviors from within rather than only providing incentives to change.

After doing this foundational secondary research, we consulted experts in our field. First, we interviewed professional DOTA 2 player Ravi Kawade who was familiar with DOTA 2’s intrinsic reward system to mitigate toxicity. Next we interviewed System Scientist at Carnegie Mellon University’s Human-Computer Interaction Institute, Erik Harpstead, who provided further explanation of the current landscape of the market as various companies aim to combat toxicity within their games as well as his opinion on the potential effectiveness of our proposed system. Then we interviewed Lead Game Designer at CMU’s Center for Transformational Games, Elaine Fath, who provided research methods to test the long term effects of transformational games on players. Lastly, we interviewed Games Researcher at Utrecht University, Julian Frommel, who specializes his research specifically in the causes toxicity in gaming. Our system piqued the experts' interest, as our approach mirrors the industry’s approach towards toxicity: to use intrinsic motivation as a new way to combat toxicity.

Expert Consultations

After doing this foundational secondary research, we consulted experts in the gaming space.

Julian Frommel

HCI & GAMES RESEARCHER AT UTRECHT UNIVERSITY

Julian’s research in human-computer interaction and human-centered AI focuses on the positive and negative impacts of interactive systems. This involves assessing user states to predict negative experiences, leveraging insights from game user research, implementing new interaction concepts for emerging technology, and studying social interactions to address negative human factors like online toxicity. Additionally, he explores how interactive systems can be adapted and intervene to mitigate these negative outcomes, including mitigating exposure to online harassment.

Elaine Fath

LEAD GAME DESIGNER AT CARNEGIE MELLON UNIVERSITY

Elaine is currently a Lead Game Designer at Carnegie Mellon University’s Center for Transformational Games. Her research focuses on personal development from playing games. Previously, she worked at Meta Reality labs developing their VR space with high-realism avatars.

Erik Harpstead

SYSTEM SCIENTIST AT CARNEGIE MELLON UNIVERSITY HCI

Erik is a System Scientist at the Human-Computer Interaction Institute at Carnegie Mellon University. His research focuses on educational games, particularly developing new data science methods for their unique design challenges. He aims to create tools and techniques for designers and researchers to ask questions about their games and use the answers to guide effective redesigns. He earned his PhD at CMU in 2017 under the guidance of Vincent Aleven.

Ravi Kawade

PROFESSIONAL DOTA 2 PLAYER

Ravi is a professional DOTA 2 player with years of experience under his belt. DOTA 2 recently implemented a similar reward system to our research that is aimed at mitigating toxicity in-game.

Testing

Our Team Conducted a series of usability tests before developing a high-visual-fidelity prototype. This consisted of a non-competitive match in which we observed the participants and conducted a semi-structured interview afterwards.

Given our research and the focus group we conducted, our team developed our new in-game endorsement system for League of Legends. This system aimed to promote positive player behavior by allowing teammates to endorse each other based on pre-selected values like teamwork or sportsmanship.

To measure the effectiveness of this system, our team conducted a play test with 5 League of Legend players. This session involved introducing the system to the participants, players choosing personal values, participating in an non-competitive match, and then endorsing teammates based on their chosen values and observed behavior.  Following endorsements, players were the interviewed to understand their experience with the system.  The interview  focused on how players chose their values, how they used endorsements, and their overall impression of the tier ranking system (a potential reward for positive endorsements).  Below are photos from an early prototype involving Wizard of Oz testing with participants. Later on, we developed a full digital prototype to be used in the same testing plan.

Participants selected their values before playing.

Participants then played a match with their values displayed on screen.

Participants then endorsed their fellow teammates on their values.

Participant chat log data was recorded and analyzed for toxicity.

Findings

Users found our intervention easy to learn as well as beneficial in future games. Thus, showing promise in our intervention and prompting a full research project.

Based on interview findings as well as aggregating and analyzing chat data from 3 rounds of testing with prototypes of varying fidelity we concluded promising results. Our efforts to promote more prosocial behaviors were met with a wide range of reactions from League of Legends players, with some hopeful that our design could potentially begin to promote change in the community, and others skeptical about the community’s capacity to shift away from toxic behaviors. Some participants liked that players could upvote, but not downvote, teammates on their selected values, but others thought that some consequence mechanism was necessary for players who exhibit poor behavior. Many players felt that they did not particularly think about their values during the game, but did put deliberate thought into making their selection decisions before the game. A longer period of user testing would be beneficial to see the long-term effects of the value selection on player’s self-reflection and intrinsic motivation. Below are results from the System Usability Scale conducted after Round 3 of prototype testing:

80%
of users rated this a 4/5 or above in terms of using this system in their future games.
60%
of users found this system extremely easy to learn.

Next Project:

Automating the Parachute Pack Process