Behavioral Systems: April 2021

Hey all. This is our first major update in 2021, and we apologize for the long delay. 2021 started quietly for the team while we focused on the backend systems that we are now applying to real problems we’d like to solve. We delayed this update a few times to make sure it'd feel worth reading, but the result was an unacceptably long period of radio silence.

To avoid making the same mistake twice, we want to let you know that updates will be more spread out this year. Specifically, we're altering our blog cadence to quarterly. That means you can expect our next update in mid-July.

Now onto the first update of the year.

GOALS

As it's been a while since we last chatted, let's talk goals. Broadly speaking, Behavioral Systems has a few evergreen goals...

• Reduce the frequency of disruptive behavior
• Mitigate harm caused by disruptive behavior

...as well as shorter-term goals specific to individual problems we want to solve. Right now, those are:

• Specifically, reduce the frequency of repeat disruptive behavior
• Use the newly-launched reporting confidence system to increase confidence (hence the name) of automated detection and penalties for disruptive behavior

In addition to the above behavioral goals, we’re partnering with Competitive in tackling problems endemic to matchmaking and competitive integrity:

• Continue improving queue matchmaking quality without compromising queue time and availability
• Reduce Champion Select dodges across all MMRs

As a reminder, we're tackling these problems via iterative, solution-driven experiments aimed at measurable results. We want to get our ideas in front of you for testing to find which ones have potential for meaningful, immediate change.

RECENT RESULTS: REPORTING CONFIDENCE (INTENTIONAL FEEDING)

In late 2020 and early into 2021 we talked about plans for a reporting confidence solution that took into account quality of reports (players who are great at consistently recognizing disruptive behavior vs. others who spam the report button every match). Over time, this helps our systems refine their understanding of disruptive behavior that may be less blatant, as well as penalize players who are continually skirting our existing detection lines.

We've rolled reporting confidence out for Intentional Feeding and have been seeing some promising results. We’ve started to confidently detect behaviors like jungle griefing, under-the-radar intent to lose or feeding behaviors, intentional lane taxing and griefing, and other behaviors that our automated systems are less likely to identify:

Since the system is still new, we're being very cautious with actioning on these behaviors until we're sure the system won't evolve in an unforeseen way and penalize innocent players en masse. It might feel like there are still people getting through the cracks right now, and for the time being, that’s ok. This is just the start of training our models to adapt to high-quality player reports., We're excited to continue monitoring results and in the future move forward with similar features for other prevalent disruptive behaviors.

ALMOST HERE: QUEUE DODGING PENALTY ESCALATION

We mentioned above that we're partnering with Compet to address growing issues with dodging and match quality. The first approach outlined below is to immediately address the escalation of these behaviors via stronger penalties. The second is a longer-term effort to address the underlying reasons for dodging in the first place.

First, a single dodge doubles the overall time-to-game for 9 other players and creates a jarring experience as players are forced in and out of queue. At higher levels of play, we even see some players say it's better to dodge a champ select with picks they don't like than it is to play the match out. Dodging should never be so optimal that players treat it like a tool in their toolkit. It should be a last resort 'break in case of emergency' switch.

One takeaway from last time we discussed dodge penalties is that you want us to avoid overly punishing players who had an unintentional disconnect, an emergency, or someone intentionally trying to ruin the game—in other words, the forgivable dodge cases. We're addressing that by adding a penalty tier but leaving the current tiers unchanged:

Existing Tiers (unchanged)

• 1st dodge in 24 hours: 6 min lockout (15 for ARAM) and -3 LP in ranked
• 2nd dodge in 24 hours: 30 min lockout and -10 LP in ranked

New Tier

• 3rd dodge in 24 hours: 12 hour lockout and -10 LP in ranked

We expect the above changes to limit current abuse of the system while we work with our other Riot teams to really dig into areas like:

• Autofill/Off-role disparity concerns
• Perceived between-team mismatches (a separate problem space from actual mismatches)
• Hostage situations in Champ Select
• One-trick limitations at higher MMRs

Though there are no magic bullets for these in the short-term, similar to last year, we think we can make iterative improvements across the year to drive meaningful impact on these underlying problems.

ALMOST HERE: AFK PENALTY ESCALATION

In addition to dodges, we’ve been assessing penalties for players repeatedly AFKing or leaving games. Again, we don’t want to overly punish players who have rare disconnects out of their control, but through monitoring, we've learned that repeat AFKs/leaves are relatively rare and tied pretty directly to game-ruining behavior. So, we're opting to increase the lockout severity of our existing higher penalty tiers, rather than add new ones. When players AFK or leave games they increase their penalty tier and when they play games without AFK they will slowly decrease it back towards 0.

Today's Tiers:

• 1st Tier: 5 min lockout and auto-loss
• 2nd Tier: 10 min lockout and auto-loss
• 3rd Tier: 15 min lockout and auto-loss

Revised Tiers:

• 1st Tier: 5 min lockout and auto-loss
• 2nd Tier: 30 min lockout and auto-loss
• 3rd Tier: 14 day lockout and auto-loss

Separately, there is an escalating LP penalty assessed for going AFK or leaving ranked games. This goes up by one tier when you AFK/Leave and goes down by one tier when you do not. This doesn’t apply to promotion series or when because of server issues the game is declared to have not counted.

• Tier 1: -2 LP
• Tier 2: -3 LP
• Tier 3: -5 LP
• Tier 4: - 6 LP
• Tier 5: -8 LP

This has been live for a few months and we are actively monitoring and looking for opportunities to improve the system.

THE SECOND HALF OF 2021 AND BEYOND

After this round of updates, here are the next areas we'll be looking at:

• In-game reporting options to allow players immediate response to bad behavior that further improve detection systems with time data indicating when the disruptions occurred in the course of the game
• Further refinement of penalties for extreme offenders of all kinds, including an extension on the use of bans.
• Continued improvement on AFK, Intentional Feeding, and Verbal Abuse detection using our reporting confidence system.