How Our Team Moderates Content on AI Dungeon

Our moderation team plays an important role in ensuring that players of different age groups and interests can discover content they are interested in all while supporting our community's creative freedom. Specifically, moderators monitor the accuracy of content ratings and check for unpublishable content so players can find what most interests them and avoid what doesn’t. Our creators also play a pivotal role in this mission by doing their best to correctly rate the content they create so all players have a good experience on AI Dungeon.

Moderation is complex and difficult. We made extraordinary efforts to develop our content guidelines, and we are constantly looking at feedback from all levels of users to inform adjustments that better reflect the needs of our players and creators. We want to be as open and transparent as possible with our process and give some details on how we moderate our content ratings and some of the areas we investigate.

Here are a few things our moderators consider when making moderation decisions.

Matching Player Expectations with Platform Safety

Our goal with moderation is to ensure that players find the content they are looking for and content that is suitable for their set content rating. The goal of moderation is not to push any sort of moral agenda, and we have no interest in being the judge of right and wrong. Our goal is simply to give players the experience they want.

Doing that effectively means paying very close attention to player feedback. We pay attention to feedback from Discord, Reddit, support emails, player surveys, user testing, in-game data, and content reported on AI Dungeon. Thanks to the volume of feedback we receive from players, we can identify trends and community sentiment on everything from themes to specific scenarios. It is important that we listen to all forms of this feedback because we’re aware that a vast majority of our community isn’t always vocal.

While the feedback we receive informs how we create and enforce our guidelines, we also have to ensure that we create a safe environment for younger players or for players who want a safe, curated environment free from more sensitive topics.

Sexual Content

Most moderation feedback we hear from players deals with sexual content. As one would expect, there’s a wide variety of opinions about the type of sexual content players are comfortable seeing. Our content rating system options of Everyone, Teen, Mature, and Unrated help us categorize sexual content for audiences. It ranges from mildly suggestive (for Teen) to explicit (for Unrated). We also have to evaluate what we consider unacceptable to publish.

When considering where content falls on that spectrum, we reference player feedback and assess each audience’s general comfort level (Everyone, Teen, Mature, Unrated, Unpublishable) to the themes and content in the scenario.

When evaluating sexual content, here are some of the areas we look at:

• Plot prominence—Is this a mature story with sexual references? How developed or significant are non-sexual plot lines? Is the entire setup of the scenario focused or foreshadowing or alluding to a sexual encounter?
• Depiction Style—Is it descriptive and lewd, or subtle and innocent? How much detail goes into describing appearances or anatomical features?
• Age appropriateness—How does the content align with the expectations of our different audiences? How is similar content rated (such as movies, films, or books)?
• Underage Characters or Themes—Does the scenario knowingly involve, or is ambiguous around, underage characters? We take a strict stance when minors are involved in any context that could be perceived as sexual. If situations are ambiguous, we will always err on the side of safety and mark them as unpublishable.
• Thematic Content—Are players generally accepting of the types of acts or relationships? Are there any taboo subjects? Does the scenario depict kinks or fetishes that some players may find disturbing?
• Language and Tone—Is the overall tone meant to be provocative and stimulating? Or more serious, educational, or artistic? Is crude or profane language used?
• Pop-Culture Interpretations—Does the scenario reference known characters from other fictional works? How are these characters viewed in these fictional pieces? Are they known for being violent? For their sexuality, or being a specific age?
• Consent—What are the power dynamics in the relationships? Is it clear from the plot that consent is given?
• Keywords—Are there words that are generally seen as sexual terms? In cases where words might have a potential sexual meaning, we may assess what a player seeking information on these words will find. How are these potential words or concepts understood broadly?

Note: We consider sexual content to be “explicit” if it’s more likely to be seen as objectionable by players.

Hopefully, it’s clear that there is a lot to consider when moderating content. There isn’t a simple set of rules we can use to determine a rating, nor will every situation have a simple ‘black & white’ solution. Typically, our team analyzes and considers multiple elements of a story and determines whether, on the whole, players would agree that the scenario fits one of our content ratings or should be unpublishable.

Allusion and Chekov’s Gun

Players have also shared that finding content alluding to disturbing or explicit themes can be just as frustrating as seeing content that clearly depicts such themes.

Many of our players are probably familiar with the writing principle called “Chekov’s Gun.” The principle states: "If in the first act, you have hung a pistol on the wall, then in the following one, it should be fired. Otherwise, don't put it there.” The idea behind Chekhov's gun is that every element in a story should be necessary and irreplaceable. If something is introduced into a narrative, particularly something as significant as a weapon, it should serve a purpose in the plot.

Similarly, when players (and moderators) look at the characters, settings, and objects included in AI Dungeon scenarios, it sets expectations for the type of content that they’ll be experiencing. If the content being created isn’t intended to be sexual or disturbing, then according to Chekov’s gun (and player feedback), it doesn’t make sense to include story elements alluding to those themes in a scenario. We have to evaluate the content at face value. Our moderation team has learned to identify creators who are using sophistry to try to get the moderation decision they want. For instance, tagging content as “wholesome” or “innocent” won’t influence the rating we assign. Nor does saying, “All characters are 18 and consenting adults,” if the elements of the story clearly indicate otherwise.

This is particularly relevant when determining if content should be Mature or Unrated. Our Mature content rating definition states: "May not contain or allude to disturbing or explicit sexual content." Alluding to disturbing content or explicit sexual content means:

• Creating scenarios with a clear setup for disturbing or explicit sexual content
• Including subtext, context, or innuendo that hints toward disturbing or explicit sexual content
• Featuring situations, character descriptions, or story details that imply or foreshadow explicit sexual situations or disturbing content
• Using terms that are commonly interpreted or understood as sexual but implying that they are innocent

While we know that creators want to get as many views on their content as possible, we also need support in ensuring that content is crafted with content ratings in mind. While content might be suitable for your tastes, be mindful that players with various preferences visit our platform daily, and we are responsible for ensuring their experience meets their standards.

There is no internal strike or demerit system we’re keeping on creators. If someone frequently discusses their content with our team or even constructively provides feedback or criticism about our process, we’re okay with that. Our main consideration is the creator’s willingness to help us achieve our goal of giving our broader community the experiences they want on AI Dungeon by rating content accurately. The only creators who lose publishing permissions are those who are intentionally breaking rules, antagonizing the moderators, or taking other actions that may harm our community.

More Ways to Provide Feedback

All AI Dungeon users—creators or players—are invited to share feedback on how we’re doing with our content moderation. We’d love to understand whether the content being discovered on AI Dungeon meets player expectations, or if we can improve how we moderate content. Our goal is to ensure the content experience on AI Dungeon meets players’ expectations, but we also have to ensure we have protective safety measures for those who may not want to seek out sensitive topics. This will always be a delicate balance.

The best way to share feedback is by emailing us at [email protected]. The feedback we receive here is reviewed by our moderation team and company leadership, and we will always strive to optimize efforts in moderating content to meet the needs of our community.