Edit check/Reference Check - MediaWiki

Reference check Prompt newcomers to reference information they are adding to Wikipedia. Group: Editing Start: 2023-02-01 Team members: David Chan, David Lynch, Esther Akinloose, Marielle Volz, Rummana Yasmeen, Nicolas Ayoub, Benoît Evellin, Ed Sanders, Zoe, Caro Medelius Backlog: #EditCheck Management: Peter Pelberg (product), Valerie Puffet-Michel (engineering)

Reference Check is an Edit check that prompts newcomers to add citations before they publish an edit that adds new text to a Wikipedia article.

This Check is an effort to increase the likelihood that the new content people are adding to Wikipedia is aligned with WP:Verifiability.

To participate in and follow this project's development, we recommend adding this page to your watchlist

As of February 2026, Reference Check is available by default at all Wikipedias .

Reference Check aims to:

1. Cause newcomers to publish new content edits that include references while lowering the likelihood those edits are reverted
2. Reduce the amount of effort and attention experienced editors need to allocate to ensuring text in Wikipedia's namespace is accompanied by a reliable source

Accompanying additions to Wikipedia with citations is an important part of Wikipedia's five pillars.

Accompanying additions to Wikipedia with citations is also a practice many newcomers do not follow.

A July 2023 analysis found that 17% of the new content edits newcomers (≤100 cumulative edits) publish using the visual editor include a new reference.

With the above in mind, Reference Check is meant to address two core issues:

1. Newcomers publishing new text to Wikipedia that does not include a reference
2. Experienced volunteers being burdened by the effort and attention they need to allocate towards finding sources to verify the reliability of the information available on Wikipedia. This effort can come at the expense of identifying and addressing more subtle and complex forms of undesirable editing.

Reference Check is designed to address these two issues by:

1. Offering new(er) volunteers feedback while they are editing so they can avoid unintentionally publishing edits that are likely to violate policies
2. Offering patrollers/reviewers deeper insight into the edits they are reviewing (and the intentions of the people publishing them) by logging the moderation feedback newcomers are being presented with, and the actions they do/do not take in response.

The initial version of the Reference Check experience will prompt people to decide whether they want to add a reference or not.

This prompt will be presented to people who add a minimum of one new paragraph of text when they tap "Publish changes..." without accompany this new text with a reference.

For the first version of Reference Check, the Editing Team is pursuing an approach that minimizes the likelihood of false positives and is implemented in ways that empower volunteers, on a per-project basis, to evolve the detection heuristic to become more robust over time.

This strategy amounts to the initial Reference Check becoming activated if/when all of the following conditions are met:

1. A minimum of one new paragraph of text is added to the article someone is editing
2. The "new paragraph(s) of text" someone has added does NOT include a reference
3. The changes described in "1." and "2." are happening on a page within the main namespace (NS:0)

The conditions above are implemented and maintained in code here: editcheck/init.js.

The Editing Team arrived at the decision to start with a relatively limited and straightforward set of rules in order to:

1. Increase the likelihood that newcomers find the guidance Reference Check is presenting them with, and the editing experience more broadly, to be intuitive and straightforward so that they feel encourage to return to edit again
2. Decrease the likelihood that Reference Check is creating more work for experienced volunteers by prompting newcomers to add sources when they are not needed

You can learn more about the assumptions that informed the thinking above in phab:T329988#8654867.

Reference Check is implemented – like all Edit Checks – in a way that enables volunteers to explicitly configure how it behaves and who the Check is made available to.

Configurability happens on a per project basis so that volunteers can ensure the Reference Check experience is aligned with local policies and conventions.

A 2025 analysis of Reference Check’s A/B experiment on English Wikipedia found that when Reference Check was shown, edits were much more likely to add a new reference, constructiveness (not reverted within 48 hours) increased and 48-hour revert rates declined (especially on mobile web), and edit completion declined modestly.

Below you can read more about what this experiment demonstrated and more details about the experiment's design.

Conclusion

Reference Check was associated with a statistically significant increase in the share of edits that added a new reference. Edits where Reference Check was shown were also less likely to be reverted within 48 hours, especially on mobile web. The clearest evidence comes from the mobile-web-only results. Reference Check also added modest friction that reduced edit completion (saveIntent → saveSuccess; statistically significant).

Findings

• References added
  • When Reference Check was shown, edits were substantially more likely to include at least one new reference.
    • Results:
      • On desktop, constructive new-content edits were about 2× more likely to include a reference (30.7% → 68.2%; ~2.2×).
      • On mobile web, constructive new-content edits were ~17.5× more likely to include a reference (2.8% → 48.9%; ~17.5×).
  • Under a more conservative assignment-based view (availability / ITT), Reference Check still increased the likelihood that new-content edits included a reference, with the strongest gains on mobile web (statistically significant in the adjusted ITT model).
    • Results:
      • Overall: 56.3% → 68.3% ( +12.1 pp, +21.5% relative)
      • Desktop: 60.5% → 70.6% ( +10.2 pp, +16.8% relative)
      • Mobile web: 22.0% → 47.8% (~2.2×)
    • Context: These results are directionally consistent with the 2024 Reference Check A/B experiment, which also found large increases in reference inclusion when Reference Check was shown, with particularly strong effects on mobile.

• Constructive edits (not reverted within 48 hours)
  • Edits shown Reference Check were more likely to be constructive overall.
    • Desktop edits showed a modest improvement (not statistically significant in the across-platform adjusted model).
    • Mobile web edits showed a much larger improvement (within-platform contrast statistically significant).
  • Results:
    • Overall: 71.8% → 75.9% ( +4.1 pp, +5.7% relative)
    • Desktop: 75.5% → 77.9% ( +2.4 pp, +3.2% relative)
    • Mobile web: 56.4% → 66.7% ( +10.3 pp, +18.2% relative)
  • When accounting for whether a reference was added, the mobile-web effect attenuates, suggesting part of the improvement operates through increased reference inclusion.
  • Context: Similar patterns were observed in earlier Reference Check and Multi-Check experiments, where increased reference inclusion was associated with higher rates of constructive edits.

• Revert rate (within 48 hours; lower is better)
  • Edits shown Reference Check were reverted less frequently overall.
    • Desktop edits saw a moderate reduction in revert rate.
    • Mobile web edits saw a large reduction in revert rate (within-platform contrast statistically significant; across-platform adjusted treatment term not statistically significant).
  • Edits that added a new reference were much less likely to be reverted, reinforcing the proposed mechanism behind the feature.
  • Context: The magnitude of the revert-rate reduction on mobile web is larger than what was observed in the 2024 Reference Check experiment and aligns with improvements seen in Multi-Check experiments when multiple reference prompts were shown.
  • Results:
    • Overall: 28.2% → 24.1% ( −4.1 pp, −14.5% relative)
    • Desktop: 24.5% → 22.1% ( −2.4 pp, −9.8% relative)
    • Mobile web: 43.6% → 33.3% ( −10.3 pp, −23.6% relative)

• Edit completion (saveIntent → saveSuccess)
  • Showing Reference Check reduced edit completion rates across platforms (statistically significant).
    • The decrease was modest on both desktop and mobile web.
  • This reduction is consistent with introducing an additional decision point in the publishing workflow and coincides with improvements in reference inclusion and the downstream outcomes above.
  • Context: The completion decrease is smaller than that observed in the 2024 Reference Check experiment, suggesting that later UX changes (e.g., siderail presentation) may have mitigated friction.
  • Results:
    • Overall: 88.3% → 84.1% ( −4.2 pp, −4.8% relative)
    • Desktop: 94.0% → 87.6% ( −6.4 pp, −6.8% relative)
    • Mobile web: 74.1% → 69.4% ( −4.7 pp, −6.3% relative)

Experiment scope and design

• Wiki: English Wikipedia
• Dates: 8 November 2025 – 8 December 2025
• Participants: Unregistered editors, newcomers, and registered editors with ≤100 cumulative edits
• Platforms: Desktop and mobile web
• Editing surface: VisualEditor edits in the main namespace
• Intervention: Users in the test group were shown Reference Check when publishing new content without a reference; control users experienced the default editing flow
• Primary outcomes: Reference inclusion; constructiveness (not reverted within 48 hours)
• Guardrails: Revert rate (48 hours); edit completion (saveIntent → saveSuccess)

Prior to the final results, we published a leading indicators report on December 4, 2025, with preliminary findings.

Multi-Check includes a series of changes to the visual editor user experience that enables multiple Edit Checks, of the same or different types, to be presented within a single edit session.

To learn whether these changes were effective at causing, newcomers to publish new content edits that include references while lowering the likelihood those edits would be reverted, we ran an A/B experiment with 12 Wikipedias.

Below you can read more about what this experiment demonstrated, what the Editing Team is planning in response, and more details about the experiment's design.

Multi-Check caused statistically significant increases in the proportion of edits that include a reference and are not reverted within 48 hours without causing undesirable changes in edit completion or other forms of disruption (e.g. blocks).

In response, Multi-Check was deployed to all wikis where Reference Check is currently available on June 25, 2025.

• Proportion of new content edits with a reference
  • Users are more likely to include at least one reference with their new content edits when multi-check (references) is available.
    • Edits that were shown multiple Reference Checks in a session are 1.3 times more likely to include at least one new reference in the final published edit compared to sessions shown a single Reference Check.
• Revert Rate
  • Overall, we did not identify any significant changes in new content edit revert rate between the control and test group.
  • However, there was a -34.7% decrease in revert rate when directly comparing edits presented multiple checks compared to edits presented a single reference check
• Proportion of users that publish at least one new content edit with a reference
  • Overall, there was a 5.5% increase in the proportion of distinct users who published a new content edit with a reference when multi-check was available.
• Guardrails
  • There were no decreases in edit completion rate for up to 5 reference checks being presented in a single session (which accounts for the majority of multi-check edits).
  • We also did not identify any increases in revert rate at any number of reference checks presented.

• New(er) volunteers are encountering Multi-Check
  • In the test group, multiple reference checks were shown within a single editing session at 19% of all published new content VE edits (549 edits) by unregistered users and users with 100 or fewer edits.
  • For edits shown multiple checks, the majority of edits (73%) were shown between 2 to 5 Reference Checks.
• People shown multiple Reference Checks w/in an edit go on to publish at a relatively high rate
  • The edit completion rate for sessions that were shown multiple checks within a session was 76.1% compared to 75% for sessions shown only one check, indicating that multiple checks are not causing significant disruption or confusion to the editors.
  • Likelihood to include a reference
    • Sessions shown multiple checks are more likely to include at least one new reference in the final published edit compared to sessions shown just a single check.
    • In the test group, 52.5% of all published edits shown multiple Reference checks included at least one new reference compared to 39.7% of edits that were shown a single check.
  • Disruption (revert and block rates)
    • In the test group, the revert rate of new content edits shown multiple Reference Checks (17%) is currently lower compared to sessions shown a single Reference Check (26%).
    • No significant changes in the proportion of users blocked after being shown multiple Reference Checks compared to a single Reference Check.

In December 2024, we released a new design for the Edit Check desktop experience. This change shifted Edit Checks from appearing within articles to appearing alongside them, in a new "siderail."

To decide how – if at all – this change impacted volunteer disruption and edit quality, we compared analyzed several key metrics shifted before and after this change.

Findings

• The revert rate of new content edits where Reference Check was activated decreased by 15.7%.
  • 20.4% (pre) →  17.2% (post).
• The proportion of new content edits that included a reference following the change increased 8%.
  • 34.8% (pre) → 37.9% (post)
• Excluding reverted edits, there was 3.2% increase [2 percentage points] in edit completion rate.
  • 68% of edits where Reference Check was presented were successfully saved and not reverted following the change in the Edit Check UX.
• The rate at which people declined to add a reference when Edit Check prompted them decreased 4.7%.

Conclusion(s)

The findings above suggest this design change has been net positive. As a result, we will continue depending on this new design paradigm as we introduce new types of Edit Checks and moments within the editing workflow when they are presented.

To learn whether the Reference Edit Check is effective at causing newcomers to make edits they intended and experienced volunteers value , we conducted an A/B experiment with 15 Wikipedias.

Below you can read more about what this experiment demonstrated, what the Editing Team is planning in response, and more details about the experiment's design.

Conclusion and next step(s)

Reference Check caused an increase in the quality of edits newcomers publish and did not cause any significant disruption.

This combination is leading the Editing team to be confident that offering Reference Check as a default-on feature would have a net positive impact on all wikis and the people who contribute to them.

You can read the full A/B test report here.

New content edits *with* a reference

People shown the Reference Check are 2.2 times more likely to publish a new content edit that includes a reference and is constructive (not reverted within 48 hours).

• Increases were observed across all reviewed user types, wikis, and platforms.

• The highest observed increase was on mobile where contributors are 4.2 times more likely to publish a constructive new content edit with a reference when Reference Check was shown to eligible edits.

Revert rate

• New content edit revert rate decreased by 8.6% if Reference Check was available.
  • New content edits by contributors from Sub-Saharan Africa are 53% less likely to be reverted when Reference Check is shown to eligible edits.

While some non-constructive new content edits with a reference were introduced by this feature (5 percentage point increase), there was a higher proportion of constructive new content edits with a reference added (23.4 percentage point increase). As a result, we observed an overall increase in the quality of new content edits.

Constructive Retention Rate

• Contributors that are shown Reference Check and successfully save a non-reverted edit are 16 percent more likely to return to make a non-reverted edit in their second month (31-60 days after).
  • This increase was primarily observed for desktop edits. There was a non-statistically significant difference observed on mobile.

Edit Completion Rate

• We observed no drastic decreases in edit completion rate from intent to save (where Reference Check is shown) to save success overall or by wiki.
• Overall, there was a 10% decrease in edit completion rate for edits where Reference Check was shown.
  • There was a higher observed decrease in edit completion rate on mobile compared to desktop. On mobile, edit completion rate decreased by -24.3% while on desktop it decreased by -3.1%.

Block Rate

• There were decreases or no changes in the rate of users blocked after after being shown Reference Check and publishing an edit compared to users in the control group.

False Negative Rate

• There was a low false negative rate. Only 1.8% of all published new content edits in the test group did not include a new reference and were not shown Reference Check.

False Positive Rate

• 6.6% of contributors dismissed adding a citation because they indicated the new content being added does not need a reference. This was the least selected decline option overall.

11 Wikipedias participated in the experiment. At each wiki, 50% of users were randomly assigned to a test group and 50% were assigned to a control group.

Users in the test group were shown the Reference Check notice prompting them to decide whether the new content they were adding need a reference (if they had not already added one themselves).

User in the control group were shown the default editing experience, even if they did not accompany the new content they were adding with a reference.

This analysis was completed on 16 April 2024 and analyzed engagement data at the 11 participating wikis from 18 February 2024 through 4 April 2024.

This initiative sits within the larger Edit check project – an effort to meet people while they are editing with actionable feedback about Wikipedia policies.Edit Check is intended to simultaneously deliver impact for two key groups of people.

Experienced volunteers who need:

1. Relief from repairing preventable damage
2. Capacity to confront complexity

New(er) volunteers who need:

1. Actionable feedback
2. Compelling opportunities to contribute
3. Clarity about what is expected of them