Deployment timing
Tuesday, March 25, 2025
Bucketing criteria
Bucketing Requirements:
- Bucketing should include both registered and unregistered users at all identified participating wikis.
- All users, who are editing a desktop or mobile main namespace page (NS:0), at any of the participating wikis should have a 50% chance of being included/bucketed into the A/B test's control or treatment group.
- Bucketing should be done on a per-Wikipedia basis. 50% of people within a given wiki should be placed within the control group; 50% should be bucketed in the treatment group
- The test group should have multi-check (references) experience enabled while the control group should only have the single Reference Check enabled (the current default experience at those wikis).
- People should remain in the same test group for the duration of the test (and across sessions and pages).
Instrumentation-Related Requirements
- A bucket is applied to these events so we can distinguish all events logged for the control group and the test group within the A/B test.
- The bucket should be descriptive of the test in case there are every overlapping AB tests. For example, 2025-03-multicheck-reference-[test/control]
- The anonymous_user_token field is populated for unregistered users in the test.
Participating wikis
| Wiki | Feedback/Notes | Status |
|---|---|---|
| ar.wp | People are being bucketed in correct groups,bucketing includes both registered and unregistered users, users are assigned the same test group for the duration of the test and across sessions and pages | |
| afwiki | People are being bucketed in correct groups,bucketing includes both registered and unregistered users, users are assigned the same test group for the duration of the test and across sessions and pages | |
| eswiki | People are being bucketed in correct groups, bucketing includes both registered and unregistered users, users are assigned the same test group for the duration of the test and across sessions and pages | |
| frwiki | People are being bucketed in correct groups, bucketing includes both registered and unregistered users,users are assigned the same test group for the duration of the test and across sessions and pages | |
| igwiki | People are being bucketed in correct groups, bucketing includes both registered and unregistered users,users are assigned the same test group for the duration of the test and across sessions and pages | |
| itwiki | People are being bucketed in correct groups, bucketing includes both registered and unregistered users,users are assigned the same test group for the duration of the test and across sessions and pages | |
| jawiki | People are being bucketed in correct groups, bucketing includes both registered and unregistered users,users are assigned the same test group for the duration of the test and across sessions and pages | |
| ptwiki | People are being bucketed in correct groups, bucketing includes both registered and unregistered users,users are assigned the same test group for the duration of the test and across sessions and pages | |
| swwiki | People are being bucketed in correct groups, bucketing includes both registered and unregistered users,users are assigned the same test group for the duration of the test and across sessions and pages | |
| yowiki | People are being bucketed in correct groups, bucketing includes both registered and unregistered users,users are assigned the same test group for the duration of the test and across sessions and pages | |
| viwiki | People are being bucketed in correct groups, bucketing includes both registered and unregistered users,users are assigned the same test group for the duration of the test and across sessions and pages | |
| zhwiki | People are being bucketed in correct groups, bucketing includes both registered and unregistered users,users are assigned the same test group for the duration of the test and across sessions and pages | |
Tech/News Draft
The Editing team will test a new way to display [[ mw:Help:Edit check#ref | References Check ]]. During this test, we will show several invitations to add a reference (called "Multi-check") instead of limiting ourselves to a single one, as is the case today. 50% of users will see this new configuration, while 50% will have the default configuration. The test will be conducted on the following wikis: ar.wp, afwiki, eswiki, frwiki, igwiki, itwiki, jawiki, ptwiki, swwiki, yowiki, viwiki,zhwiki
Edit Check Configuration
For the purposes of this A/B test, we're going to strive for the wikis that will participate to configure Edit Check in a consistent way. Doing so will enable us to draw wiki-agnostic conclusions about the impact(s) Edit Check causes.
Prompted by the question @MNeisler posed in T346837#9180760 and what we talked about offline on 20 Sep,
Done
- A/B test is started
- @MNeisler to verify ≥1 day after test start date that people are being bucketed as expected