Page MenuHomePhabricator

Fix pinnable element instrumentation in desktopWebUIActions
Closed, ResolvedPublic3 Estimated Story Points

Description

In T329333, we identified an issue with the click tracking for pinnable elements in the desktopWebUIActions schema.
In summary, the ToC and associated elements produce the following event.name values in the following configurations:

ToC placementSection clickDropdown button click[hide]/[move to sidebar] toggle click
page titletoc-{{section ID}}.unknown.unpinnedui.dropdown-vector-page-titlebar-tocpinnable-header.vector-toc.pin
sticky headertoc-{{section ID}}.unknown.unpinnedui.dropdown-vector-sticky-header-tocpinnable-header.vector-toc.pin
sidebarui.sidebar-tocN/Apinnable-header.vector-toc.unpin
floaty hamburger TOCtoc-{{section ID]].unknown.pinnedui.dropdown-vector-page-titlebar-tocN/A

The Dropdown button and [hide]/[move to sidebar] events are largely fine, other than the floaty ToC button having the same event name as the header ToC.
The main issue is the ToC section clicks in when they are a pinned state. There are a few reasons for this:
#1 The sidebar ToC has a parent element with an event-name attribute:

<nav id="mw-panel-toc" role="navigation" aria-label="Contents" data-event-name="ui.sidebar-toc" class="mw-table-of-contents-container vector-toc-landmark">

Whereas the ToC in a pinned state does not:

<nav role="navigation" aria-label="Contents" class="vector-toc-landmark">

However, simply adding this attribute to the nav elements leads to two events firing for one section click.
#2 There also seem to be issue with the WikimediaEvent/desktopWebUIAction.js getMenuLinkEventName() code which formats the event.name value. The pinnableElementHeader.dataset.name returns unknown and the ID is drawn from the section ID of the list item. This leads to the incorrect toc-{{section ID}}.unknown.unpinned values.

Event Timeline

ovasileva triaged this task as Medium priority.Mar 20 2023, 5:24 PM
ovasileva moved this task from Incoming to Current Quarter on the Web-Team-Backlog-Archived board.

Same behavior is also replicable in Page Tools (Logged In) Sidebar, as well as Dropdown

Page Tools placementSection clickDropdown button click[hide]/[move to sidebar] toggle click
Page Tools Dropdownt-{{section ID}}.unknown.pinnedui.dropdown-vector-page-tools-dropdownpinnable-header.vector-page-tools.pin
sticky headerN/AN/AN/A
sidebart-{{section id}}.unknown.pinnedN/Apinnable-header.vector-page-tools.unpin
Page Tools HamburgerN/AN/AN/A

See the rest of the QA results in T322204

LGoto set the point value for this task to 3.Mar 23 2023, 5:44 PM

Change 902523 had a related patch set uploaded (by Kimberly Sarabia; author: Kimberly Sarabia):

[mediawiki/extensions/WikimediaEvents@master] Adds condition for new data attribute

https://gerrit.wikimedia.org/r/902523

New events per patch

ToC placementSection clickDropdown button click[hide]/[move to sidebar] toggle click
page titleNEW toc-heading.toc-pinned-disabledui.dropdown-vector-page-titlebar-tocpinnable-header.vector-toc.pin
sticky headerNEW toc-heading.toc-pinned-disabledui.dropdown-vector-sticky-header-tocpinnable-header.vector-toc.pin
sidebarui.sidebar-tocN/Apinnable-header.vector-toc.unpin
floaty hamburger TOCNEW toc-heading.toc-pinned-disabledui.dropdown-vector-page-titlebar-tocN/A

Page Tools placementSection clickDropdown button click[hide]/[move to sidebar] toggle click
Page Tools DropdownNEW t-{{heading-ID}}.page-tools-pinned-disabledui.dropdown-vector-page-tools-dropdownpinnable-header.vector-page-tools.pin
sticky headerN/AN/AN/A
sidebarNEW t-{{heading-ID}}-page-tools-pinned-enabledN/Apinnable-header.vector-page-tools.unpin
Page Tools HamburgerN/AN/AN/A

@jwang Is the current level of specificity for the event.action as outlined in the table above enough, or do we need to further specify if the click is coming from the titlebar, sticky header, or sidebar?

For page tools, the instrumentation is specific enough to measure the questions in T321229.

For ToC, @ovasileva, can you list the questions you want to have data for?
With the instrumentation in T332612#8740244, we can measure

  1. the clicks on new ToC by location: leftside, or page title
  2. pin/unpin

Do we still need to compare to clicks on legacy ToC? What else you need answer for?

Have confirmed with @ovasileva in meeting that 1) we don't need to track clicks on legacy ToC as of now, 2) we will measure the distribution of clicks on new ToC per session per page in the future if needed. The current instrumentation is specific enough to answer this question.

Change 902523 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] Adds condition for new data attribute

https://gerrit.wikimedia.org/r/902523

Edtadros subscribed.

Test Result - Beta

Status: ✅ PASS
Environment: beta
OS: macOS Ventura
Browser: Chrome
Device: MBP
Emulated Device:NA

Test Artifact(s):

QA Steps

Verify the events for the scenarios in the tables below

ToC placementSection clickDropdown button click[hide]/[move to sidebar] toggle click
page titleNEWtoc-heading.toc-pinned-disabled see note belowui.dropdown-vector-page-titlebar-toc
Screenshot 2023-04-10 at 7.01.55 PM.png (357×778 px, 69 KB)
pinnable-header.vector-toc.pin
Screenshot 2023-04-10 at 7.07.20 PM.png (355×993 px, 71 KB)
sticky headerNEWtoc-heading.toc-pinned-disabled
Screenshot 2023-04-10 at 6.58.51 PM.png (353×784 px, 69 KB)
ui.dropdown-vector-sticky-header-toc
Screenshot 2023-04-10 at 6.56.32 PM.png (359×793 px, 69 KB)
pinnable-header.vector-toc.pin
Screenshot 2023-04-10 at 7.01.05 PM.png (360×779 px, 69 KB)
sidebarui.sidebar-toc
Screenshot 2023-04-10 at 6.54.25 PM.png (358×791 px, 73 KB)
N/Apinnable-header.vector-toc.unpin
Screenshot 2023-04-10 at 6.57.28 PM.png (360×787 px, 69 KB)
floaty hamburger TOCNEWtoc-heading.toc-pinned-disabled
Screenshot 2023-04-10 at 7.14.04 PM.png (354×784 px, 69 KB)
ui.dropdown-vector-page-titlebar-toc
Screenshot 2023-04-10 at 7.11.42 PM.png (355×800 px, 70 KB)
N/A

@ovasileva NOTE: The event happens when <1000px and the TOC is in the page title and you select a section. This wasn't in the table below, unless pagetitle-section click is mislabed as disabled.
toc-heading.toc-pinned-enabled

Screenshot 2023-04-10 at 7.04.15 PM.png (360×780 px, 69 KB)

Page Tools placementSection clickDropdown button click[hide]/[move to sidebar] toggle click
Page Tools DropdownNEWt-{{heading-ID}}.page-tools-pinned-disabled
Screenshot 2023-04-10 at 7.23.20 PM.png (357×797 px, 71 KB)
ui.dropdown-vector-page-tools-dropdown
Screenshot 2023-04-10 at 7.20.11 PM.png (352×805 px, 70 KB)
pinnable-header.vector-page-tools.pin
Screenshot 2023-04-10 at 7.21.33 PM.png (357×784 px, 73 KB)
sticky headerN/AN/AN/A
sidebarNEWt-{{heading-ID}}-page-tools-pinned-enabled
Screenshot 2023-04-10 at 7.24.19 PM.png (352×777 px, 70 KB)
N/Apinnable-header.vector-page-tools.unpin
Screenshot 2023-04-10 at 7.19.49 PM.png (354×798 px, 70 KB)
Page Tools HamburgerN/AN/AN/A

@jwang - not sure if you want to double-check if things look okay from your side.

@ovasileva , yes, I will double-check if fixed events are available. Please assign it to me when instrumentation QA is done on engineer side and it is ready for schema QA.

Hi and thank you for your interest! Please check thoroughly https://www.mediawiki.org/wiki/New_Developers (and all of its communication section!). The page covers how to get started, assigning tasks, task status, how to find a codebase, how to create patches, where to ask general development questions and where to get help with setup problems, and how to ask good questions. Thanks a lot! :)

@ovasileva , yes, I will double-check if fixed events are available. Please assign it to me when instrumentation QA is done on engineer side and it is ready for schema QA.

@jwang - thank you! It should be ready for schema QA. Instrumentation QA was done in T332612#8769874

Hi and thank you for your interest! Please check thoroughly https://www.mediawiki.org/wiki/New_Developers (and all of its communication section!). The page covers how to get started, assigning tasks, task status, how to find a codebase, how to create patches, where to ask general development questions and where to get help with setup problems, and how to ask good questions. Thanks a lot! :)

Hi @Edtadros , can you show me what's "floaty hamburger TOC"? Originally, I thought it was the scenario when the ToC at narrow width. But from your QA summary, it seems not. Thank you!

QA clicks on links on page tools

What have been verified
  • ✅ New field values have been logged

The events with new field value are available since 2023-04-11. The new field values in format of {{ }}.page-tools-pinned-disabled or {{ }}.page-tools-pinned-enabled.

  • ✅ The links on page tools have click events logged as expected

The events of following links are confirmed to be captured: What links here, Related changes, Special pages, Page information, Cite this page, Wikidata item, Who Wrote That, Delete, Move, Change protection, Download as PDF, Printable version.
The events of clicks on Who Wrote That are not logged. It's expected.

  • ✅ Old field values are not logged anymore.

The events with old field value are not available since 2023-04-16. The old field values were in format of {{ }}.vector-page-tools.unpinned or {{ }}.vector-page-tools.pinned.

Codebase: github link

QA clicks on table of contents

What have been verified
  • ✅ New field values have been logged.

The new event values, toc-heading.toc-pinned-disabled and toc-heading.toc-pinned-enabled have been captured since 2023-04-13.

  • ✅ Old field values are not logged any more.

The old event values, toc-{{}} , toc-{{section ID}}.vector-toc.unpinned, or toc-{{section ID}}.vector-toc.pinned, are not logged as of 2023-4-28. (ref: T329333)

  • ✅ All event names for ToC under vector 2022 skin are logged in DesktopWebUIActionsTracking schema as expected.

All 7 types of ToC events under vector-2022 are available in schema. The events are: toc-heading.toc-pinned-enabled , ui.dropdown-vector-page-titlebar-toc , ui.dropdown-vector-sticky-header-toc , ui.sidebar-toc , pinnable-header.vector-toc.unpin , pinnable-header.vector-toc.pin ,toc-heading.toc-pinned-disabled.
As Only logged-in users can see sticky header feature, it is expected that ui.dropdown-vector-sticky-header-toc events are only captured from logged in users.

  • ✅ ToC clicks under vector skin are not captured. Confirmed with Web team that it is expected.

Codebase: github link