Wikipedia talk:Exceptional reviews

Background

For background, see Wikipedia talk:Good Article proposal drive 2023#Proposal 4A: Recognize exceptional reviews from the January 2023 GA proposal drive. czar 03:51, 27 February 2023 (UTC)[reply]

Process workshop

Pulling from the proposal drive discussion, we have some options so I wanted to open a workshop. I started with a few basic ideas: (a) That this shouldn't be any more complicated than need be, (b) That we're looking to select and encourage exceptional reviews with a community affirmation process (rather than a unilateral barnstar, which any editor can send), (c) That we we need no set criteria/benchmark as long as we have a common comparative period. I wrote out the following based on what I had in mind, for discussion:

(1) Anyone may nominate a review for consideration (including peer, GA, FA, and talk page reviews) with a concise rationale for what distinguishes the review from the norm. Please use {{u}} to notify the reviewers under consideration. Self-nominations are welcome.
(2) Editors peruse and discuss these nominees when posted.
(3) In the last week of the month, nominees go up for a vote by acclamation. All may comment but only support votes by registered users in good standing are tallied. After the 7th day (UTC) of the new month, the top nominees are recognized.

This combines the FASA and Milhist project-style vote by acclamation during a common nomination period. A rolling 30-day nomination period would be easier to maintain, but we'd have no reference point without multiple noms in comparison.

On timing, I wanted to give a cutoff point and an end point for a comparative period, so I figured that the 14-day period at the turn of the month is the easiest way to do that, with late nominees held to the next period. I don't think there's any issue with people supporting noms early (before the comparative period) as long as there is a common consideration phase (minimum of 14 days) and an end date.

Pinging discussion participants @Unexpectedlydian, Kusma, Etriusus, JPxG, Iazyges, Oltrepier, Gen. Quon, Chipmunkdavis, and SandyGeorgia for your feedback:

Would this proposal work?
Should we limit to two top nominees per month?
Should there be an expiration date for nominating older reviews?

czar 03:51, 27 February 2023 (UTC)[reply]

Discussion

Do you have an idea on how the voting system will work? The phrasing 'the top nominees are recognized' implies that only a set number of people can be recognized per month. I'd personally like to see it where if you get, say 10 votes in favor, you're given the award rather than having to compete. It also avoids the issue of what happens if no reviews are of decent quality. I'm more than okay opening a dedicated tab on the tab header for the review recognition page, and would prefer to have it out in the open since we historically have a problem of burying things in redirects. 🏵️Etrius ( Us) 14:00, 27 February 2023 (UTC)[reply]

That's a good point. It does seem that adding a competitive bent to this might cause some issues. Conversely, a competitive edge might also encourage participants to produce good reviews so that they can be recognized.--Gen. Quon_[Talk] 14:33, 27 February 2023 (UTC)[reply]
As long as there are no problems with "review farming" (although that might be pretty difficult, given the standard required), I support this point, too. Oltrepier (talk) 18:06, 27 February 2023 (UTC)[reply]
I started with two per month as an arbitrary cutoff (better than one sole "review of the month") but yes that's part of why I wanted to open it for feedback. There's no issue of scarcity but when there isn't a common period, I can see it becoming a popularity contest/vote, subject to the whims of who shows up and when rather than as a slate of several candidates to review. czar 08:08, 4 March 2023 (UTC)[reply]
I also think overall approval is preferable to monthly. Say we take the top two each month, and then there are three great reviews in January but only one in February. A great one in January would be lost, while a sub-par one in February would be awarded. We'd also need to determine what actually constitutes an exceptional review. Is it one that thoroughly checks for all of the good article criteria? Because that should be most if not all reviews. Thebiguglyalien (talk) 22:45, 27 February 2023 (UTC)[reply]
Good question on what should constitute an exceptional review. I don’t want to over-prescribe or over-complicate things so it should probably be down to the nominator of the exceptional review to give a reason. Then, people can judge whether the reason is sufficient by voting or not voting. To give examples of what types of reviews I was thinking of: maybe ones where the reviewer has helped the nominator turn around what might have otherwise been a fail, or a thorough review of an article which has had a turbulent history. Unexpectedlydian^♯4 talk^‽ 17:30, 28 February 2023 (UTC)[reply]
I like the idea of having a threshold of x votes for a review to qualify as a quality review. Not sure how we would set a bar, though. If loads of people participate in voting, 10 could be quite a low threshold. Maybe instead it could be something like x% of all votes casted that month? Additionally, presumably there would have to be an expiry date for each review. For example: a review is nominated on 1 January, other editors can vote for it until 1 February, if it receives x votes then it qualifies as a quality review. Unexpectedlydian^♯4 talk^‽ 17:20, 28 February 2023 (UTC)[reply]
How do you reconcile x% of votes per month with the idea of a rolling window? czar 08:10, 4 March 2023 (UTC)[reply]

@Olivaw-Daneel:

I have had to do almost 100% of the work at WP:FASA to make it happen, and advertise for feedback; for this to work, someone has to take the lead. I wonder if monthly is overkill, and quarterly may be better. Also, FASA has a designated person (FAR Coords) to close the noms and send the rewards, not sure how this would work without that. SandyGeorgia (Talk) 19:38, 27 February 2023 (UTC)[reply]

(Sub-thread about GA-save awards; feel free to move if offtopic)@SandyGeorgia: yes, I'd be willing to volunteer to make it work. One possibility for GASA is that a single uninvolved editor could decide on the award, mirroring GAN which is a single-reviewer process. I like the monthly/quarterly round-up suggested here as well, might make things easier. Olivaw-Daneel (talk)

That's the ticket. SandyGeorgia (Talk) 20:51, 27 February 2023 (UTC) [reply]

@Czar: Just one more question: so, the voting process by acclamation should be similar in concept to a process like, for example, an AfD discussion or an In The News candidate review, right? Oltrepier (talk) 20:13, 27 February 2023 (UTC)[reply]
AfD and ITN are by consensus, so editors side for or against and someone closes the discussion on behalf of the group's judgment, whereas acclamation is more like a voice vote, where those in favor say aye and the one with the most support is chosen (see FASA and Milhist examples above). czar 23:24, 27 February 2023 (UTC)[reply]
@Czar Got it, thank you for clarifying. Oltrepier (talk) 09:40, 28 February 2023 (UTC)[reply]
Why not keep the competitive aspect, but disqualify past winners? This ensures a varied number of entries is steadily collected. I don't think that such a disqualification could be gamed, although it could be time-limited. An issue may emerge if it ends up disqualifying everyone though. CMD (talk) 09:51, 3 March 2023 (UTC)[reply]

Icon workshop

@JPxG, you had mentioned repurposing an icon you had used previously. I like the fuchsia motif but was considering how to distinguish a “review” award from a regular/unadorned article award. One thought was to add some kind of overlay atop the icon (à la ) but with something like an "👀" icon to represent the act of reviewing. czar 03:51, 27 February 2023 (UTC)[reply]

It definitely needs some sort of overlay or other distinguishing mark. Like I said in the proposal drive, it can't just be a color shift, because we need color-blind accessibility. Thebiguglyalien (talk) 03:59, 27 February 2023 (UTC)[reply]

I have used 📝 before as an icon to signify reviewing. Unexpectedlydian^♯4 talk^‽ 17:33, 28 February 2023 (UTC)[reply]

The pencil icon more readily signifies "writing" or "editing" to me (like the {{wikidata pencil}}) not necessarily "reviewing". czar 08:13, 4 March 2023 (UTC)[reply]

Simpler proposal

Would there be interest in a simpler option like this:

A template that any user could place at the top (or bottom if that's preferred) of any review.
A category that would be populated by articles with the template.
Perhaps a bot that randomly selects one noted article from the past month/quarter to highlight either on WT:GA or some kind of reverse notice board?
Perhaps a bot that congratulates the reviewer?

This could reduce the overhead that stalled out the earlier proposal, and fit more neatly into many editors' existing workflows (templating articles, "wikilove" templates on user talk pages). Rjjⁱⁱⁱ (talk) 18:10, 28 October 2023 (UTC)[reply]

@Czar, Unexpectedlydian, Thebiguglyalien, Chipmunkdavis, Oltrepier, Olivaw-Daneel, Gen. Quon, Etriusus, Kusma, JPxG, Iazyges, and SandyGeorgia: pinging users active or pinged above to see if there is still any interest in the idea Rjjⁱⁱⁱ (talk) 02:34, 19 November 2023 (UTC)[reply]

@Rjjiii Thank you for flagging this!

I'm not sure about the third option, but all of the other ones look nice, and especially the first. Oltrepier (talk) 15:06, 19 November 2023 (UTC)[reply]

This is a neat idea. Would there be any risk of it being spammed/abused? And if so, how might we head that off at the pass?--Gen. Quon_[Talk] 17:46, 19 November 2023 (UTC)[reply]

@Gen. Quon @Rjjiii Wouldn't one of the two bots already take care of it, though? Oltrepier (talk) 21:26, 30 November 2023 (UTC)[reply]