Research:Newsletter/2021/August

Vol: 11 • Issue: 08 • August 2021 [contribute] [archives]

IP editors, inclusiveness and empathy, cyclones, and world heritage

By: Tilman Bayer

"The Hidden Costs of Requiring Accounts: Quasi-Experimental Evidence From Peer Production"

Wikipedia is unusual among user-generated content websites in that it allows contributions – even changes to widely read existing content – without creating an account, in exchange to contributors agreeing to have their IP address published alongside their edit. This has been a core feature since the project's beginnings, but has come to be questioned from several angles in recent years. In "The Hidden Costs of Requiring Accounts: Quasi-Experimental Evidence From Peer Production", well-known Wikipedia researchers Benjamin Mako Hill and Aaron Shaw provide some insights on what might happen if IP editing were to be disabled on Wikimedia wikis. As they summarize in the abstract:

"We conduct an empirical test using longitudinal data from 136 natural experiments where would-be contributors to wikis were suddenly required to log in to contribute. Requiring accounts leads to a small increase in account creation, but reduces both high- and low-quality contributions from registered and unregistered participants. Although the change deters a large portion of low-quality participation, the vast majority of deterred contributions are of higher quality."

These 136 wikis are drawn from a larger dataset from Wikia (today known as Fandom, a commercial wiki hoster co-founded by Wikipedia founder Jimmy Wales), consisting of wikis where local administrators had requested for IP editing to be disabled.

According to the paper's literature review (of general research on peer production),

"One body of prior studies argue that stable identifiers work as catalysts of cooperation by facilitating accountability, group identification, boundaries, and commitment that lead to higher quality participation. Others contend that requiring accounts imposes costly obstacles that deter participation. A related set of studies suggests that unregistered contributions may introduce diverse perspectives and stimulate activity among experienced community members ..."

Based on this, the authors come up with four hypotheses for the effects of requiring account registration:

• "an increase in the number of newly registered accounts (H1)"
• "a reduction in the number of subsequently removed contributions (H2)" (operationalized as reverted edits)
• "a reduction in high quality contributions (H3)" (operationalized via non-reverted edits and via content persistence)
• "a reduction in contributions from participants who contributed with accounts prior to the requirement (H4)"

In their analysis, the authors find "substantial discontinuous shifts in each of our dependent variables when wikis began requiring accounts", supporting all four hypotheses.

The authors first presented such results at Wikimania 2015 and published them in peer-reviewed form last year. Just a few months afterwards, a large Wikimedia wiki enacted such a change for the first time (see e.g. the Signpost's coverage: Portuguese Wikipedia bans IP editing). A recent evaluation of its impact by the Wikimedia Foundation does not quite match the results from the Wikia wikis, finding "no significant negative impact in the analysis conducted thus far" (see also the more detailed summary in this issue's "News and notes" section: "Portuguese IP ban on track"). The statistical techniques used in that analysis appear to be somewhat simpler (e.g. only considering year-over-year changes in monthly counts of active editors, in contrast to Hill's and Shaw's analysis of weekly data using a regression discontinuity design in the paper). On the other hand, the WMF analysis also considered additional metrics designed to measure "Impacts on administration", namely the number of user blocks, page protections, and Checkuser checks, finding (somewhat unsurprisingly) "a significant reduction in administration actions".

There are other reasons to question how much the Wikia results would generalize to Wikipedias or other Wikimedia wikis. For example, in recent years the Wikimedia Foundation has invested significantly in technical measures to improve the experience of newcomers. But IP editors do not benefit from such improvements ("newcomers" meaning newly created accounts), so requiring them to register might have the added benefit that every contributor to a wiki will have been reached by such newbie-supporting measures as they start to edit.

Other recent publications

Other recent publications that could not be covered in time for this issue include the items listed below. Contributions, whether reviewing or summarizing newly published research, are always welcome.

"New maps for an inclusive Wikipedia: decolonial scholarship and strategies to counter systemic bias"

From the abstract:

"Since early in the development of the project, Wikipedia editors have been concerned with overcoming “systemic biases” in coverage of the world’s knowledge, especially those rooted in forms of social marginalization. [...] However, many Wikipedia editors and observers have argued that the systemic biases of Wikipedia are inherent to current global distribution of knowledge production, and can only be overcome by changing the encyclopedia’s standards of inclusion. This article reframes this debate by comparing the project of “countering systemic bias” on Wikipedia with the effort within Western/Northern academia to decolonize and diversify scholarship. Since this project began at least fifty years ago, it has led to abundant peer-reviewed scholarship, all of which qualifies as “reliable sources” for Wikipedia articles. [...] The article proposes that critical scholarship, historical maps, and maps in contemporary scholarship can all contribute to addressing Wikipedia’s systemic biases."

"Empathy plasticity: Decolonizing and reorganizing Wikipedia and other online spaces to address racial equity"

From the abstract:

"... Focussing on a social psychological concept of “empathy plasticity”, the critical developmental window in which humans begin to formulate attitudes and dispositions on phenomena around them, this commentary explores how a re-organization of online spaces, such as Wikipedia and social media, can be used to more holistically present knowledge on topics in the realm of race, ethnicity, class, and gender. Such a technocratic shift is one means of content democratization and dialectical gatekeeping to ensure our contemporary epistemology acquires and maintains methodological, ideological and ethical merit."

"Wikiproject Tropical Cyclones: The most successful crowd-sourced knowledge project with near real-time coverage of extreme weather phenomena"

From the paper:

"Wikiproject Tropical Cyclones delivers the highest quality content of all Wikiprojects on English Wikipedia. We also find that both readership and editing of the articles on tropical cyclones are highly correlated with cyclones’ occurrences, which indicates that Wikipedia is a go-to source for many people [...] For the purpose of identifying the most successful Wikiprojects, we have ranked all projects based on the total number of FA and GA articles that are within their scope [...] Since projects vary in size, and the sheer number of articles can impact the relative number of quality articles, we normalized the totals by the square root of the number of articles that a project curates."

"World Heritage sites on Wikipedia: Cultural heritage activism in a context of constrained agency"

From the abstract:

"... we investigate the patterns of production, consumption, and spatial and temporal distributions of Wikipedia pages for World Heritage cultural sites. We find that Wikipedia provides a distinctive context for investigating how people experience and relate to the past in the present. The agency of participants is highly constrained, but distinctive, behind-the-scenes expressions of cultural heritage activism are evident. Concerns about state-like actors, violence and destruction, deal-making, etc. in the World Heritage inscription process are present, but rare on Wikipedia’s World Heritage pages. Instead, hyper-local and process issues dominate controversies on Wikipedia."

References

Wikimedia Research Newsletter
Vol: 11 • Issue: 08 • August 2021
About • Subscribe: Email • [archives] • [Signpost edition] • [contribute] • [research index]