Wikipedia talk:Wikidata/2018 Infobox RfC

Pre-RfC discussion

• I think that Wikidata should only be allowed to be used when it is sourced, and that the Wikidata reference must included in the reference box, so it is easy to check, and that it must remain possible to override it from Wikipedia without leaving your Wikipedia edit mode of choice. Overriding it would mean either deleting it in some way, or manually providing an in-Wikipedia alternative. A null marker could be used to indicate that a parameter should not be displayed at all in a specific case (for example as the default for religion in BLPs). All editing of Wikidata must be done from in Wikidata, all editing of Wikipedia should be done from in Wikipedia, and the editor must be clearly aware of which environment they are in at all times. A link from Wikipedia edit mode to Wikidata edit mode that opens in a new tab or popup would be good for this purpose, as we don't want people to be confused about which project they are editing at any time. This does not appear to be covered by any of the current options. · · · Peter (Southwood) ^(talk): 10:43, 5 October 2017 (UTC)
• I think that the present option 4 is a non-starter. I tried that route years ago using Module:Wikidata, but it runs into trouble for two reasons:
  1. Enthusiastic new editors come along and see |year= – which is set to fetch Wikidata – then fill in a value locally, thereby defeating the whole point of importing data (projects-wide updateability/maintainability).
  2. There is now no way of indicating that we never want a value displayed in a particular field in a particular article, apart from messy hidden comments. See the discussion on Template talk:Infobox book #Wikidata, especially the subsection Template talk:Infobox book #Wikidata subsection 1 about ways of using blank or absent parameters suggested by Nikkimaria, and the desirability of disabling |genre= in specific articles, for example, as explained by Sarah at Talk:Night (book)/Archive 2#Infobox 2 --RexxS (talk) 19:21, 6 October 2017 (UTC)

     Present option 5 suffers a worse problem: no way at all to specify when we don't want that field to appear in an infobox. Propose merging options 4 and 5 into a new option ("Fill-absent-fields" / "Opt-out"?), where Wikidata values are used to fill absent fields, but parameters with blank values are not displayed at all— essentially the converse of present option 4. This would allow full local override. Snuge purveyor (talk) 20:07, 6 October 2017 (UTC)

• I think that the present option 3 is unlikely to be a useful choice. See Template talk:Infobox book #Wikidata subsection 1. Is anybody likely to want to go to the effort of adding infobox wikitext like this for every article where they want wikidata to be fetched when there's a simple alternative? --RexxS (talk) 19:31, 6 October 2017 (UTC)

{{Infobox book|<!-- See Wikipedia:WikiProject_Novels or Wikipedia:WikiProject_Books -->
| name          = '''Animalia'''
| image         = Animalia (book cover).jpg
| caption       = 
| alt           = Book cover: a larger picture framed by smaller pictures, all of which contain different animals, and title with author at the top
| author        = wikidata
| illustrator   = wikidata
| country       = wikidata
| language      = wikidata
| genre         = wikidata
| publisher     = wikidata
| release_date  = wikidata
| media_type    = wikidata
| pages         = wikidata
| isbn          = wikidata
| oclc          = wikidata
... etc.
}}

• Fully agree with this. Propose deleting present option 3 in favour of present option 3b. Snuge purveyor (talk) 20:10, 6 October 2017 (UTC)
• Consider a new editor. Field=wikidata is the only option that really makes it clear what the heck is going on on the page. All of the other options are varying levels of mystical values coming from nowhere and obscure template-coder keywords. Alsee (talk) 19:07, 8 October 2017 (UTC)
  • No: the [edit on wikidata] link also does this job, particularly when editors are using the visual editor (and Wikidata editing in the visual editor at some point in the future will also do this). Mike Peel (talk) 19:15, 8 October 2017 (UTC)
    • It's currently listed as a type of opt-in. If opt-in is the result I think it may be worth considering how opt-in should work. As far as the [edit on Wikidata], that provides jack-squat help to someone who clicks the edit button and is trying to figure out how to REMOVE wikidata or override it. And that's aside from the fact that the last thing I want to do is confuse the hell of new editors having to figure out both wikitext and ^nonWiki^structureddata. God-forbid some poor soul tries to put "1 October 2012 - 1 September 2013" as date for a Solargraph, or "1906 or 1907" as a year of birth. I think most wikidatans would find those painful or impossible. Alsee (talk) 19:41, 8 October 2017 (UTC)
      Before you try to answer those examples, consider what is going to be displayed in the article if the infobox has |fetchwikidata=date. Alsee (talk) 19:58, 8 October 2017 (UTC)
      • If the infobox has |fetchwikidata=date, then a field named "date" in the infobox will have a value fetched from Wikidata if it's sourced there, unless (i) a local value is provided in the infobox like |date=1 October 2012 - 1 September 2013 (which would display instead of whatever was on Wikidata); or (ii) the field is already suppressed by |suppressfields=date (whch would stop the "date" field from displaying). What's to consider? As for date of birth (P569), see Edward the Confessor (Q130005) for example (earliest date (P1319) = 1003; latest date (P1326) = 1005). Wikidatans are not as incompetent as you seem to think. --RexxS (talk) 20:15, 8 October 2017 (UTC)
        • RexxS, your response was utterly unhelpful. Perhaps you didn't follow the context above. However here is your chance to impress me. Please, teach me that wikidata is easier than I think. Given a NEW USER, and an infobox fetching a wikidata-DATE field, tell me how that new user clicking [edit on wikidata] is going to get the article to display my two examples. Alsee (talk) 21:09, 8 October 2017 (UTC)
          • Your entire example is utterly unhelpful. It's easy to make up contrived cases that require some knowledge of editing. How many NEW USERs can you draw my attention to who have been frustrated by their inability to edit Wikidata? For each one you find, I'll find you ten who have been frustrated by their inability to edit Wikipedia when they started. --RexxS (talk) 21:31, 8 October 2017 (UTC)
            • RexxS I hardly think "born in 1906 or 1907" is an unreasonable example. But let's up the ante. Instead of a NEW USER, let's go with someone skilled in wikidata. Let's take you as our editor in question. Given an infobox fetching a DATE field, you click the [edit on wikidata] button. Now walk me through how you would make the wikidata edits to show (A) "1 October 2012 - 1 September 2013" and (2) "1906 or 1907" in the article. These are trivial tasks in a standard non-wikidata infobox. I most profusely and most graciously thank you in advance for improving my wikidata knowledge, and teaching me how to do this myself. Alsee (talk) 22:16, 8 October 2017 (UTC)
              • @Alsee: RexxS already answered your second example by pointing to the date of birth entry at Edward the Confessor (Q130005). For your other example, maybe look at the construction values at South Pole Telescope (Q1513315). Of course, these can also be locally defined if that's easier - although in that case they'd probably get reverted as unreferenced, and understanding enwp citations templates is a whole new level of complicatedness. Mike Peel (talk) 22:35, 8 October 2017 (UTC)
                • Mike Peel, perhaps I am mistaken but I do not believe either of my examples were answered. Could you please clarify what will appear on the article page for fetchwikidata DATE, if someone attempts to use [edit on wikidata] to update a year field to "1906 or 1907"? Will the article show both 1906 and 1907 and clearly indicate the "or" relationship between those numbers? Alsee (talk) 23:56, 8 October 2017 (UTC)
                  • @Alsee: OK, let's look at some live examples. See Ariane in Naxos where "c. 1630" is shown (which seems to be better/more standard here than "this or that year"), and see South Pole Telescope for the date range. Once the information is in a structured format, other styles can be coded up as needed. And, of course, it can be defined locally if that doesn't work quite right. Thanks. Mike Peel (talk) 00:15, 9 October 2017 (UTC)

Arbitrary break 1

• Added case-by-case options to all three tables. Also split the policy-compliant option of the second table in two slightly different ones: i.e., policy-compliant for Wikipedia (there's no "policy" in Wikipedia that data which are duly referenced in the body of an article need a repeat of that reference in an infobox), and Wikidata complying to the same policies as Wikipedia. --Francis Schonken (talk) 20:05, 8 October 2017 (UTC)
  • Francis Schonken, you caught an oversight/error in the Policy-compliant wording. "Double-safety policy-compliant" is silly both in name and in practice. I'll try to sort this out as one good listing. Alsee (talk) 20:19, 8 October 2017 (UTC)
    • Tried "two-tier policy compliance" – does that sound better? --Francis Schonken (talk) 20:43, 8 October 2017 (UTC)
• @RexxS:
  • re. " – even though there is no requirement at present for the infobox to stop displaying the information if a local value is altered, which also in that case may go unnoticed by Wikipedia editors." – seems a misunderstanding: in the case a local value is altered that shows up in the Wikipedia edit history of the page, there's nothing different there with non-Wikidata infoboxes: the whole RfC is about different ways to implement Wikidata infoboxes, not about what remains unchanged whether or not it is a Wikidata infobox.
  • re. "Editors may decide to hide changes made on Wikidata from their Wikipedia watchlists, which reduces the chances of vandalism and divergent values being noticed." not specific to 2E
  • re. "d:Wikidata:Verifiability, d:Wikidata:Biographies of living persons exist, but are much less developed than on Wikipedia. Wikipedia has no policy on WP:Reliability. The present Wikipedia policy on WP:Identifying reliable sources has rudimentary coverage in the Wikidata verifiability proposal." Please make up your mind or they exist, or they don't: it doesn't help if the fourth column says something different from the third. --Francis Schonken (talk) 21:06, 8 October 2017 (UTC)
    • (edit conflict) Nobody uses article histories to check for ongoing changes to articles they are interested in; they use their watchlist. Why should we insist on hiding a value changed on Wikidata, but accept a value changed directly on Wikipedia? What proportion of bad edits occur on Wikidata and are not noticed either there or here? What proportion of bad edits occur on Wikipedia and are not noticed here? You don't know, but you are pre-emptively imposing a solution on a problem that you have no measure of.
    • Many of the details are not relevant solely to the option where they are presented. That's not a good reason to exclude them when they are useful to put context to a particular option. Feel free to repeat them.
    • Are you having a reading problem? Those pages exist on Wikidata: the first is a proposed policy and the second is policy. It certainly doesn't help if the third column misrepresents the situation, repeating the usual "fake information". --RexxS (talk) 21:22, 8 October 2017 (UTC)
      • RexxS, I thought we resolved this when you tried to make the same edits on the Wikidata/2017_State_of_affairs page. No one here gives flying-fig if Wikidata has dead-ass pages for failedinactive-for-years proposals, or nothing-pages mentioning the WMF-Board's advice. Alsee (talk) 21:34, 8 October 2017 (UTC)
        • Don't quote that half-arsed propaganda page to support you. It cuts no ice with me. Wikidata has a proposed policy on verifiabilty which goes some way to cover relaible sources as well. Wikidata has a policy on BLP. You need to stop parrotting the fake info you've been fed. --RexxS (talk) 21:39, 8 October 2017 (UTC)
          • RexxS, nobody disputes that the page exists. I am baffled how you cannot understand that one gives a flying-fig about {{Failed proposal}}sinactive-for-years proposals. A failedinactive-for-years proposal on Wikidata is even more irrelevant than a failedinactive-for-years proposal on EnWiki. There are areas were we respectfully disagree, however tenaciously trying to cite a failedinactive-for-years proposal just comes across as desperate. If you truly insist on adding a link to it in the RFC, that link WILL be labeled as a failedinactive-for-years proposal. That undermines Wikidata's credibility even more than leaving out the link. Alsee (talk) 22:07, 8 October 2017 (UTC)
            • @Alsee: Your comment here breaches WP:CIVIL and WP:PERSONAL. Please revert/revise it. Mike Peel (talk) 22:40, 8 October 2017 (UTC)
              • Mike Peel, I believe this is the first time anyone has ever accused me per CIVIL or PERSONAL. Based on some non-sanctionable cases I've seen, I believe I'm on the safe side of the line. Nonetheless it was probably the most strongly worded comment I've ever posted, and I edited it to tone it down. The original is now buried in the history. As an unrelated matter, I struck all of my "failed proposal" mentions. I believe I was recalling a Wikidata policy proposal for BPL or ReliableSource which had failed. Alsee (talk) 00:22, 9 October 2017 (UTC)
      1. What shows up in article history shows up in watchlists in Wikipedia for the Wikipedians who have watchlisted the page. "May go unnoticed" refers to Wikipedia editors who don't know (yet) they have to turn on Wikidata watchlisting, or who never watchlisted the page, and thus have only the article history to go by.
      2. If it's a general remark it should go in the #Background information section
      3. Try to sort it in the third column, or put it all in the fourth column. I see Alsee finetuned a bit already, but anyhow third and fourth column shouldn't contradict one another. --Francis Schonken (talk) 21:37, 8 October 2017 (UTC)
      • Viewing of Wikidata changes is easy and fairly obvious to enable for those who want to keep track of Wikidata changes. Editors don't use article histories to keep track of changes to articles they are interested in; they watchlist them. That's the point of having a watchlist.
      • If it's a detail relevant to the wording of the option, it goes in the "Detail" column. You don't get to decide where I choose to make my edits.
      • You sort it in the third column - that's the one that contains the fake info. And don't tell me what edits to make. --RexxS (talk) 21:49, 8 October 2017 (UTC)
        • At the risk of telling you what edits to make ( ;-) ), it might be worth covering this and its nuances in the "What about references?" section rather than in the tables. Thanks. Mike Peel (talk) 21:53, 8 October 2017 (UTC)
          • Thanks for the suggestion, Mike, but I think I'm done here. I'm sick of the peanut gallery second-guessing what's possible and quoting the fake propaganda from the Wikidata-haters. I'm taking this off my watchlist. --RexxS (talk) 22:09, 8 October 2017 (UTC)
• Just to note: I've opened up this talk page to separate this discussion from the content. When the RfC opens then we can re-open discussion on the RfC page. Thanks. Mike Peel (talk) 21:42, 8 October 2017 (UTC)
• Are the Case-by-case options useful? They do exactly zero to resolve the ongoing conflicts, and they have little practical reality. It's not like people on individual pages are forming consensus picking what infobox to use and how it should be coded, it's a tiny number of people who go around changing infoboxes across vast numbers of articles. Alsee (talk) 07:34, 9 October 2017 (UTC)
  • Re.:
    • "Are the Case-by-case options useful?" – yes
    • "They do exactly zero to resolve the ongoing conflicts" – maybe the ongoing conflicts will remain unresolved as long as they're framed the way they are now
    • "they have little practical reality" – don't think so
    • "It's not like people on individual pages are forming consensus picking what infobox to use and how it should be coded" – I could give links to dozens of sections on different article talk pages which do exactly that (quite a few of them in RfC format too), and that are only the ones I know about.
    • "it's a tiny number of people who go around changing infoboxes across vast numbers of articles" – which is OK for infoboxes that usually have little problems on the question of whether one should be included or not (Wikidata application has happily enough not yet moved to these more contentious types of infoboxes). In the more contentious realms (think ArbCom cases, the type of infoboxes which are often referred to with the "war" moniker) a decision on whether or not an infobox is included in a particular article (discussions need to be article-by-article per ArbCom decision) is always coupled with a discussion of the content and values retained in the infobox.

  I'm thinking e.g. a |precedence= parameter, which could be set to "local" or "Wikidata" and decides which values to take for all fields that have as well locally defined values as Wikidata-defined values; such parameter would not be too difficult to implement. --Francis Schonken (talk) 08:18, 9 October 2017 (UTC)

Reference requirements

I'd like to support the options 2D and 2E in "Reference requirements" when the RfC starts, but I can't support an option if there is a pre-judged condition tacked onto it like "It seems that this would require the infobox to stop displaying the information if the Wikidata value is altered" which is completely impractical to code at the project level. As far as i can see, I'd need to request the devs to modify the php or the database for Wikidata to flag changed data and make that available via the Lua API, so that the modules that fetch Wikidata could tell when data has changed on Wikidata. That's so unlikely to happen, you might just as well support "1A - Wikidata is not used in infoboxes" because there's no way such a demanding constraint would be practical in any foreseeable timescale. --RexxS (talk) 22:06, 8 October 2017 (UTC)

Systemic issue that has serious implications for increased use of Wikidata

There's what appears to be a rather large systems issue for projects that use a lot of wikidata: Phabricator ticket. Given that the issue is so severe that it has resulted in Wikidata changes being removed (until resolution) from the watchlists and recent changes pages of some projects that are comparatively heavy users of Wikidata - one of the big selling features of linking to Wikidata - this needs to be fixed before Enwiki can logically proceed. From the looks of things, this is not going to be a quick fix, since it goes right to the heart of watchlists and recent changes. Risker (talk) 02:32, 10 October 2017 (UTC)

Some choice comments from that alarming ticket. The problems were first detected when people with large watchlists on some wikis (e.g. Commons, rowiki, itwiki) reported very slow watchlists. "Its how wikidata is used in infoboxes. en wikipedia is kind of skeptical of using wikidata in infoboxes, so there is less wikidata entries there. Commons for example uses author infoboxes all over the place that use wikidata." On 7 october, the database administrators became aware of this ticket, its origin, and the issues it caused, and quickly escalated things: "This was a) Not discussed previously with us DBAs b) created huge performance issues c) Made maintenance impossible d) break recentchanges functionality for users e) break watchlist functionality for users f) after report of breakage almost 3 months ago, no workaround/mitigation as been done g) we are close of running out of space on several main database servers, breaking all of Wikipedia h) propose to create even newer indexes, which only makes all of the above problems worse." (emphasis mine, but alarming comment made by someone who should really know what they are saying) And then yesterday: "Notification for users: We are going to disable wikidata recentchanges (meaning, changes on pages on other wikis coming from changes done on wikidata; the recentchanges at wikidata is not a problem) due to performance concerns. Once those have been fixed, the functionality could be enabled again."

So it is clear that at the moment, using Wikidata in infoboxes comes with a serious cost, which luckily hasn't affected enwiki too much yet. This is something that most likely wasn't known to the users implementing these Wikidata-infoboxes either, so I don't mean it as something they can be blamed for. However, it is somethig we should take into account moving forward from now on. Fram (talk) 07:02, 10 October 2017 (UTC)

See also the ticket linked in the next section, [1]: "On dewiki I have a lot of articles about people on my watchlist. Every change on i.e. Q5 (=human) makes my watchlist unusable for hours." (emphasis again mine). Fram (talk) 07:05, 10 October 2017 (UTC)

Hey folks :) We're working on improving this. Marius has prepared a number of patches that will significantly improve the situation by introducing more specific Lua functions. A large part of the issue comes from the specific way Commons and the Russian Wikipedia are using Wikidata's data. This should not be anywhere near as much of an issue for English Wikipedia. However the new things Lua functions Marius is working on will also make things better here. --Lydia Pintscher (WMDE) (talk) 18:08, 10 October 2017 (UTC)

"On dewiki I have a lot of articles about people on my watchlist. Every change on i.e. Q5 (=human) makes my watchlist unusable for hours. I know how to filter but then Wikidata changes directly related to the watched Wikipedia article are filtered too :-("[2] (I said dawiki in previous comments, should have been dewiki). Apparently the problem isn't just these two wikis (and itwiki, and ...), but a more general problem with Wikidata changes heavily polluting watchlists and recent changes (and the databases). It is indeed not so much an issue for enwiki, who have so far been intelligent enough to keep Wikidata largely out of it. Fram (talk) 20:52, 10 October 2017 (UTC)

• As I have mentioned on the Phab ticket itself, based on the research of some of the developers, it seems this is more a problem of tacking more and more tables into recent changes/watchlist, because there are indications that other software (ORES) is possibly also having an effect. There is also a strong desire (and in fact an action plan) to provide similar feeds of changes from Commons into Wikipedia RC/watchlists, which would likely also have the same effect. I think the reason this bug was surfaced was that Wikidata has been so strongly embraced by some communities - over 90% of edits on the two most affected wikis were from Wikidata. (I was surprised to see that Wikidata makes up about 20% of edits to Enwiki.) This is a systemic issue, and I really do not think it is just about Wikidata; the use of Wikidata just triggered it. Risker (talk) 21:02, 10 October 2017 (UTC)
  • But Wikidata has much more frequent minimal edits than e.g. Commons. "Random article", first one with an article: Miyazaki Kūkō Line: four Wikidata edits in August, and five more across 2017; compared to one edit to the enwiki article this year (and none in 2016 and 2015), and none in 2017 at Commons. This seems to be the rule, not the exception; Wikidata is much more edit-intensive in general, creating much more entries than other wikis. Fram (talk) 04:35, 11 October 2017 (UTC)
• @Lydia Pintscher (WMDE): this RfC may result in further restraining use of Wikidata on en.Wikipedia or further expanding that use. If the latter is the case, from what point should we get worried (or: ...never)? Maybe let's ask the question this way: an optimistic estimate may be that use of Wikidata would be doubled every month on this Wikipedia from now on, so if that is the case, how many months would we still have before the slowing down of loading a large en.Wikipedia watchlist gets problematic? And how many more months before the Wikidata/en.Wikipedia coupling of watchlists would need to be turned off for overburdening the system? How much guarantee can WMF give that neither of these two issues will occur before the technical aspect is addressed comprehensively? --Francis Schonken (talk) 07:02, 11 October 2017 (UTC)
  • It heavily depends on the way Wikidata's data is used. The problematic usage on Commons and Russian Wikipedia will be addressed with a patch that is going live in the next days as well as improvements to their Lua modules. A more complete overhaul of the system is necessary for long-term scalability. We are discussing different options for this. However it should not be necessary for further usage on English Wikipedia. Now for your concrete example of doubling the usage every month: I don't think this is anywhere near realistic - not even Commons manages anywhere near that and there it is much much easier to increase it than here because a huge number of pages make use of the same templates. --Lydia Pintscher (WMDE) (talk)
    • Re. "doubling the usage every month: I don't think this is anywhere near realistic" – Disagree. Let's do the math:
      • Currently there are 92 full mainspace infoboxes using Wikidata, and 7 more under development (recognisable by the "/Wikidata" extension of the name here); also there are currently less than 200 mainspace uses of the {{Cite Q}} template. Afaik these are the two main implementations for pulling Wikidata to Wikipedia (if I missed a major one, could someone correct me?)
      • Wikipedia template programmers get more experienced with infobox-to-Wikidata conversions, and the current RfC hopefully would fix some rules on acceptable conversion formats. If there are three experienced Lua programmers, each converting one infobox per day per the fixed ruleset and the increasing experience making this a standard operation, there's no reason why we shouldn't have 184 infoboxes next month. Ideally the RfC would leave a set of rules that are easy to implement by these programmers (I don't say that *will be* the outcome of the RfC, but optimistically there's no reason to say it can't be), and programmers can share their experience with others, so next month we may have 3 programmers doing two infobox conversions per day, or six programmers still doing one per day, and so on. Concurrently, as we speak, others are cleaning out the {{Cite Q}} code, and once these issues are sorted, the template may (optimistically) find wide acceptance, also leading from less than 200 mainspace implementations this month to 400 next month, and a continued exponential growth of usage of this (and similar) quotation templates: once the general Wikipedia editing community becomes aware how easy it is to use (as compared to filling in the individual parameters in traditional cite templates), and more Q items become available for producing decent Cite Q results, there's no reason to preclude that such exponential growth would be unrealistic in an optimistic scenario. Also, up to now, experimentation has taken place mostly on templates that have less heavy mainspace use, in order to limit risks of mainspace disturbance during the initial phase: the growth speed of Wikipedia calling Wikidata values may gain much more momentum once the "heavy use" infoboxes are included in the system: e.g. when {{Infobox sportsperson/Wikidata}} goes live at {{Infobox sportsperson}} there will be, with one single "copy-paste" operation, another 37,000 pages pulling multiple values from Wikidata – which can be compared to e.g. {{Infobox badminton player}}, currently full Wikidata, with less than one tenth of that number of transclusions.

    So please, Lydia Pintscher (WMDE), instead of commenting on what is likely/unlikely to happen in Wikipedia editing environment (which is apparently not your cup of tea, with all due respect), could you answer the questions based on this scenario, it is not half as unrealistic as you seem to imply? Tx. --Francis Schonken (talk) 11:50, 12 October 2017 (UTC)

    • (ec)"doubling the usage every month: I don't think this is anywhere near realistic " It depends: there is a Wikidata version of the infobox "person". The standard (non-wikidata- version is used on more than 250,000 pages: if the wikidata version would become the default (as has happened with some infoboxes with minimal use already), we wouldn't be discussing doubling the usage, it would be a much more massive increase instead. Fram (talk) 11:56, 12 October 2017 (UTC)
      • You assume that every usage of Wikidata's data is the same and has the same consequences. That is not the case. I assume the usage pattern on English Wikipedia to be very different from Commons and the usage I am seeing so far confirms this for me. A large part of the reason why the usage on Commons was so problematic is going to be fixed with the new Lua modules we are introducing and a smarter tracking of existing usage. I am not saying nothing bad can ever happen again but let's please not exaggerate things. --Lydia Pintscher (WMDE) (talk) 14:01, 13 October 2017 (UTC)
        • Which is not a reply to my comment, it seems, just some vague hope and handwaving. You are discussing the "usage so far", I am giving you an example of a potential but very realistic use case of a very popular infobox being changed to full wikidata mode, impacting on many more watchlists and massively on recent changes. Why, if someone takes the trouble of providing you with a real life example, do you continue with assumptions based on the current, limited use instead? Fram (talk) 14:09, 13 October 2017 (UTC)
        • Also, for the current RfC draft it would mean that whatever the community wants we cannot change our current use of Wikidata patterns (which would effectively make the RfC moot, while its objective is to see whether we want to change that MO w.r.t. Wikidata usage, for instance intensifying it with a manifold). Also the effect on what shows up in watchlists is something at the root of why Wikipedia editors would choose one or another of the options proposed in the RfC: failing a clear picture, we'd need to say that it is "uncertain" what shows up in Wikipedia watchlists, putting one of the key assumptions about the en.Wikipedia-Wikidata integration (namely, that when you put a page on your watchlist you can follow all changes that affect its content from your Wikipedia watchlist) on a slippery slope.

        My impression is that when the watchlist overload issue broke that the WMF had no clue this was going to hit them, and that the recommendation now continues to be: pretend you don't know anything about what might hit you, while that's what we did, and that was an acceptable course of action, no? --Francis Schonken (talk) 14:36, 13 October 2017 (UTC)

  • Francis Schonken, I'm a programmer merely reading the discussion, but I'll fill in what I can. As I understand it. If EnWiki used Wikidata as heavily as some other Wikis, Wikidata would have effectively taken down EnWiki (and possibly other wikis). If Wikidata use doubled every month, without a fix, I would make a wild-guesstimate that 4 months might dangerous to the entire Wiki and 6 months would almost certainly be overwhelming.
  • As I understand it, the current emergency fix is to totally shut off Wikidata notifications here. If an article is attacked via Wikidata, no one where can detect it unless they read the article just like the general public.
  • As I understand it, @Lydia Pintscher (WMDE): (project director of Wikidata) is advocating a temporary band-aid. Changes that affect one article or a small number of article would appear in your watchlist (assuming you turn watch-wikidata on). However changes that affect a large number of articles would only appear on the watchlist for a few random articles. If you watchlisted a page, there would be a very very low random chance you'd see it.
  • The long term answer is currently unknown. The technical details were hard to follow, but it sounded like they were sorting through options which were difficult and/or inadequate.
  • In my option "a little bit of Wikidata" doesn't make much sense. The only stable outcomes are to either fully convert Wikipedia to Wikidata, or to limit Wikidata to almost nothing. If we fully convert to Wikidata, the math on this issue appears to explode in an unmanageable fashion. Either wikidata edits are invisible to article watchers, or a revert war on a widely used Wikidata item turns into global spam and an inter-language trainwreck. Alsee (talk) 02:19, 12 October 2017 (UTC)
    • If that analysis is more or less correct (Lydia Pintscher (WMDE) could you please confirm or amend?) there's another way: keep Wikidata to various experiments on en.Wikipedia. A lot of experimentation is needed anyway to make the various templates work satisfactorily. Limit the implementation of each experimental template to say a dozen mainspace instances (and a few more out of mainspace). That way we'll be ready for a more general roll-out if and when the technical issues are sorted comprehensively at Foundation level. And we're not just twisting our thumbs, i.e. postponing the lion share of the Lua programming of templates and the like till after the technical breakthrough at Foundation level. --Francis Schonken (talk) 05:34, 12 October 2017 (UTC)
      • I don't agree with the analysis, no. We are making several short-term fixes (not showing changes for heavily used items on every affected article, turning off showing changes for Commons and Russian Wikipedia) as well as several long-term fixes (new Lua functions that are more specific in what part of a Wikidata item they use so we can better filter out irrelevant changes) that will have a big impact. Additionally others are fixing issues that affect the watchlist independent of Wikidata. And last but not least we are discussing how to improve the overall system. That discussion doesn't have a conclusion yet. But that does not preclude more usage on enwp. --Lydia Pintscher (WMDE) (talk) 10:38, 12 October 2017 (UTC)
        • "But that does not preclude more usage on enwp. " That's rather optimistic. We know that at the moment, using Wikidata intensively on another wiki breaks Wikipedia in general and that wiki in particular. But that does not preclude more usage? Then what does? Fram (talk) 11:59, 12 October 2017 (UTC)
        • Lydia Pintscher I find it amusing that you "don't agree" with my analysis, and you proceeded to repeat my key points. (1) In the short term turning off changes and (2) implementing your band-aid of mostly not showing changes of heavily used items. (3) You acknowledge the long term is unknown. As director of the Wikidata project you seem to have a very enthusiastic/optimistic outlook on the future, which appears to be based more on positive-thinking than any deep examination of the technical or social issues.
        • In the future please don't put the "political issues" of your pet-project against the safety of the encyclopedia as a whole.[3] Especially not when a database administrator declares an emergency that Wikidata was close to breaking all of Wikipedia.[4] In my opinion it makes you rather less reliable than a "sourced" Wikidata item containing a circular reference to an unsourced wikidata item. Alsee (talk) 14:36, 12 October 2017 (UTC)
          • We are not just implementing band-aids. Besides one (limiting the number of changes for highly-used items) the fixes we are making now are not band-aids but actual improvements. And if the database admin responsible for it (Jaime) says we should turn off showing changes on more wikis then I am happy to do so. He has not asked for it. And as I said before with the fixes we are making now it should not be necessary. --Lydia Pintscher (WMDE) (talk) 14:01, 13 October 2017 (UTC)

RfC moot?

Lydia Pintscher has made it quite clear that en.Wikipedia-Wikidata integration is safe as long as the growth path does not alter significantly from what we've been doing up to now. An exponential growth path has been qualified as unthinkable (passons regarding the WMF's vote of no confidence in what Wikipedia editors and Lua developers can do), and has to be excluded anyhow. What will show up in Wikipedia watchlists has been affected, so that Wikidata content watchlisted in Wikipedia will not necessarily show up comprehensively in such watchlist: what exactly shows up is no longer determined by Wikipedians, but is pruned at WMF/software level. Thus one of the tenets of en.Wikipedia-Wikidata integration is no longer a fully reliable component.

I propose to give it some thought whether we need the RfC now: an optimistic growth path that may come out as consensus of the RfC may not be technically viable, maybe not even in the short run. Alternatively, all options that could lead to an untenable growth path could be barred from the options that can be chosen in the RfC, ... which would leave us with a more or less empty box I suppose (there are few options left if we can only choose to continue what we are doing now or take a step back).

• Put RfC on hold until there is more clarity where the "Watchlist overload" issue is going. --Francis Schonken (talk) 06:48, 14 October 2017 (UTC)

I was expecting that we'd do this the other way around: that we can continue drafting the RfC here with the options for what we want to do with Wikidata in the future, performance issues notwithstanding, and then take a !vote on whether it's ready to go live/discuss the remaining prerequisites. Personally, I agree that there needs to be more clarity on the watchlist question first, though. We have the Wikidatacon at the end of the month, hopefully we'll know more by then / at that. Thanks. Mike Peel (talk) 22:33, 14 October 2017 (UTC)

Proper QA

We need to have improvement in the watchlist functioning of WD items before we should really be expanding its usage significantly. The phab ticket. Doc James (talk · contribs · email) 05:48, 10 October 2017 (UTC)

Is this accurate?

Something like the following should be in the introductory stuff... would you please review this for accuracy?

If there is data in an infobox from Wikidata, can it be edited?

• Depending on the coding used to build the infobox, there are several options:
  • In some versions of infoboxes, the data resides solely in Wikidata and is just displayed in the infobox in the article; the data itself is not in Wikipedia. If you look at wikitext in the edit window, you will see only a short template for the infobox like {{infobox_gene}}. The infobox will change whenever Wikidata is changed (it is rendered anew each time the Wikipedia page is displayed). These infoboxes have a link that sends you to Wikidata to change the data there. You cannot edit the data from within Wikipedia. The coding used for these templates is generally Lua and takes expertise to change. This coding can determine what fields to pull and how to label the fields, and can also evaluate the data and only pull data with certain qualities. For example, you can set the code so that only data that is referenced can be pulled, and unreferenced data will not be pulled.
  • In other versions, the infobox pulls data from Wikidata one time, and the data resides here in Wikipedia. If you look in the edit window you will see the full infobox template with all the fields. For these templates the data is fully editable here in Wikipedia.
  • Some infobox templates have hybrids of these two models.

-- Jytdog (talk) 12:24, 10 October 2017 (UTC)

@Jytdog: It seems a lot more accurate than other info being added to the draft, so I've added it to the page and will tweak it there. Thanks. Mike Peel (talk) 00:16, 11 October 2017 (UTC)

Thanks! I am interested to see what you tweak. :) Jytdog (talk) 01:12, 11 October 2017 (UTC)

Re. "...a lot more accurate than other info being added to the draft" – yes, the introduction section is getting unwieldy. It talks about possible solutions quite a lot: I'd throw that out of the introduction section, and translate such topics into sound questions in the "Options" area.

The more unwieldy the introduction gets, the less it will be read, and thus make the share of !votes starting from preconceived ideas bigger. --Francis Schonken (talk) 06:47, 11 October 2017 (UTC)

We don't really have an introduction yet; 'background information' is more of a 'frequently asked questions' section. Maybe we should move that below the options, or to the bottom of the page, or maybe another page completely. How about we keep working on it/playing around with the structure, and when we're closer to starting the RfC (which is at least a few weeks, maybe a month or more, away) we can see how things look. Thanks. Mike Peel (talk) 20:45, 11 October 2017 (UTC)

I don't know whether you saw it, but you're proposing exactly the opposite of what I intended. We don't want editors to !vote from prejudice: thus a compact & readable presentation of the background is needed before the !vote. --Francis Schonken (talk) 06:06, 12 October 2017 (UTC)

Enthusiasm vs caution

Hi Mike, about this... I think the question is a good one and i think the main answer should be "there is a dispute between X and Y". (as there generally is behind RfCs) So.. between what? I said "enthusiasm" vs. "caution" but "go fast" vs "go slow" or other ways would be fine.. but defining the dispute should be upfront i think.... Jytdog (talk) 01:20, 11 October 2017 (UTC)

@Jytdog: Far too often in Wikidata discussions do people try to use an "us vs them" argument to polarise the discussions. It's not black-and-white, and it's not one group of people vs another, there are a lot of intermediate views between the extremes. That's why I wrote this specific piece in a more neutral way, and why there are a range of different options rather than yes/no questions. Thanks. Mike Peel (talk) 20:41, 11 October 2017 (UTC)

The question needs to be answered. What is the dispute or set of disputes driving the RfC? If there were none, we would not be having this. The way the question is answered now, is teleological - we are on an inexorable process of using Wikidata more broadly, and here is where we are in that process. I reckon you see it that way but it begs the question.

This RfC cannot launch with a fake answer like this. Jytdog (talk) 21:08, 11 October 2017 (UTC)

As I wrote in the draft, the key thing that's changed from my perspective is that we are now in the implementation rather than concept stage. That has provoked a number of responses, but few are black-and-white (apart from those that say 'don't use wikidata at all'). If you can figure out another way of explaining the issue without oversimplifying it, please go ahead. Thanks. Mike Peel (talk) 21:14, 11 October 2017 (UTC)

Re. "...we are now in the implementation rather than concept stage..." – we're still in an experimental stage, not full roll-out. --Francis Schonken (talk) 06:08, 12 October 2017 (UTC)

Cite Q template

{{Cite Q}} has now been mentioned in the introduction. A recent lengthy TfD on the template resulted in "no consensus". Do we want a few RfC questions about this template in the "Options" area? That area is currently only about "... in infoboxes". Cite Q is used as well in as outside of infoboxes, so if we want questions about this template: only w.r.t. its use in infoboxes, or questions about some of the other unresolved issues w.r.t. this template too? On the other hand, if we don't want questions being asked about this template in this RfC I wouldn't mention it in the introduction. --Francis Schonken (talk) 06:31, 11 October 2017 (UTC)

TfD is here Jytdog (talk) 17:53, 11 October 2017 (UTC)

We already have too many options here and need to trim them down a bit, adding more about a separate template doesn't make sense to me. I only mentioned the template as an example, prefixed with "References from Wikidata can be shown here" (i.e., it's not the only way, it's just an example). Mike Peel (talk) 20:34, 11 October 2017 (UTC)

Then better not to mention {{Cite Q}} – it is a controversial template ("no consensus" per TfD). Or, if we want to cover "Wikidata&infoboxes" more or less comprehensively: ask this single RfC question: can Wikidata infoboxes make use of the {{Cite Q}} template? --Francis Schonken (talk) 06:11, 12 October 2017 (UTC)

Instability of Wikidata-based infoboxes

Screenshot from Arcadia (painting) from today

Considering Template talk:Infobox World Heritage Site, where the RfC seems to result in removing the Wikidata-version of the infobox at least in part due to instability (getting unexpected and unwanted results with e.g. links to disambiguation pages), then Template talk:Infobox person/Wikidata#Age issue from earlier this week, and now the image on the right from the full-Wikidata infobox version Template:Infobox artwork/wikidata, where the issues disappear when using the non-Wikidata version, I think the RfC should (or better, will) focus on these issues as well. If Wikidata infoboxes are, for whetever reason, more prone to unexpected problems and errors (separate from pure data problems and referencing and so on), then it is an important factor in the decision to reduce or increase the number of such infoboxes. Fram (talk) 08:22, 13 October 2017 (UTC)

That error was due to Module:Wd being broken for around six hours recently (see Module talk:Wd#Recent edits). Unfortunately the error caused errors in hundreds of articles, although it was only visible on articles which were purged during the time the module was broken. There is a sensational example (seven errors) at Bhadla Solar Park although that will disappear when someone purges it after seeing this. Strangely I do not see the module in "related changes" for Arcadia (painting) but it's pretty certain that the module was responsible. I'm not saying that should cheer you up, but the problem was not actually Wikidata. Johnuniq (talk) 08:56, 13 October 2017 (UTC)

Not Wikidata, but only Wikidata-based infoboxes. Thanks for the explanation.

Template usability

Many people here complain about usability of infoboxes using Wikidata, and some of them imply that they believe all information shown in the article should be contained in the code of the article. I invite those to inspect Myakinino (Moscow Metro). This article, which does not use Wikidata, contains a template just below the infobox, which says that the next station is Streshnevo (Moscow Central Circle). This is incorrect, the next station in that direction is Volokolamskaya (Moscow Metro). Streshnevo occurs exactly zero times in the code of the page. I am kind of experienced user, and I do not understand WTF is going on, and I do not know how to replace Streshnevo with Volokolamskaya. As far as I am concerned, this usability problem by far surpasses any possible Wikidata-related problems, and whoever wants to completely prohibit usage of Wikidata on the basis of usability must first make sure that Wikipedia own templates are usable.--Ymblanter (talk) 14:20, 13 October 2017 (UTC)

"Something here is broken, so you are not allowed to oppose something else that is broken". No thank you! That something here needs fixing (and a lot of things around here do!) is not a reason to "whoever wants to completely prohibit usage of Wikidata on the basis of usability must first make sure that Wikipedia own templates are usable." If someone was replacing specific Wikidata infoboxes with some unusbale, worse local ones, then you might have a point. As it stands though, your argument is utterly invalid. Fram (talk) 14:58, 13 October 2017 (UTC)

To be honest, I did not expect you to react differently.--Ymblanter (talk) 15:17, 13 October 2017 (UTC)

No, I don't like to give compliments for incorrect arguments and faulty logic. "X is bad so Y should not be criticized" is nonsense. "Don't replace poor Y with worse X" is a valid argument, but no one is proposing that. Fram (talk) 15:29, 13 October 2017 (UTC)

Oh, and the reason for that problem is that Template:MOSMETRO stations has an automatic "|Volokolamskaya=Streshnevo" switch (because Volokolamskaya (Moscow Ring Railway) has been renamed). I have correcetd the two articles were this was a problem (correcting them wasn't hard, finding the cause was rather complicated). Fram (talk) 15:29, 13 October 2017 (UTC)

Good, thanks.--Ymblanter (talk) 15:33, 13 October 2017 (UTC)

Looking at this another way, people wanted to have some sort of database system that could be used in Wikipedia long before Wikidata existed, and those are invariably complicated hacks of template code that only a few people understand. This is not the only case of this, there are others. If we can improve things by migrating them to Wikidata, with a clearer editing mechanism (compared with template code!) where we can also have the help of locals who can edit the information in Russian as needed, then that is a step forward. Thanks. Mike Peel (talk) 21:32, 13 October 2017 (UTC)

Time to start phase 1 of this RfC

In addition to all already listed problems and examples with infoboxes (e.g. at the World Heritage Site infobox RfC), the last few days have seen issues with the changes from Wikidata coming through with a delay of many hours (currently, according to this, it's 4 hours and 45 minutes), the general issue of Wikidata use in other wikis nearly crashing Wikipedia, and issues with blatant vandalism not getting intercepted for hours (if at all). This was this week e.g. the case with Romania which was renamed "Moldova" for a few hours (many similar examples available if people need them).

Yesterday, this item was vandalized for five hours before the change was reverted. This change affected about 1 in 5 items at Wikidata, as it is the name of a major, major reference for them. If vandalism to one in five items can remain undetected (or at least unreverted) for 5 hours, then any claims of sufficient vandal fighting and large groups of editors swiftly correcting things is totally moot.

So, to resume:

• Wikidata vandalism can affect countless Wikidata items and relatively many enwiki articles at once
• Even blatant Wikidata vandalism usually doesn't get reverted for hours
• Wikidata changes can't be checked through our Recent Changes (due to the severe delay)
• Wikidata changes are had to check through a watchlist (due to the delay, which means that the changes may well appear off-screen if you have a somewhat longer watchlist; and due to the rather opaque way these are shown, genre "Created claim: Property:P131: Q239")
• Wikidata changes don't appear in the page history
• Wikidata-based infoboxes have clear problems (see e.g. Template talk:Infobox World Heritage Site#RfC: revert back to non-Wikidata version?) which can be avoided in enwiki-based infoboxes
• Four years after the previous RfC, it has become clear that Wikidata isn't (yet?) ready to play the role of reliable datasource to enwiki infoboxes, for a number of reasons (too many to list here), and it is time to review and perhaps revert the decision of that RfC

Can we please now finalize and start phase 1 of the RfC? That would be the Wikipedia:Wikidata/2017 RfC draft#Mainspace use of Wikidata infoboxes question. Fram (talk) 14:13, 19 October 2017 (UTC)

Request for help posted at WP:ANI#Help wanted in finalizing Phase 1 of the new Wikidata infoboxes RfC. Fram (talk) 14:29, 19 October 2017 (UTC)

No, this is not ready, it should not be split up into separate phases, and it certainly shouldn't be started prematurely due to your trolling. My current estimate is that this will be ready to run early next year. Mike Peel (talk) 17:00, 19 October 2017 (UTC)

Er... I'm sympathetic to wanting to avoid rushing into things, but that seems a very long way off. Why so long? Nikkimaria (talk) 17:15, 19 October 2017 (UTC)

@Nikkimaria: Mostly because of the issue discussed at #Systemic issue that has serious implications for increased use of Wikidata above, which should settle down a bit before we run this. That made me think 'a month or two', which I rounded up to early next year. Thanks. Mike Peel (talk) 17:37, 19 October 2017 (UTC)

Bye, Mike Peel. Contact me when you are ready to be polite and avoid the personal attacks again, otherwise don't bother. Disagreeing with you and pointing out serious problems with something you like is not "trolling". If you don't want to collaborate in preparing the RfC now, fine, then it will happen without you. Fram (talk) 18:04, 19 October 2017 (UTC)

I'm always happy to collaborate and work through problems. I'm not willing to listen to rhetoric and hyperbole in place of a reasoned argument, and I'm not willing to waste time posting replies that aren't listen to - hence why I'm not bothering to reply to most of your posts now. Bye. Mike Peel (talk) 18:26, 19 October 2017 (UTC)

Sounds like "Great, today Wikidata has issues, let us keep RfC ASAP, otherwise the issues may go away and it would be difficult to convince Wikipedia users that Wikidata is junk". The RfC must be prepared and run when it has been properly prepared.--Ymblanter (talk) 17:33, 19 October 2017 (UTC)

"Today Wikidata has issues"? Funny. I started the WHS RfC before today, you know? I started the actual 2017 state of affairs page before today as well. Wikidata has issues today, had issues yesterday, had issues last year, and will have issues next year. But the more one looks into it, the more (and more severe) issues one encounters. Fram (talk) 18:04, 19 October 2017 (UTC)