Magnus Manske

Previous discussion was archived at User talk:Magnus Manske/Archive 9 on 2024-01-01.

Could you please scrape the dates?

We also need dates on https://mix-n-match.toolforge.org/#/catalog/6530 and https://mix-n-match.toolforge.org/#/catalog/6531. Thanks!

6530 and 6531 have dates where available, I believe. Can you find me an example of 6530 that has dates in the source but not on MnM?

I have no examples, in fact; I see dates everywhere in the entries. But if dates are everywhere in the entries, I don't understand why in the "Jobs" I don't see the options to match on name-date, on birthdate and on deathdate, that should appear since the dates are present in the entries.

3534 done

6530 and 6531 had their dates imported rather than scraped from the description, so the "has_person_date" flag was not set for the catalog. It is set now, and the date jobs should be available.

Thanks! I was thinking if the possibility of setting the flag "has_person_date" could be made available in catalog_editor (e.g. https://mix-n-match.toolforge.org/#/catalog_editor/6530) so that MnM admins can set this flag in case of dates imported without disturbing you.

Another small request: I'm often using the https://mix-n-match.toolforge.org/#/issues/ to find wrong matches in the catalogs, or duplicates, and it is very useful; it is unclear to me when and how MnM analyses a catalog to find potential issues, and if it does it only one for each catalog (when it is imported) or it does it for each catalog periodically; would it be possible to add a job for each catalog that allows to trigger a new check of potential issues? I think it would be very useful!

Could you extract the dates from the description? Thanks

Done.

Hi Magnus, could you take a look at this and see if there is an easy fix for Cradle? Thanks!

https://github.com/magnusmanske/cradle/issues/15

Could you please asign property to catalog?

Thank you

Are you sure about this? The BioMed Central journal ID (P13124) identifiers appear to be alphanumeric, wheres the catalog IDs are numeric.

Good point. We should change the catalog or the property.

I made a new catalog with a new scrape, linked it to the property, synced from Wikidata, and set the remaining ones from the old catalog where available, preserving user and timestamp.

Not sure why it's only 300 and not 328 though.

this has only 300 entries

Thank you. Something changed since 2018...

For example: World Allergy Organization Journal is no longer published by BMC. The journal is continuing in cooperation with a new publisher, Elsevier.

Hi, could you please remove autoscrape settings from these catalogues? I've been monitoring them for a long time and they stopped having autoscrape working a long time ago, but people are constantly restarting autoscrape job which hangs up in the queue for quite a long without any results.

• https://mix-n-match.toolforge.org/#/catalog/1 odnb (at one point it gave wrong id's which had to be manually marked as not applicable, but now it has stopped creating anything, although new id's appear monthly on the site)
• https://mix-n-match.toolforge.org/#/catalog/1011 WNS number (stucks in the queue for a few days and gives nothing for a couple of years)
• https://mix-n-match.toolforge.org/#/catalog/1486 mb artists (musicbrainz blocks request with non-browser UserAgent - so mix-n-match now always sees white page at every request)
• https://mix-n-match.toolforge.org/#/catalog/3476 mb genre (same as above)
• https://mix-n-match.toolforge.org/#/catalog/2043 letterboxd (gives no results for quite some time, although new ids appear regularly)
• https://mix-n-match.toolforge.org/#/catalog/3658 pitchfork (the scrapped page has been heavily redesigned and it no longer working)

There is also still the problem I mentioned in this post. I've launched through api autoscrape start for a set of catalogues. And the ones where autoscrape doesn't exist hang in the queue with a constant restarts via schedule. The catalogues are:

• https://mix-n-match.toolforge.org/#/jobs/5923 Deezer music genre
• https://mix-n-match.toolforge.org/#/jobs/5161 Napster music genre [archived]
• https://mix-n-match.toolforge.org/#/jobs/4905 MasterClass music genre guide
• https://mix-n-match.toolforge.org/#/jobs/4939 Free Music Archive music genre
• https://mix-n-match.toolforge.org/#/jobs/5000 PromoDJ music genre

This probably needs handling with a response and stopping the task if autoscrape is requested when it can't be executed.

(I also tried running a "pause" task in https://mix-n-match.toolforge.org/#/jobs/5923 , which is also saved in the tasks. This probably needs handling too to avoid potential vandalism)

Done, and thanks for the list. I made a new job status "BLOCKED" that can not be started from the web interface.

I also filter for existing job action types now

Thanks for the quick resolution. Btw I also saw PAUSE status in the queue a few days ago. I think it would be useful to have it in the web interface (or ABORT button) to stop at least your own tasks to avoid misclicks or unnecessary schedules as in the case of music genres above.

Hi. Could you please also block autoscrape on these catalogs. Most of them have either changed their layout and no longer work, or are stuck permanently with no new results for over a year.

• https://mix-n-match.toolforge.org/#/jobs/1
• https://mix-n-match.toolforge.org/#/jobs/1011
• https://mix-n-match.toolforge.org/#/jobs/1486
• https://mix-n-match.toolforge.org/#/jobs/2043
• https://mix-n-match.toolforge.org/#/jobs/3658

UPD: Oops, it appears I've reported them all above. They've probably been unblocked and are back in the queue?

And by the way, is it possible to remove the "automatch by search" task from the regular repetition here https://mix-n-match.toolforge.org/#/catalog/3789 and https://mix-n-match.toolforge.org/#/jobs/5195 ? I hit "purge automatches" every quarter, as without types syncing is more harmful due to clogging than helpful. It would probably be useful to have a button to remove regular tasks from the schedule in such cases (at least for those who have admin rights).

Hi, I can't see any problem with the https://mix-n-match.toolforge.org/#/catalog/1011 WNS number . The URL and IDs seems to be the same. The web site is updated periodically, last time 9 October 2024. The web has now 116 680 postage stamps registered but in the catalog there are only 90 241. What can be the problem? Thank you.

Hi, Gerwoman. Autoscrape doesn't work in this catalogue. Given that the catalogue is relatively small, autoscrapping should take no more than half an hour, by my experience. But in this catalogue autoscrape regularly falls into the queue and hangs there for several days/weeks without adding anything new for years. This can be verified in this way: The latest mix-n-match ID in this catalogue is https://mix-n-match.toolforge.org/#/entry/91669282. The latest mix-n-match IDs in recent catalogues have 172699175 ID. # 91m vs # 172m means that none of the autoscrape jobs for 2-3 years has not added a single new ID in this catalog. Thus, it does not work and constantly hangs in the queue.

If you know that IDs can still be autocollected from the site, you just need to reconfigure autoscrape from scratch by specifying 1011 in Catalog ID here https://mix-n-match.toolforge.org/#/scraper/new and configuring everything as you did the first time. If you can't reconfigure it again, it's better to just disable the autoscraper in it.

Thank you Solidest, but I can't remember how configured it the first time. I don't have access to the URL or regex...

I've reconfigured the autoscrape. Let's see.

Yeah, it worked, the catalogue entries are now at 116k. Thanks! I've crossed that catalogue off the list.

1. https://www.wikidata.org/w/index.php?title=Q130717777&action=history
2. https://www.wikidata.org/w/index.php?title=Q130723639&action=history

https://www.wikidata.org/w/index.php?title=Q55862034&action=history

The users that made the duplicate visible via DDB/GND IDs:

1. 19:48, 18 October 2024 User:KababyZMinsem 22,455 bytes +352 ‎Created claim: DDB person (GND) ID (P13049): 104207116, batch #238665 undothank Tag: quickstatements [2.0] (restore)
2. 08:34, 23 October 2024 User:Lorenz Karsten 22,803 bytes +348 ‎Created claim: GND ID (P227): 104207116, batch #238900 undothank Tag: quickstatements [2.0] (restore)

have been blocked.

https://www.wikidata.org/w/index.php?title=Q113446597&action=history

Magister should probably not be in the name field, cf. GND - not doing it that way

https://www.wikidata.org/w/index.php?title=Q108811951&action=history

The users that made the duplicate visible via DDB/GND IDs:

1. 16:05, 27 October 2024 User:KababyZMinsem 6,952 bytes +355 ‎Created claim: DDB person (GND) ID (P13049): 1031582746, batch #239034 undothank Tag: quickstatements [2.0] (restore)
2. 01:42, 29 October 2024 User:Daubpushyd 7,303 bytes +351 ‎Created claim: GND ID (P227): 1031582746, #quickstatements; #temporary_batch_1730166071073 undothank Tag: quickstatements [2.0] (restore)

have been blocked.

https://www.wikidata.org/w/index.php?title=Q119999189&action=history

The users that made the duplicate visible via DDB/GND IDs:

1. 18:48, 23 October 2024 User:KababyZMinsem 5,868 bytes +355 ‎Created claim: DDB person (GND) ID (P13049): 1012276988, batch #238911 undothank Tag: quickstatements [2.0] (restore)
2. 06:29, 24 October 2024 User:Lorenz Karsten 6,219 bytes +351 ‎Created claim: GND ID (P227): 1012276988, batch #238929 undothank Tag: quickstatements [2.0] (restore)

have been blocked.