https://maps.wikimedia.org/geoline?getgeojson=1&ids=Q79
P17431

This shows that the geoline service returns these points. It seems to be the location of the label, and the location of the capital of the country/region...
If this is new then it's definitely a Karthoterian issue, not the Kartographer side.

K. it might be new... but i do note that OSM itself also shows this geometry with two points. So the data is probably correct, but it might be that we filtered this out in the past etc etc. Hard to say.

I found the bug described for geoline objects, too. See for instance: https://de.wikivoyage.org/wiki/Vorlage:Mapshapes . The map shows several subway lines of Vienna whereas the stations ar marked with a pushpin marker.

If you call https://maps.wikimedia.org/geoline?getgeojson=1&ids=Q873397 to get the geoline object of U2 subway you will detect a lot of point objects which should not be part of a line. You can see from the GeoJSON array returned:

I think the problem can be solved by removing the point features and keeping the Linestring or similar objects.

By the way, the pushpin markers are not only needless and cover the geoshape of interest. The authors have no mean to modify the pushpin (colors, sizes etc).

I agree that it is suboptimal. On the other hand, it is data which is in the OSM database, I don't think it is unreasonable to have the geoshape service return it too.. Maybe we should filter them out when rendering ?

It's hard to decide without more people weighing in :(

Those pushpins should definitely be filtered out. I haven't seen a single dynamic map where those pins provide any added value. They only clutter the map.

This was discussed on Wikivoyage as well: https://en.wikivoyage.org/wiki/Template_talk:Mapshapes#Grey_dots_are_now_appearing_on_all_maps_with_railway_lines
This is, in my opinion, the most pressing problem related to dynamic maps in all Wikimedia projects.
Cannot be more than a few lines of code to fix it?

In T292613#7591429, @Renek78 wrote:

Cannot be more than a few lines of code to fix it?

Cool, contributions are welcome

Good idea. Somehow Kartographer doesn't show up in the list of software projects though (https://www.mediawiki.org/wiki/New_Developers#Choose_a_software_project)

Hi, see https://phabricator.wikimedia.org/project/profile/1829/ for the Source Repo (as it's not a project recommended for completely new developers). :) Thanks!

In T292613#7591785, @Aklapper wrote:

Hi, see https://phabricator.wikimedia.org/project/profile/1829/ for the Source Repo (as it's not a project recommended for completely new developers). :) Thanks!

Aklapper, thanks for the link. I found some set-up instructions. Is there a place where I can ask questions if I get stuck?

@Renek78: I don't know a great place, probably https://lists.wikimedia.org/hyperkitty/list/[email protected]/ or other places listed on https://www.mediawiki.org/wiki/Communication

In T292613#7540300, @TheDJ wrote:

I agree that it is suboptimal. On the other hand, it is data which is in the OSM database, I don't think it is unreasonable to have the geoshape service return it too.. Maybe we should filter them out when rendering ?

It's hard to decide without more people weighing in :(

It's not just suboptimal, but it is leading to nearly unusable maps: https://de.wikivoyage.org/wiki/Emmerich_am_Rhein#Mobilit%C3%A4t

I hope the priority of this bug will be raised.

Maybe it makes sense to filter these markers while rendering. But the communities have no change to do it by themselves. At least, there should be a hook to add this filter to MediaWiki:Kartographer.js.

Meanwhile, I added a CSS rule to MediaWiki:Kartographer.css to suppress the grey markers. This rule can alternately be added to MediaWiki:Common.css/Mediawiki:Mobile.css.

/* Removing grey pushpin markers on mapframe maps */
img[src$="[email protected]"] {
	display: none;
}

In T292613#7406972, @TheDJ wrote:

... it might be that we filtered this out in the past etc etc. Hard to say.

I'm fairly sure points were filtered out. This is affected by patches in T288400. Previously geolines were queried from planet_osm_line table which has linestring table type set in imposm_mapping.yml. Now geolines come from wikidata_relation_members table of relation_member table type (not limted to linestrings). @MSantos: maybe it wouldn't be too difficult to adjust the mapping so that this new table would also include only linestrings?

In T292613#7911183, @Pikne wrote:

Now geolines come from wikidata_relation_members table of relation_member table type (not limted to linestrings). @MSantos: maybe it wouldn't be too difficult to adjust the mapping so that this new table would also include only linestrings?

This looks like a promising approach! The OSM relationship corresponding to Q873397 above is https://www.openstreetmap.org/relation/27114 , which is composed of node and way members.

We can probably set up a second level of filtering in kartotherian, as an immediate workaround. For example, compare the results of:
select ST_AsGeoJson(geometry) from wikidata_relation_members where wikidata='Q168162' and GeometryType(geometry) != 'POINT'
select ST_AsGeoJson(geometry) from wikidata_relation_members where wikidata='Q168162' and GeometryType(geometry) == 'POINT'

Change 790260 had a related patch set uploaded (by Awight; author: Awight):

[mediawiki/services/kartotherian@master] [WIP] Filter out points from geoshape and geoline query

https://gerrit.wikimedia.org/r/790260

Part of this ticket should also be creating follow up tickets for a long term fix to filter points in the imposm import.

I tried to dig a bit into the code that is ultimately responsible for these points.

• https://gerrit.wikimedia.org/g/operations/puppet/+/production/modules/osm/files/imposm_mapping.yml appears to be the configuration we are looking for. Specifically the wikidata_relation_members part.
• relation_member here means this is going to import all members of an OSM relation that have a wikidata item id assigned. This can include points.
• According to the documentation at https://imposm.org/docs/imposm3/latest/relations.html it should be possible to add something like filters: ↲ reject: ↲ type: point at the end of the wikidata_relation_members section to exclude points.
• It might be necessary to also add - name: type ↲ type: member_type to the list of columns: to make this work. (This is really confusing because type means two different things here.)
• However. The name of the table wikidata_relation_members sounds like it's used for other things as well, not only for the geoline feature we are talking about here. To not accidentally destroy other features that want these points we would need to stick to https://gerrit.wikimedia.org/r/790260 then. Which might be ok.

Patch for review: https://gitlab.com/wmde/search-mapframe-insource/-/merge_requests/2

I looked for any pages where geoline is used in combination with any of the marker- styles, which would indicate that users were relying on the points intentionally.

./search_insource 'insource:geoline insource:/geoline.+marker-/'

There were a few (40 or so) hits, and they all seem to be false positives, just a geoline coexisting on the page with other points.

Just for fun, I've demonstrated that the usage is possible but seems to not be done anywhere (and is undocumented). Given all these facts, it makes sense to disable support now and possibly have a later conversation about whether it's desirable to reintroduce a feature like this on purpose.

For reference, this is how the demo looks like:

What's special about this is that the (accidental, not officially supported) markers are styled in red, and with an icon. Source code:

<mapframe width="full" height="500">
{
    "type": "ExternalData",
    "service": "geoline",
    "ids": "Q873397",
    "properties": {
        "marker-symbol": "rail-underground",
        "marker-color": "#f00"
    }
}
</mapframe>

In case we ever want to officially support something like this, it should be a separate feature, separated from geoline.

Change 790260 merged by jenkins-bot:

[mediawiki/services/kartotherian@master] Filter out points from geoshape and geoline query

https://gerrit.wikimedia.org/r/790260

Still needs to be deployed to the beta cluster. Locally demonstrated.

Here's a demonstration that we've fixed the bug with the latest code:
https://de.wikipedia.org/wiki/Benutzer:Adamw/Spielwiese/T292613 (note that the snapshot is broken, and the interactive map shows a marker near the center)
vs.
https://en.wikipedia.beta.wmflabs.org/wiki/User:Adamw/sandbox/T292613 (just a line, in snapshot and interactive maps)

Another example that should be fixed with this solution:
map of Central Tokyo

Several issues that are probably related to this bug are collected in this task: T310289

In T292613#7994180, @awight wrote:

Here's a demonstration that we've fixed the bug with the latest code:
https://de.wikipedia.org/wiki/Benutzer:Adamw/Spielwiese/T292613 (note that the snapshot is broken, and the interactive map shows a marker near the center)
vs.
https://en.wikipedia.beta.wmflabs.org/wiki/User:Adamw/sandbox/T292613 (just a line, in snapshot and interactive maps)

Great work, Adam! The Tokyo article on Wikivoyage seems to be fine now (no more grey markers). If you go to the article MRT Circle Line, click on edit, uncomment the public transport lines for the dynamic map and preview it then all the grey markers are still there. Maybe the update needs some time to be applied to all Wikimedia sites?

In T292613#8077306, @Renek78 wrote:

The Tokyo article on Wikivoyage seems to be fine now (no more grey markers).

This is due to local CSS hack (see T292613#7911179), added to English Wikivoyage a few days ago.

Awight's patch for some reason hasn't been deployed in production yet, as far as I can see.

FYI: The fix in Maps (Kartotherian) should be deployed and working. The CSS workarounds should not be necessary anymore.