GNOME Bugzilla – Bug 757171
Add KML/KMZ support
Last modified: 2018-03-26 13:02:21 UTC
KML, Keyhole Markup Language https://en.wikipedia.org/wiki/Keyhole_Markup_Language is "[...]an XML notation for expressing geographic annotation and visualization within Internet-based, two-dimensional maps] sort of like our GeoJSON support. It would be nice to support this in Maps. Maybe we could convert KML to GeoJSON and then use the GeoJSON support we have to render? Worth investigating. Or maybe there is a nice library out there to render KML? Mapbox has a JavaScript library for converting to GeoJSON: https://github.com/mapbox/togeojson
I don't think that converting KML to GeoJSON is a good idea. KML has a lot more functionality then geoJSON. Especially the KML files from DigiKam and darktable would be useless without the preview images. Information from the old bug report: Google provides a library to work with KML: https://github.com/google/libkml A developer documentation can be found here: https://developers.google.com/kml/documentation/?csw=1 KDE Marble and Google Earth are able to im- and export KML/KMZ files. DigiKam and darktable have KML/KMZ exports for images with GPS informations. KMZ is just a ZIP compressed KML file. ---- Mattias Bengtsson 2015-09-03 03:33:44 UTC There's some resources and code here[1] as well for converting KML, WKT, GeoCSV, GPX etc. Might be worth taking a look at. 1: https://github.com/mapbox/leaflet-omnivore ---- Andreas Nilsson 2015-09-07 10:32:26 UTC I have no prior knowledge of this, so sorry for asking stupid questions. So this is for displaying custom things that are not in OSM as a layer on top of our current layers? ---- TJ: I usually take photos with a GPS tracker. The coordinates are saved into the exif tags of the photos. Then I import this photos into darktable and export an KML-File with thumbnails of the photos. This file can now be opened in Google Earth or Marble. ---- Andreas Nilsson 2015-09-07 10:45:10 UTC Ah, thanks for the info. What kind of information is common for the formats to have? Does the file usually itself have a title, more than the filename? ie. "GUADEC 2016 map" vs. guadec2016.kml ---- TJ: Yes, KML-Files usually have a Title but the title tag is not necessary.
(In reply to tobias from comment #1) Thanks for commenting! > I don't think that converting KML to GeoJSON is a good idea. KML has a lot > more functionality then geoJSON. Especially the KML files from DigiKam and > darktable would be useless without the preview images. > Ok, depending on how the conversion is done this could be preserved in the geojson properties fields, maybe? > Information from the old bug report: > > Google provides a library to work with KML: > https://github.com/google/libkml > The problem with this is that it is a lot of work to support libraries that are not based on GObject. We would have to shim it in C and then use it in gjs. And maybe we will have a problem with large data sets, will we then have to write code to break this down in to tiles? That seems daunting. Since we stole that code for GeoJSON from clever people. It is of course doable to use libkml and might be the only way. But it makes the support landing in 3.20 less likely. Since our main problem is developer bandwidth. I have about ~2h a week to work on Maps. And there are not many others. But if we get someone from GSoC or outreachy that wants to give this a shot, things change. Thanks!
I've googled a little bit, and found libgeoformats: https://gitlab.com/libgeoformats/libgeoformats
Created attachment 321189 [details] [review] Alphabetize src.gresource.xml file list
Created attachment 321190 [details] [review] Layers: Add 'All Layer Files' to the FileFilter. This makes it quicker to find the file you need, and we'll soon have multiple supported file types.
Created attachment 321191 [details] [review] Layers: Load GeoJSON file outside of geoJSONSource This decouples the on-disk storage from the contents of a GeoJSON layer in geoJSONSource. This is beneficial if we ever want to load GeoJSON layers that aren't stored as on-disk geojson files.
Created attachment 321192 [details] [review] Layers: Add togeojson & xmldom packages We're going to use the mapbox togeojson package and the xmldom package to convert KML and GPX files to GeoJSON for rendering. xmldom adapted for gjs based on commit 29a83b315 https://github.com/jindw/xmldom togeojson adapted for gjs based on commit f2818295b https://github.com/mapbox/togeojson
Created attachment 321193 [details] [review] Layers: Fix warnings in togeojson Several functions did a bare return, when for strict correctness based on mozilla's js engine they should return none.
Created attachment 321194 [details] [review] Layers: Fix whitespace in xmldom Convert to unix line endings. Remove trailing whitespace. Convert tabs to spaces.
Created attachment 321195 [details] [review] Layers: Fix warnings in xmldom There were several errors from returning with "return" when to be pedantic, the functions should "return none". Also functions should only be declared at top level.
Created attachment 321196 [details] [review] Layers: Add backend for KML shape layer files This commit leverages the GeoJSONSource backend by using togeojson to convert the KML files to GeoJSON for rendering.
Created attachment 321197 [details] [review] Layers: Add backend for GPX shape layer files This commit leverages the GeoJSONSource backend by using togeojson to convert the GPX files to GeoJSON for rendering. To get the mime type support working I needed to add: <mime-type type="application/gpx+xml"> <comment>GPS Exchange Format</comment> <sub-class-of type="application/xml"/> <glob pattern="*.gpx"/> </mime-type> to /usr/share/mime/packages/freedesktop.org.xml then run update-mime-database /usr/share/mime.
Review of attachment 321189 [details] [review]: Thank you!
Review of attachment 321190 [details] [review]: Seems right! ::: src/layersPopover.js @@ +68,3 @@ + [filter, allFilter].forEach(function(f) { + layerClass.mimeTypes.forEach(f.add_mime_type.bind(f)); + }); This would be two lines without the fancy loop :) But I guess it makes it clear what is going on.
Review of attachment 321191 [details] [review]: Thanks! I like the idea of this! ::: src/geoJSONShapeLayer.js @@ +48,3 @@ + let [status, buffer] = this.file.load_contents(null); + if (!status) + throw new Error(_("Failed to load file")); This changes the string from all lower case, does that trigger the need of new translations all around? Dould we avoid that and have it lower-case as it was? @@ +55,1 @@ this._mapSource.parse(); Maybe send the json in here instead? ::: src/geoJSONSource.js @@ +62,3 @@ + set json(json) { + this._json = json; + }, Until json is used in more places than parse, maybe we could keep it just there, as an argument?
Review of attachment 321192 [details] [review]: Thanks for providing the sha1 for the imports! Makes it easier to re-import updates later! Have you made any changes to the code in order to make it work? I would prefer one commit that import and then another with changes so it is apparent what is original and what we added.
Review of attachment 321193 [details] [review]: What warnings are this? I really want to avoid doing changes like this in imported code. I like to treat them as blackbox unless we really need to change. Will you also send these changes upstream? So we get them when we re-import. I want to avoid tricky merging.
Review of attachment 321194 [details] [review]: Thanks If this is only cosmetic, I think not. I want to treat this as a library and re-import when we need changes. Unless it is code that is needed to actually use it.
Review of attachment 321195 [details] [review]: Thanks! Same questions as the other warnings!
Review of attachment 321193 [details] [review]: These messages are printed as GJS loads/scans the js files (i.e. before their code is run). I didn't see a way to silence the errors. It seems related to mozilla's javascript.options.strict. Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/togeojson/togeojson.js 307]: function getTrack does not always return a value Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/togeojson/togeojson.js 327]: function getRoute does not always return a value Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/dom.js 24]: in strict mode code, functions may be declared only at top level or immediately within another function Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/dom.js 150]: function _findNodeIndex does not always return a value Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/dom.js 194]: anonymous function does not always return a value Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/dom.js 429]: test for equality (==) mistyped as assignment (=)? Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/dom.js 432]: test for equality (==) mistyped as assignment (=)? Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/dom.js 425]: function _visitNode does not always return a value Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/dom.js 601]: anonymous function does not always return a value Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/dom.js 961]: function serializeToString does not always return a value Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/dom.js 963]: function serializeToString does not always return a value Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/dom.js 965]: function serializeToString does not always return a value Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/dom.js 967]: function serializeToString does not always return a value Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/dom.js 987]: function serializeToString does not always return a value Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/dom.js 989]: function serializeToString does not always return a value Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/dom.js 991]: function serializeToString does not always return a value Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/dom.js 919]: function serializeToString does not always return a value Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/dom.js 1119]: in strict mode code, functions may be declared only at top level or immediately within another function Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/domparser.js 96]: variable namespaceURI redeclares argument Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/domparser.js 98]: variable qName redeclares argument Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/domparser.js 143]: test for equality (==) mistyped as assignment (=)? Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/domparser.js 187]: function _locator does not always return a value Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/sax.js 577]: test for equality (==) mistyped as assignment (=)? Gjs-Message: JS WARNING: [resource:///org/gnome/Maps/js/xmldom/sax.js 571]: function split does not always return a value
Review of attachment 321196 [details] [review]: Love how simple this became! :) ::: src/kmlShapeLayer.js @@ +47,3 @@ + this._mapSource.json = json; + + this._mapSource.parse(); as before, I prefer sending json to parse!
Review of attachment 321194 [details] [review]: OK.
Review of attachment 321197 [details] [review]: Thanks! Looks nice!
Review of attachment 321191 [details] [review]: Thanks! Fixed in the next version... ::: src/geoJSONShapeLayer.js @@ +48,3 @@ + let [status, buffer] = this.file.load_contents(null); + if (!status) + throw new Error(_("Failed to load file")); hm... I'll change it to lowercase for now and read about pofiles and gettext when I get a little itme. @@ +55,1 @@ this._mapSource.parse(); ok ::: src/geoJSONSource.js @@ +62,3 @@ + set json(json) { + this._json = json; + }, OK
Review of attachment 321192 [details] [review]: Adding a new commit with unmodified files.
Review of attachment 321196 [details] [review]: OK. I also made the "Failed to load file" have a lowercase 'f'.
Created attachment 321203 [details] [review] Layers: Add togeojson & xmldom packages We're going to use the mapbox togeojson package and the xmldom package to convert KML and GPX files to GeoJSON for rendering. xmldom from commit 29a83b315 https://github.com/jindw/xmldom togeojson from commit f2818295b https://github.com/mapbox/togeojson
Created attachment 321204 [details] [review] Layers: Gjs-ify togeojson & xmldom packages These packages are made with nodejs in mind, so we have to tweak a few things to get them to laod in gjs.
Created attachment 321205 [details] [review] Layers: Fix warnings in togeojson Several functions did a bare return, when for strict correctness based on mozilla's js engine they should return none.
Created attachment 321206 [details] [review] Layers: Fix warnings in xmldom There were several errors from returning with "return" when to be pedantic, the functions should "return none". Also functions should only be declared at top level.
Created attachment 321207 [details] [review] Layers: Add backend for KML shape layer files This commit leverages the GeoJSONSource backend by using togeojson to convert the KML files to GeoJSON for rendering.
Created attachment 321208 [details] [review] Layers: Add backend for GPX shape layer files This commit leverages the GeoJSONSource backend by using togeojson to convert the GPX files to GeoJSON for rendering. To get the mime type support working I needed to add: <mime-type type="application/gpx+xml"> <comment>GPS Exchange Format</comment> <sub-class-of type="application/xml"/> <glob pattern="*.gpx"/> </mime-type> to /usr/share/mime/packages/freedesktop.org.xml then run update-mime-database /usr/share/mime.
Attachment 321189 [details] pushed as 12a5f1c - Alphabetize src.gresource.xml file list Attachment 321190 [details] pushed as 7d3d5d0 - Layers: Add 'All Layer Files' to the FileFilter.
Created attachment 321209 [details] [review] Layers: Load GeoJSON file outside of geoJSONSource This decouples the on-disk storage from the contents of a GeoJSON layer in geoJSONSource. This is beneficial if we ever want to load GeoJSON layers that aren't stored as on-disk geojson files.
Review of attachment 321208 [details] [review]: Looks good! ::: src/gpxShapeLayer.js @@ +41,3 @@ + let [status, buffer] = this.file.load_contents(null); + if (!status) + throw new Error(_("failed to load file")); So this is looking a bit similar for all shapeLayers? Maybe move to parent load? And that can set something like this.data? Then we can have the check for "large files" in one place for instance?
Review of attachment 321209 [details] [review]: Looks nice! Same question in this about moving reading content to super class shapeLayer.js
Created attachment 321210 [details] [review] Layers: Load GeoJSON file outside of geoJSONSource This decouples the on-disk storage from the contents of a GeoJSON layer in geoJSONSource. This is beneficial if we ever want to load GeoJSON layers that aren't stored as on-disk geojson files.
Created attachment 321211 [details] [review] Layers: Add togeojson & xmldom packages We're going to use the mapbox togeojson package and the xmldom package to convert KML and GPX files to GeoJSON for rendering. xmldom from commit 29a83b315 https://github.com/jindw/xmldom togeojson from commit f2818295b https://github.com/mapbox/togeojson
Created attachment 321212 [details] [review] Layers: Gjs-ify togeojson & xmldom packages These packages are made with nodejs in mind, so we have to tweak a few things to get them to laod in gjs.
Created attachment 321213 [details] [review] Layers: Fix warnings in togeojson Several functions did a bare return, when for strict correctness based on mozilla's js engine they should return none.
Created attachment 321214 [details] [review] Layers: Fix warnings in xmldom There were several errors from returning with "return" when to be pedantic, the functions should "return none". Also functions should only be declared at top level.
Created attachment 321215 [details] [review] Layers: Add backend for KML shape layer files This commit leverages the GeoJSONSource backend by using togeojson to convert the KML files to GeoJSON for rendering.
Created attachment 321216 [details] [review] Layers: Add backend for GPX shape layer files This commit leverages the GeoJSONSource backend by using togeojson to convert the GPX files to GeoJSON for rendering. To get the mime type support working I needed to add: <mime-type type="application/gpx+xml"> <comment>GPS Exchange Format</comment> <sub-class-of type="application/xml"/> <glob pattern="*.gpx"/> </mime-type> to /usr/share/mime/packages/freedesktop.org.xml then run update-mime-database /usr/share/mime.
Review of attachment 321209 [details] [review]: I put more stuff in the super class.
Review of attachment 321210 [details] [review]: ::: src/shapeLayer.js @@ +100,3 @@ + _setUpMapSource: function() { + /* Unimplemented */ + }, Why do it this way and not keep load? This way seems more magic? having a load that does this.parent() first to populate a this._fileContents? That you can use after this.parent() ? If you insist on this way: setupMapSource seems prettier?
Review of attachment 321210 [details] [review]: reviewed I mean
Created attachment 321217 [details] kml sample file
Created attachment 321218 [details] gpx sample
Created attachment 321220 [details] [review] Layers: Load GeoJSON file outside of geoJSONSource This decouples the on-disk storage from the contents of a GeoJSON layer in geoJSONSource. This is beneficial if we ever want to load GeoJSON layers that aren't stored as on-disk geojson files.
Created attachment 321221 [details] [review] Layers: Add togeojson & xmldom packages We're going to use the mapbox togeojson package and the xmldom package to convert KML and GPX files to GeoJSON for rendering. xmldom from commit 29a83b315 https://github.com/jindw/xmldom togeojson from commit f2818295b https://github.com/mapbox/togeojson
Created attachment 321222 [details] [review] Layers: Gjs-ify togeojson & xmldom packages These packages are made with nodejs in mind, so we have to tweak a few things to get them to laod in gjs.
Created attachment 321223 [details] [review] Layers: Fix warnings in togeojson Several functions did a bare return, when for strict correctness based on mozilla's js engine they should return none.
Created attachment 321224 [details] [review] Layers: Fix warnings in xmldom There were several errors from returning with "return" when to be pedantic, the functions should "return none". Also functions should only be declared at top level.
Created attachment 321225 [details] [review] Layers: Add backend for KML shape layer files This commit leverages the GeoJSONSource backend by using togeojson to convert the KML files to GeoJSON for rendering.
Created attachment 321226 [details] [review] Layers: Add backend for GPX shape layer files This commit leverages the GeoJSONSource backend by using togeojson to convert the GPX files to GeoJSON for rendering. To get the mime type support working I needed to add: <mime-type type="application/gpx+xml"> <comment>GPS Exchange Format</comment> <sub-class-of type="application/xml"/> <glob pattern="*.gpx"/> </mime-type> to /usr/share/mime/packages/freedesktop.org.xml then run update-mime-database /usr/share/mime.
Review of attachment 321220 [details] [review]: Thanks!
Review of attachment 321221 [details] [review]: \o/
Review of attachment 321222 [details] [review]: Thank you Hashem!
Review of attachment 321223 [details] [review]: Thanks!
Review of attachment 321224 [details] [review]: \o/
Review of attachment 321225 [details] [review]: lgtm!
Review of attachment 321226 [details] [review]: great!
Oops prematurely closed. KMZ support is still in the works.
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/gnome-maps/issues/30.