After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 791601 - new element: gpsdsrc
new element: gpsdsrc
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
git master
Other Linux
: Normal enhancement
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2017-12-14 00:01 UTC by Martin Kelly
Modified: 2018-11-03 14:16 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
0001-new-element-gpsdsrc.patch (17.96 KB, patch)
2017-12-14 00:01 UTC, Martin Kelly
none Details | Review
v2-0001-new-element-gpsdsrc.patch (18.54 KB, patch)
2017-12-14 20:41 UTC, Martin Kelly
none Details | Review

Description Martin Kelly 2017-12-14 00:01:15 UTC
Created attachment 365510 [details] [review]
0001-new-element-gpsdsrc.patch

new element: gpsdsrc
   
This element reads data from gpsd (http://www.catb.org/gpsd/). Right now
the GPS data type is not used in other plugins, but eventually it could
be used to multiplex and integrate with other A/V data types.
Comment 1 Nicolas Dufresne (ndufresne) 2017-12-14 00:31:24 UTC
I must admit that streaming a c structure in native endian is far from ideal. Isn't the any standard binary format, or could we at least make this endian safe ?

p.s. you have neglected to use GST_ELEMENT_ERROR in your code.
Comment 2 Martin Kelly 2017-12-14 00:46:11 UTC
As far as I can tell, there is not a standard binary format for the GPS fix outputs themselves. For streaming within a pipeline, I think this format is fine. However, once you persist it, it would of course need a standard format. I have not written an element to do persist it yet, but I plan to.

I will issue a followup patch to change my GST_ERROR calls to GST_ELEMENT_ERROR  calls.
Comment 3 Tim-Philipp Müller 2017-12-14 01:17:20 UTC
What is the actual / eventual purpose of this element, if I may ask?

It sounds rather niche to me.

What is the data format being output?

What would seem more generally useful to me is for example an element that puts current GPS co-ordinates as metadata on buffers, for example.

OOC, what's the dbus service used by e.g. GNOME to get current geo co-ordinates?
Comment 4 Nicolas Dufresne (ndufresne) 2017-12-14 01:44:57 UTC
(In reply to Tim-Philipp Müller from comment #3)
> What is the actual / eventual purpose of this element, if I may ask?

Real-Time stream geolocalisation seems very useful to me. Combine this with color video, depth stream and gyroscopic data, you could turn a video recording into 3D.

> 
> It sounds rather niche to me.
> 
> What is the data format being output?

Currently it is gps_fix_t from gpsd library.

> 
> What would seem more generally useful to me is for example an element that
> puts current GPS co-ordinates as metadata on buffers, for example.

Though that needs to happen in two steps, since you need to first synch the GPS fixes with the data because of the delays. You cannot request a fix in the past, you need to capture, synch and then you can merge or mux.

> 
> OOC, what's the dbus service used by e.g. GNOME to get current geo
> co-ordinates?

Gypsy, but it's a totally different multiplexer. It does not matter really, one can add a source for that.

The format:

typedef double timestamp_t;	/* Unix time in seconds with fractional part */

struct gps_fix_t {
    timestamp_t time;	/* Time of update */
    int    mode;	/* Mode of fix */
#define MODE_NOT_SEEN	0	/* mode update not seen yet */
#define MODE_NO_FIX	1	/* none */
#define MODE_2D  	2	/* good for latitude/longitude */
#define MODE_3D  	3	/* good for altitude/climb too */
    double ept;		/* Expected time uncertainty */
    double latitude;	/* Latitude in degrees (valid if mode >= 2) */
    double epy;  	/* Latitude position uncertainty, meters */
    double longitude;	/* Longitude in degrees (valid if mode >= 2) */
    double epx;  	/* Longitude position uncertainty, meters */
    double altitude;	/* Altitude in meters (valid if mode == 3) */
    double epv;  	/* Vertical position uncertainty, meters */
    double track;	/* Course made good (relative to true north) */
    double epd;		/* Track uncertainty, degrees */
    double speed;	/* Speed over ground, meters/sec */
    double eps;		/* Speed uncertainty, meters/sec */
    double climb;       /* Vertical speed, meters/sec */
    double epc;		/* Vertical speed uncertainty */
};
Comment 5 Martin Kelly 2017-12-14 20:39:15 UTC
(In reply to Nicolas Dufresne (stormer) from comment #4)
> (In reply to Tim-Philipp Müller from comment #3)
> > What is the actual / eventual purpose of this element, if I may ask?
> 
> Real-Time stream geolocalisation seems very useful to me. Combine this with
> color video, depth stream and gyroscopic data, you could turn a video
> recording into 3D.

Yes, this is a great use case, and there are many others. Others include
geofencing such that you start recording video/audio when you reach a certain
longitude/latitude region, or multiplexing GPS data along with A/V data into a
single mp4 file containing everything correlated on a per-frame basis. This also
can be used to easily send GPS data over a network. And, at its most basic, it
can be a one-liner to sample from the GPS in real-time.

Of course, to fully flesh out these use cases, we'd need a few more elements:

- gpsdenc: transform from in-memory gps_fix_t format to a persistent,
  machine-independent format. This would be used prior to sending off the
  machine or saving to a file.
- gpsddec: transform from a persistent, machine-independent format back into
  in-memory gps_fix_t format.
- gpsconvert: convert from gps_fix_t format to something human-readable, like
  text or JSON. Then GPS data could be displayed by a text sink, over sent over
  the network to a non-gstreamer application.

I would like to do these in the future, or someone else could, but we need to
start with a source to acquire the data.

> > 
> > What would seem more generally useful to me is for example an element that
> > puts current GPS co-ordinates as metadata on buffers, for example.
> 
> Though that needs to happen in two steps, since you need to first synch the
> GPS fixes with the data because of the delays. You cannot request a fix in
> the past, you need to capture, synch and then you can merge or mux.
> 

Yes. I think that treating GPS and other sensors as first-class citizens with
buffers dedicated to them will help in multiplexing and syncing up with A/V
data. In addition, it allows all the above use cases that might not even use A/V
data but be interesting in their own right.
Comment 6 Martin Kelly 2017-12-14 20:41:30 UTC
Created attachment 365560 [details] [review]
v2-0001-new-element-gpsdsrc.patch

Fixes a few issues:

- GST_ERROR --> GST_ELEMENT_ERROR (thanks!)
- Catch failing status of gps_stream
- Use application/gpsd as caps, since the gps_fix_t format is specific to gpsd and not general to GPS itself.
Comment 7 Martin Kelly 2017-12-19 18:35:20 UTC
FYI I will be out on vacation until January 3 but will respond to any new review comments after that.
Comment 8 Martin Kelly 2018-01-03 18:22:20 UTC
I am back. What is needed to move forward here?
Comment 9 Nicolas Dufresne (ndufresne) 2018-01-03 20:03:26 UTC
I think what's needed is a consensus on the data format. Best is to discuss our options on freenode/#gstreamer, __tim and ndufresne are our nicknames there. For my part, I'm back on Jan 8th though.

Tim was proposing, with a maybe, that we could serialized the C structure to KLV, so we gain muxer/demuxers support for free. As a second step, I'd go read the equivalent structure in gypsy, just to ensure it will be possible to replace the GPS source if needed for infrastructure coherence.

Though, my feeling is that caps should indicate that this is GPS data somehow.
Comment 10 Tim-Philipp Müller 2018-01-03 20:10:52 UTC
I very much see the use of geo localisation data in connection with media streams, what it not clear to me is that we should be sending this as a raw data stream of what is ultimately a non-standardised (?) format.

I think this information should rather be attached to media data via metas instead (whether KLV or a new geodata specific meta).
Comment 11 Martin Kelly 2018-01-03 22:21:20 UTC
Yes, this is a great discussion to have. I have thoughts but they are not fully baked yet. I'll get on freenode next week once Nicolas is back to discuss.
Comment 12 Martin Kelly 2018-01-18 00:31:17 UTC
After a discussion with Tim and Nicolas, we have agreed to keep gpsdsrc in a separate repo for now until the plugin matures, and we build more plugins supporting GPS data (encoder, decoder, mux, etc.). Once that happens, we can consider inclusion in plugins-bad.

I added gpsdsrc to this repo:
https://github.com/XevoInc/gst-sensors
Comment 13 GStreamer system administrator 2018-11-03 14:16:56 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/643.