In GStreamer our default stride for I420 is rounded up to an integer multiple of 4. Which will give you
> 1364*768 + 1368/2 * 768/2 + 1368/2 * 768/2 = 1572864

If you want to use a different stride you'll have to use GstVideoMeta.

Note that not respecting 4 bytes alignment for stride may prevent you from using GStreamer software converter. I don't know what in WebRTC can give you raw frames, but I suspect it is a bug.

Got it. Thank you guys. I'll take a look at the webrtc code.

In case anyone else needs this. This is what I end up doing which fixes the problem.

-----

static void
WebRtcI420ToGStreamer (guint8 *dst, guint8 *src, gint width, gint height)
{
  GstVideoInfo info;

  gst_video_info_init (&info);
  gst_video_info_set_format (&info, GST_VIDEO_FORMAT_I420, width, height);

  guint src_y_stride = width;
  guint src_u_stride = (width + 1) / 2;
  guint src_v_stride = (width + 1) / 2;

  guint dst_y_stride = GST_VIDEO_INFO_COMP_STRIDE (&info, 0);
  guint dst_u_stride = GST_VIDEO_INFO_COMP_STRIDE (&info, 1);
  guint dst_v_stride = GST_VIDEO_INFO_COMP_STRIDE (&info, 2);

  // Y
  for (guint h = 0; h < height; h++)
    { memcpy (dst, src, src_y_stride);
      dst += dst_y_stride;
      src += src_y_stride;
    }
  // U
  for (guint h = 0; h < height / 2; h++)
    { memcpy (dst, src, src_u_stride);
      dst += dst_u_stride;
      src += src_u_stride;
    }
  // V
  for (guint h = 0; h < height / 2; h++)
    { memcpy (dst, src, src_v_stride);
      dst += dst_v_stride;
      src += src_v_stride;
    }
}

You can also use the GstVideoFrame API: gst_video_frame_copy()

Note, in both cases, this isn't a solution to the problem. We should be able to impose on WebRTC the alignment constraints, and allow zero-copy off of it. Otherwise we'd never get high resolution or embedded device to work properly.

Yeah, your WebRTC API should ideally allow you to provide it a buffer and the configuration of it (plane offsets, strides, etc)... and otherwise you can still add a GstVideoMeta with the WebRTC strides to the buffer as long as downstream supports GstVideoMeta. No need to copy stuff around in that case.

I assume with WebRTC you mean libwebrtc here?

(In reply to comment #7)
> Yeah, your WebRTC API should ideally allow you to provide it a buffer and the
> configuration of it (plane offsets, strides, etc)... and otherwise you can
> still add a GstVideoMeta with the WebRTC strides to the buffer as long as
> downstream supports GstVideoMeta. No need to copy stuff around in that case.
> 
> I assume with WebRTC you mean libwebrtc here?

I have an app written using the WebRTC project, https://code.google.com/p/webrtc/, like if it was a WebRTC app. I then used shmsink to pass the frames to a GStreamer only app. Everything works great so far.

This is the base class for a VideoFrame.

https://code.google.com/p/webrtc/source/browse/trunk/talk/media/base/videoframe.h

And I don't see it supports copying considering different strides, etc.

I don't think GstVideoMeta will travel with the buffer with shmsink. And the other side, where I should add GstVideoMetta, is kind of generic. So, right now, is hard for me to add a special case for the buffer coming from WebRTC.

But it's really handy to know anyways. So, thanks!

Nicolas, is the 4 byte alignment a standard thing in a raw I420 image, or is it something that GStreamer uses by default? Sorry, I am not very well versed in image/color formats. If it's something that it should be really fixed, then I would dig into it and try to fix it.

It's a "standard" thing, yes. Or even larger alignments are used, especially by hardware.

The alignments GStreamer uses by default are usually 4 bytes, as is used by the XVideo extension and many other things.

For the reference, "many other things" includes ffmpeg/libav in non-direct rendering mode (in direct rendering alignment must be 16 bytes).

Thank you guys. I have investigated a little bit more. Internally, webrtc uses libyuv.

And the VideoFrame class provides CopyToPlanes. So, the following code works for me as well:

  size_t size = GST_VIDEO_INFO_SIZE (&video_info);
  guint8 *image = (guint8 *) g_malloc0 (size);

  guint8 *dst_y = image + GST_VIDEO_INFO_COMP_OFFSET (&video_info, 0);
  guint8 *dst_u = image + GST_VIDEO_INFO_COMP_OFFSET (&video_info, 1);
  guint8 *dst_v = image + GST_VIDEO_INFO_COMP_OFFSET (&video_info, 2);

  guint dst_y_stride = GST_VIDEO_INFO_COMP_STRIDE (&video_info, 0);
  guint dst_u_stride = GST_VIDEO_INFO_COMP_STRIDE (&video_info, 1);
  guint dst_v_stride = GST_VIDEO_INFO_COMP_STRIDE (&video_info, 2);

  frame -> CopyToPlanes(dst_y, dst_u, dst_v,
                        dst_y_stride, dst_u_stride, dst_v_stride);

  GstBuffer *buffer = gst_buffer_new_wrapped (image, size);

As Nicolas suggested, it would be better to allow a zero-copy of it.