What's the status of container format support, are there specs for Matroska/WebM, MP4, ...?

Good question, I am not sure. I think for WebM there is, but need to do some digging.

So, to start off, I guess I can make a copy of the .265 encoder and decoder, and work from there.

That doesn't seem like the best approach, those are quite complicated. As this comes also from Google and seems to include various VP8/9/10 tech, maybe the encoder/decoder for those would be a good start.

Good idea. By the way, at this stage in development, encoding speed is measured in frames per minute (!) - will a pipeline even run at this very slow encoding rate ?

Sure, as long as you don't have live input that is all not a problem.

Just to keep everyone updated I have been chipping away at this.

There ended up being a large amount of changes (VPX vs AOM), so it ended being easier to start a new plugin in -bad

https://github.com/Sean-Der/gst-plugins-bad/tree/av1

Currently I just have everything linking, init'ing everything AV1 and then having the GstVideoEncoder spitting out empty buffers. Hopefully this weekend I will have something generic working (we will see!)

If anyone is interested I am more then happy to put this somewhere where the work can be shared, but I have a long way to go before this is review worthy.

Awesome! Let me know how it goes : I would like to help, time permitting.

The API is basically like libvpx, so ideally we can share code with that (and not just copy&paste).

We should review the API to make sure it doesn't force unnecessary copies and can handle strides and stuff - has someone done that yet?

I don't think any work has been done since initial plugin work by Sean-Der in the summer. I've played around with the AV1 encoder, using the Xiph.org sample video set, but haven't had time to do anything more.

Hey All!

I have a version on my Github that can encode and I updated webmmux to support it, I was able to use ffplay on a file that GStreamer created (videotestsrc ! av1enc ! webmmux ! filesink)

However, I had questions about the best way to write things and wasn't able to get anyone on IRC so gave up. If someone was willing to 'mentor' on this one I would love to do it! I assume that this is going to be big enough that someone bigger in GStreamer dev is going to want it.

Also @slomo I don't know how much we can really share (or should) while it is similar to VPX there are substantial differences in the API. It seems like the plan was to make a big break for AV1 (and then in the future there will be AV2..AV3) which we can share code between. Little questions like these are what I need help with, to make sure I am writing code that will land.

Would love to take a couple days off from work and hack on this, happy to see it all the way through if I can get someone to approve/tell me the right way. I just don't want to work on this and not be able to get it merged/rewritten by someone else.

thanks!

Sean, that's great to hear. If you can provide your code as a patch against gst-plugins-bad (place it in the ext/av1 directory), we can get it reviewed here and merged. As long as you're interested in updating the patch until it's ready, which seems to be the case :)

For sharing code with libvpx, if it's not possible it's not possible. That's perfectly fine. A similar structure with some private base class as in the libvpx plugin might make sense though (if AV2, AV3, etc might be added later), which you probably have if you started from that plugin instead from scratch again?

Sean, are you working on this still and would be ready to share it? Otherwise someone else might take this up in the future

Created attachment 365476 [details] [review]
Add AV1 to gst-plugins-good/matroska

Hey slomo, sorry about that got pulled in another direction. My goal is to put ~2 hours of work into this daily until is done.

Patch wise, am I able to do a MVP (av1enc and av1dec) that has no flags, and then add them later?

How complete does that patch have to be? I promise I will see it all the way through but I would like to do things in as small of chunks as possible. The less churn, the more likely things are to land (and then I can continue providing patches from there)

sorry not familiar with how the development of large features and GStreamer go.

If I can do minimal av1enc/av1dec patches will have them ready next day or so

thanks!

(In reply to Sean-Der from comment #16)
> Patch wise, am I able to do a MVP (av1enc and av1dec) that has no flags, and
> then add them later?

  What do you mean by "flags" ? Properties ? I think it's fine to get a piece of concept working first and then extending it afterwards.

> 
> How complete does that patch have to be? I promise I will see it all the way
> through but I would like to do things in as small of chunks as possible. The
> less churn, the more likely things are to land (and then I can continue
> providing patches from there)
> 

  The patch should be "functional". It doesn't need to have all the bells and whistles (extra configuration properties, etc....), but we should be able to feed raw video and get a (compatible) encoded bitstream. And a decoder that be fed the bitstream and output raw video.

> 
> If I can do minimal av1enc/av1dec patches will have them ready next day or so
> 
> thanks!

 Sounds good ! Thank you

Created attachment 365570 [details] [review]
Add AV1 support with gst-plugins-bad/aom

I finished the VERY basic MVP. av1enc and av1dec work.

Also here is a Dockerfile that pulls my fork and AOM and builds everything, and an example gst-launch! So it should be really easy with just three commands to see it all working.

https://gist.github.com/Sean-Der/2cb7656db4002fea6b20ff7840863b96

I will go through and fill out the error codes/handling and cleanup everything in the next day or two. 

This has no properties, and is just the bare minimum I imagine even the most basic of edge cases would blow it up but I can get it powering an autovideosink so I am super excited.

I don't want to waste anyones time, but just want to know what is the bare minimum that is needed to merge. I want to get to that finish line as soon as possible, and not have the patch fall out of date etc...

thanks!

Nice work ! At a glance it does seem correct and sufficient for initial integration into -bad, I'll do some testing and see if we can merge it.

I didn't see any out-of-band information (codec_data or sth like that) handling, is the stream self-contained ?

Edward,

Thank you so much for the review (and hopefully merge)

If you see any quick fixes I wouldn’t be offended at all if you modify the patch and commit (some people get fussy about that I have seen)

And as soon as it gets in bad I will start a bug/thread about ‘the road to good’ itemizing what needs to be done I will do my best to get it all the way.

And it's in !

In terms of things to do:
* Properly handle EOS:
** av1enc: in the finish vmethod you want to drain out the remaining frames
** av1dec: Same thing, but in both the finish and drain vmethod
* gobject properties for configuring the encoder :)
* Maybe have some "sane" default settings on the encoder
* I tried, but failed, tu get the encoder to use multiple-threads for encoding. Might be worth checking what happens there.
* As asked before, I didn't see any "out-of-band" data that needs to be passed around. Is that normal for av1 ?
* ...
* world domination :)

Closing bug. You can open a new one for the features to be added.