This is probably best solved by adding a property to video/theora caps that indicates that the theora stream should be constant frame rate.  I.e., for Ogg.  Let's call it "strict-framerate=true".  When this is set, theoraenc should act like it has an internal videorate element, and clean up the stream as necessary.  When it is not set, (i.e., for matroska or RTP), it should encode like normal.  In neither case should it be doing anything resembling what it does now.

Created attachment 200620 [details] [review]
theoraenc: emit drop frames instead of resetting decoder

When a buffer is received with a timestamp later than expected
by more than three quarters of a frame duration, theoraenc would
reset the encoder and restart encoding from the new timestamp.
This caused spurious discontinuities, a lot of keyframes, and
rate control restarting from scractch.

Instead, we now resubmit the old buffer enough times to make up
for the "lost" frames, which will allow libtheora to emit drop
frames to fill up the time between last frame and the new one.

Additionally, if caps contain a boolean strict-framerate set
to false, we just copy the incoming timestamp to the outgoing
buffer without doing any drop frames fixup. This allows any
downstream element to ignore the fixed framerate nature of
Theora and use timestamps rather than frame number.

The default is to do the fixup (ie, as if strict-framerate is
true), since this is closest to the nature of Theora (where
timing is determined by frame number and framerate).

This change will also make it easier to do fake VFR streams,
where the nominal framerate is set high (eg, 1000 fps), but
actual frames are emitted way less often, with drop frames to
fill in to 1000 fps.

Since libtheora does not offer an API to tell it that the
previous frame is to be duplicated (the dup frames ctl must
be called before encoding), this requires the previous frame
to be kept, and resubmitted at least once. This causes higher
memory consumption, but the alternative would be to only code
frames when the next one is received (and thus when we know
if a frame needs duplicates), which would introduce arbitrary
extra latency.

Created attachment 200621 [details] [review]
oggmux: explicitely require strict framerate behavior for Theora

Just to keep things in synced....
this bug connect with other one:
https://bugzilla.gnome.org/show_bug.cgi?id=627459

short description:
resubmitting frames make no sense especially if we know it is a duplicate frame. New videorate was updated to make it possible for theoraenc count dup/GAP frames and let encoder know how many granules should be produced for oggmux. So, if for example, webcam drop frame rate to 1fps and default frame rate is 30fps, theoraenc can produce valid stream without increasing cpu usage. It mean: one frame in - one frame out, no need to recreate 29 frames and make cpu beasy for no reason.

This bug has an assigned developer but has not received activity in almost a year.

Is the assigned person still working on this ?

Comment on attachment 200621 [details] [review]
oggmux: explicitely require strict framerate behavior for Theora

Not sure this is going to work like this in 1.x, with the stricter subset checks.

Comment on attachment 200620 [details] [review]
theoraenc: emit drop frames instead of resetting decoder

Needs to be updated for 1.0 and port to GstVideoEncoder base class.

I just looked at this, and the port to base classes was accompanied by the removal of the strict timestamp check and reset. So while there is no code for explicit dup frames, the main issue (encoder reset) seems gone. IIRC libtheora is able to detect identical frames and emit dup frames itself, should these be generated by upstream (though this will require a bit of CPU usage).
So I'm inclined to close this as obsolete unless somebody objects.

Right, let's close this as obsolete then.