When the RTSP server creates the RTP-Info header it asks the payloaders for the sequence number and RTP time stamps. However, when the rtponviftimestamp element is present, after the actual payloader, these values will not be taken from the first packet pushed by the payloader. This is because the payloaders always store the info for the last payloaded packet, butrtponviftimestamp the rtponviftimestamp element caches one buffer (it needs to do this to know whether to set the e-bit in the extension header or not). This means that the payloader will push (at least) two buffers while pre-rolling.

One way to fix this would be to make the payloader store the sequence number and RTP time stamp for the "pre-roll packet", e.g. the first packet payloaded when going to paused state. However, that would mean that the payloader would have to behave differently in paused and playing states. In paused state it would have to provide the sequence number and rtp timestamp for the "preroll packet", but in playing state it would have to return the values for the last payloaded packet.
Or the RTSP server would have to ask for them in different ways.

Another way would be to use an upstream custom query to ask for the sequence number and RTP time stamp. When the rtponviftimestamp element is present it would answer the query, when it's not the payloader would answer it.

I propese the latter approach. It requires changes in rtpbasepayload, rtponviftimestamp, and gst-rtsp-server. But it is a generic solution which does not require the payloaders to behave differently in different states.