*** Bug 654664 has been marked as a duplicate of this bug. ***

Created attachment 192057 [details]
Gdb (with most of the gst-xmlinspect output excised)

Created attachment 192176 [details] [review]
simple unit test for postproc

Minimal unit test that reproduces the issue for me. Somehow postproc gets to process garbage strings (looks like an issue in postproc at first glance).

This 'fixes' / hides the bug btw:

diff --git a/libpostproc/postprocess.c b/libpostproc/postprocess.c
index dd50daf..2e88b4d 100644
--- a/libpostproc/postprocess.c
+++ b/libpostproc/postprocess.c
@@ -742,7 +742,7 @@ const char pp_help[] =
 
 pp_mode *pp_get_mode_by_name_and_quality(const char *name, int quality)
 {
-    char temp[GET_MODE_BUFFER_SIZE];
+    char temp[GET_MODE_BUFFER_SIZE] = { 0, };
     char *p= temp;
     static const char filterDelimiters[] = ",/";
     static const char optionDelimiters[] = ":";

*** Bug 654780 has been marked as a duplicate of this bug. ***

This bug affects any application that tries to create a postproc based element. We should really get it fixed and make another gst-ffmpeg release..

This appears to have been fixed by commit 84fb4e9df737c856cca7767016110fc74bee56cb in ffmpeg.
It doesn't apply cleanly, but applying it by hand makes valgrind happy.
Part of that patch is to actually clear the temp buffer as you found.


From 84fb4e9df737c856cca7767016110fc74bee56cb Mon Sep 17 00:00:00 2001
From: Piotr Kaczuba <p.kaczuba@attika.ath.cx>
Date: Mon, 30 May 2011 13:19:35 +0200
Subject: [PATCH] postprocess.c: filter name needs to be double 0 terminated

---
 libpostproc/postprocess.c |    7 ++++---
 1 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/libpostproc/postprocess.c b/libpostproc/postprocess.c
index b2c35f5..bfb96e1 100644
--- a/libpostproc/postprocess.c
+++ b/libpostproc/postprocess.c
@@ -86,6 +86,7 @@ try to unroll inner for(x=0 ... loop to avoid these damn if(x ... checks
 //#define DEBUG_BRIGHTNESS
 #include "postprocess.h"
 #include "postprocess_internal.h"
+#include "libavutil/avstring.h"
 
 unsigned postproc_version(void)
 {
@@ -766,8 +767,8 @@ pp_mode *pp_get_mode_by_name_and_quality(const char *name, int quality)
     ppMode->maxClippedThreshold= 0.01;
     ppMode->error=0;
 
-#undef strncpy
-    strncpy(temp, name, GET_MODE_BUFFER_SIZE);
+    memset(temp, 0, GET_MODE_BUFFER_SIZE);
+    av_strlcpy(temp, name, GET_MODE_BUFFER_SIZE - 1);
 
     av_log(NULL, AV_LOG_DEBUG, "pp: %s\n", name);
 
@@ -823,7 +824,7 @@ pp_mode *pp_get_mode_by_name_and_quality(const char *name, int quality)
 
                 plen= strlen(p);
                 spaceLeft= p - temp + plen;
-                if(spaceLeft + newlen  >= GET_MODE_BUFFER_SIZE){
+                if(spaceLeft + newlen  >= GET_MODE_BUFFER_SIZE - 1){
                     ppMode->error++;
                     break;
                 }
-- 
1.7.1

This issue has probably allready been seen by several Fedora users at https://bugzilla.redhat.com/show_bug.cgi?id=723498 ...

I think we basically need this patch:

http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=84fb4e9df737c856cca7767016110fc74bee56cb

cherry-picked into the libav.org release/0.7/ git branch, so we can update our checkout and make a new release.

Please rebase that patch against current libav master and send the patch to the libav development mailing-list.

That's the only thing blocking it from going in the 0.7 branch.

Created attachment 194604 [details] [review]
postprocess.c: filter name needs to be double 0 terminated

Rebased patch.

commit 7b7928082b5d869f55f5d674930a610234aa8037
Author: Tim-Philipp Müller <tim.muller@collabora.co.uk>
Date:   Mon Jul 18 12:12:38 2011 +0100

    tests: add simple test for creating postproc elements
    
    https://bugzilla.gnome.org/show_bug.cgi?id=654634

commit 9d6f797f7a79c48d23f7c31fe4b267a7a5e8e0f9
Author: Tim-Philipp Müller <tim.muller@collabora.co.uk>
Date:   Sat Sep 3 12:30:26 2011 +0100

    Update internal libav snapshot to latest releases/0.7 tip for postproc fix
    
    https://bugzilla.gnome.org/show_bug.cgi?id=654634