Created attachment 15887 [details]
the text overlay plugin

I looked at it and it works quite well. (Haven't stresstested it tho)
I would not have implemented blitting to YUV420 myself however, that's
not the job of a text overlay. And it eats around 10% of CPU power on
my Athlon 2000.
You passed the x and y arguments to gst_text_overlay_blit_yuv420 the
wrong way around btw, which explains the weird position.
And you need freetype > 2.1.3, which isn't checked.

Now we just need a usable 2nd input plugin that takes some
"application/x-gst-subtitle" format or something (we really need more
work on subtitle stuff) and blits those the right way.

Changing severity to enhancement.

Created attachment 15952 [details]
Fixed x/y position, use the given plugin template to make it compile w/ gst 0.7.

  I'm not worried about performance at this point, though I wish to
get the correct architecture for best performance from the beginning.
 I am not convinced that rendering directly to a pixel buffer isn't
the best option.  But I could be wrong, we'll see.

  I looked at MPlayer's subtitle parser (subread.c) , but that code
looks terrible.  Besides, it reads directly from a FILE*, uses fgets,
fscanf, and such things porting to gstreamer is not easy.

  Next we need the plugin you mentioned that must be a "bin" element
that receives the parsed subtitles on input and renders text using a
textoverlay child element.  Am I right (I'm very green to gstreamer,
bare with me)?

 Argh! I'm getting the following bugzilla error:
No file was provided, or it was empty.

when trying to add an attachment.  So I'll put the patch inline:

Description:
Added properties to control alignment, position, font

--- gst-textoverlay/src/gsttextoverlay.c        2003-04-23
22:24:35.000000000 +0100
+++ gst-textoverlay2/src/gsttextoverlay.c       2003-04-24
02:00:08.000000000 +0100
@@ -16,9 +16,15 @@ static GstElementDetails textoverlay_det
  
 enum {
     ARG_0,
-    ARG_TEXT,
+    ARG_TEXT,
+    ARG_VALIGN,
+    ARG_HALIGN,
+    ARG_X0,
+    ARG_Y0,
+    ARG_FONT_DESC,
 };
  
+
 GST_PAD_TEMPLATE_FACTORY(textoverlay_src_template_factory,
                         "src",
                         GST_PAD_SRC,
@@ -109,25 +115,55 @@ gst_textoverlay_class_init(GstTextOverla
     gobject_class->get_property = gst_textoverlay_get_property;
  
     gstelement_class->change_state = gst_textoverlay_change_state;
-    /* FIXME: make dpi configurable (property) */
-    klass->pango_context = pango_ft2_get_context(72 /*dpi_x*/,
-                                                72 /*dpi_y*/);
+    klass->pango_context = pango_ft2_get_context(72, 72);
     g_object_class_install_property(G_OBJECT_CLASS(klass), ARG_TEXT,
-                                   g_param_spec_string("text","text",
+                                   g_param_spec_string("text", "text",
                                                        "Text to be
display,"
                                                        " in pango
markup format.",
                                                        "",
G_PARAM_WRITABLE));
+    g_object_class_install_property(G_OBJECT_CLASS(klass), ARG_VALIGN,
+                                   g_param_spec_string("valign",
"vertical alignment",
+                                                       "Vertical
alignment of the text. "
+                                                       "Can be either
'baseline', 'bottom', or 'top'",
+                                                       "baseline",
G_PARAM_WRITABLE));
+    g_object_class_install_property(G_OBJECT_CLASS(klass), ARG_HALIGN,
+                                  
g_param_spec_string("halign","horizontal alignment",
+                                                       "Horizontal
alignment of the text. "
+                                                       "Can be either
'left', 'right', or 'center'",
+                                                       "center",
G_PARAM_WRITABLE));
+    g_object_class_install_property(G_OBJECT_CLASS(klass), ARG_X0,
+                                   g_param_spec_int("x0","X position",
+                                                    "Initial X position."
+                                                    " Horizontal
aligment takes this point"
+                                                    " as reference.",
+                                                    G_MININT,
G_MAXINT, 0, G_PARAM_WRITABLE));
+    g_object_class_install_property(G_OBJECT_CLASS(klass), ARG_Y0,
+                                   g_param_spec_int("y0","Y position",
+                                                    "Initial Y position."
+                                                    " Vertical
aligment takes this point"
+                                                    " as reference.",
+                                                    G_MININT,
G_MAXINT, 0, G_PARAM_WRITABLE));
+    g_object_class_install_property(G_OBJECT_CLASS(klass), ARG_FONT_DESC,
+                                  
g_param_spec_string("font-desc","font description",
+                                                       "Pango font
description of font "
+                                                       "to be used
for rendering. "
+                                                       "See
documentation of "
+                                                      
"pango_font_description_from_string"
+                                                       " for syntax.",
+                                                       "",
G_PARAM_WRITABLE));
 }
  
  
 static void
-resize_bitmap(FT_Bitmap *bitmap, int width, int height)
+resize_bitmap(GstTextOverlay *overlay, int width, int height)
 {
-    int pitch = (width|3) + 1, size = pitch*height;
+    FT_Bitmap *bitmap = &overlay->bitmap;
+    int        pitch  = (width|3) + 1;
+    int        size   = pitch*height;
  
     /* no need to keep reallocating; just keep the maximum size so far */
-    if (width < bitmap->width && height < bitmap->rows) {
-       memset(bitmap->buffer, 0, bitmap->rows*bitmap->pitch);
+    if (size <= overlay->bitmap_buffer_size) {
+       memset(bitmap->buffer, 0, overlay->bitmap_buffer_size);
        return;
     }
     if (!bitmap->buffer) {
@@ -136,24 +172,25 @@ resize_bitmap(FT_Bitmap *bitmap, int wid
        bitmap->num_grays  = 256;
     }
     if (bitmap->buffer)
-       bitmap->buffer = g_realloc (bitmap->buffer, size);
+       bitmap->buffer = g_realloc(bitmap->buffer, size);
     else
-       bitmap->buffer = g_malloc (size);
+       bitmap->buffer = g_malloc(size);
     bitmap->rows  = height;
     bitmap->width = width;
     bitmap->pitch = pitch;
     memset(bitmap->buffer, 0, size);
+    overlay->bitmap_buffer_size = size;
 }
  
 static void
-set_text(GstTextOverlay *overlay, const gchar *text)
+render_text(GstTextOverlay *overlay)
 {
     PangoRectangle ink_rect, logical_rect;
  
-    pango_layout_set_markup(overlay->layout, text, -1);
     pango_layout_get_pixel_extents(overlay->layout, &ink_rect,
&logical_rect);
-    resize_bitmap(&overlay->bitmap, ink_rect.width, ink_rect.height +
ink_rect.y);
+    resize_bitmap(overlay, ink_rect.width, ink_rect.height + ink_rect.y);
     pango_ft2_render_layout(&overlay->bitmap, overlay->layout, 0, 0);
+    overlay->baseline_y = ink_rect.y;
 }
  
 static GstPadLinkReturn
@@ -180,9 +217,6 @@ gst_text_overlay_blit_yuv420(GstTextOver
     register int x, y;
 #define yuv420_pixel(x, y) (*(pixbuf + overlay->width*(y) + (x)))
  
-    /* FIXME: the text is not getting drawn at the correct position;
-     * my knowledge about YUV format sucks, I need help. */
-
     /* paint black margin, brute force approach */
     for (y = 0; y < bitmap->rows; y++)
     {
@@ -227,6 +261,7 @@ gst_textoverlay_chain(GstPad *pad, GstBu
 {
     GstTextOverlay *overlay;
     guchar         *pixbuf;
+    gint            x0, y0;
  
     g_return_if_fail(pad != NULL);
     g_return_if_fail(GST_IS_PAD(pad));
@@ -237,12 +272,35 @@ gst_textoverlay_chain(GstPad *pad, GstBu
      
     pixbuf = GST_BUFFER_DATA(buf);
  
+    x0 = overlay->x0;
+    y0 = overlay->y0;
+    switch (overlay->valign)
+    {
+    case GST_TEXT_OVERLAY_VALIGN_BOTTOM:
+       break;
+    case GST_TEXT_OVERLAY_VALIGN_BASELINE:
+       y0 -= overlay->baseline_y;
+       break;
+    case GST_TEXT_OVERLAY_VALIGN_TOP:
+       y0 -= overlay->bitmap.rows;
+       break;
+    }
+
+    switch (overlay->halign)
+    {
+    case GST_TEXT_OVERLAY_HALIGN_LEFT:
+       break;
+    case GST_TEXT_OVERLAY_HALIGN_RIGHT:
+       x0 -= overlay->bitmap.width;
+       break;
+    case GST_TEXT_OVERLAY_HALIGN_CENTER:
+       x0 -= overlay->bitmap.width/2;
+       break;
+    }
+
     if (overlay->bitmap.buffer)
-       gst_text_overlay_blit_yuv420(overlay, &overlay->bitmap, pixbuf,
-                                    /* horizontal alignment: center */
-                                    overlay->width/2 -
overlay->bitmap.width/2,-                                    /*
vertical alignment: bottom */
-                                    overlay->height -
overlay->bitmap.rows - 4);
+       gst_text_overlay_blit_yuv420(overlay, &overlay->bitmap,
pixbuf, x0, y0);+
     gst_pad_push(overlay->srcpad, buf);
 }
  
@@ -299,7 +357,10 @@ gst_textoverlay_init(GstTextOverlay *ove
  
     overlay->layout =
pango_layout_new(GST_TEXTOVERLAY_GET_CLASS(overlay)->pango_context);
     memset(&overlay->bitmap, 0, sizeof(overlay->bitmap));
-    set_text(overlay, "<span size=\"xx-large\" weight=\"bold\">Hello
World!</span>");
+
+    overlay->halign = GST_TEXT_OVERLAY_HALIGN_CENTER;
+    overlay->valign = GST_TEXT_OVERLAY_VALIGN_BASELINE;
+    overlay->x0 = overlay->y0 = 0;
 }
  
  
@@ -312,10 +373,60 @@ gst_textoverlay_set_property(GObject *ob
     g_return_if_fail(GST_IS_TEXTOVERLAY(object));
     overlay = GST_TEXTOVERLAY(object);
  
-    switch (prop_id) {
+    switch (prop_id)
+    {
+
     case ARG_TEXT:
-       set_text(overlay, g_value_get_string(value));
+       pango_layout_set_markup(overlay->layout,
g_value_get_string(value), -1);+       render_text(overlay);
+       break;
+
+    case ARG_VALIGN:
+       if (strcasecmp(g_value_get_string(value), "baseline") == 0)
+           overlay->valign = GST_TEXT_OVERLAY_VALIGN_BASELINE;
+       else if (strcasecmp(g_value_get_string(value), "bottom") == 0)
+           overlay->valign = GST_TEXT_OVERLAY_VALIGN_BOTTOM;
+       else if (strcasecmp(g_value_get_string(value), "top") == 0)
+           overlay->valign = GST_TEXT_OVERLAY_VALIGN_TOP;
+       else
+           g_warning("Invalid 'valign' property value: %s",
+                     g_value_get_string(value));
+       break;
+
+    case ARG_HALIGN:
+       if (strcasecmp(g_value_get_string(value), "left") == 0)
+           overlay->halign = GST_TEXT_OVERLAY_HALIGN_LEFT;
+       else if (strcasecmp(g_value_get_string(value), "right") == 0)
+           overlay->halign = GST_TEXT_OVERLAY_HALIGN_RIGHT;
+       else if (strcasecmp(g_value_get_string(value), "center") == 0)
+           overlay->halign = GST_TEXT_OVERLAY_HALIGN_CENTER;
+       else
+           g_warning("Invalid 'halign' property value: %s",
+                     g_value_get_string(value));
+       break;
+
+    case ARG_X0:
+       overlay->x0 = g_value_get_int(value);
+       break;
+
+    case ARG_Y0:
+       overlay->y0 = g_value_get_int(value);
+       break;
+
+    case ARG_FONT_DESC:
+    {
+       PangoFontDescription *desc;
+       desc =
pango_font_description_from_string(g_value_get_string(value));
+       if (desc) {
+           g_message("font description set: %s",
g_value_get_string(value));
+           pango_layout_set_font_description(overlay->layout, desc);
+           pango_font_description_free(desc);
+           render_text(overlay);
+       } else
+           g_warning("font description parse failed: %s",
g_value_get_string(value));
        break;
+    }
+
     default:
        break;
     }

You would not use a bin element that does that. Plugging the right
elements after one another is job of the user or the autoplugger.
So you would write the elements for textoverlay (taking two inputs
obviously) and for subtitle parsing and test them manually with
gst-launch.
And after that works you try the autoplugger.

btw: The best way to get help is still on IRC.

  Ok, if you think I should not use a bin element, I won't use it
(less work and everything :)
  But in that case, who controls the position of the text, font, etc.?
 I'm guessing the application, am I right?  I was thinking that the
textoverlay plugin would be very generic, not just for rendering
subtitles...
  Thanks for your comments!

Created attachment 16061 [details]
new tarball; added a text_sink pad (now text can be set with either the 'text' property or throught the text_sink pad)

Created attachment 16219 [details]
New version: optimize a bit the blitting; paint U and V fields too.

I put the source in the gst-sandbox module.  Please develop it further
there, so we don't have to keep putting big tarballs as attachments in.

Let me know if you have any problems with it

development in cvs now