bug 674658 not bug 675658

Created attachment 214157 [details]
Screenshot of erronous html output

For example, with current gtk-doc master, the following
gtk-doc comment block:

/**
 * GdkPixbuf:bits-per-sample:
 *
 * Type: gint
 * Transfer: full
 *
 * This is the same list of codecs as whatever
 * without....
 */

results in the html output as attached, whereas previous
versions of gtk-doc would have ignored the Type: and Transfer:
tags (confirmed with at least gtk-doc 1.15).

Huh, there are no fields named Type: or Transfer: that are part of gtk-doc syntax. Where do you see them specified?

(In reply to comment #3)
> Huh, there are no fields named Type: or Transfer: that are part of gtk-doc
> syntax. Where do you see them specified?

The basic set of "Tags:" recognized by gtk-doc are:
 - Description:
 - Returns:
 - Return value:
 - Since:
 - Stability:
 - Deprecated:

gobject-instrospection adds the following:
 - Virtual:
 - Attributes:
 - Rename to:
 - Type:
 - Unref func:
 - Ref func:
 - Set value func:
 - Get value func:
 - Transfer:
 - Value:

as documented on https://live.gnome.org/action/login/GObjectIntrospection/Annotations

Sigh. Are the annotation people aware that constantly inventing new stuff that conflicts with existing gtk-doc syntax in a creative way is a recipe for distaster?  Or is ensuring that gtk-doc will be constantly playing catch-up -- i.e. most people will see it broken -- a part of some grand plan?

I wonder why annotations could not settle on a single bloody syntax that is *different* from gtk-doc tags and thus *always recongnizable as such* so that it can be generically and safely ignored by any parser that does implement the annotation du jour yet.

Does this work? The syntax below is the agreed syntax with the GI folks. For the syntax below they never send a patch to gtk-doc. Besides I think it does not scale nicely. The annotations add extra information to an item and thus should be attached to the item they annotate.

/**
 * GdkPixbuf:bits-per-sample: (type gint)(transfer full):
 *
 * This is the same list of codecs as whatever
 * without....
 */


Also on the example:
"Type: gint" is redundant. I would remove that to avoid the comment going stale.
"Transfer: full" does not make sens for integers, right?

The above seems to 'work' for me. Although gtk-doc is not showing anything for the property in the generated docs. If he syntax works for gi, I'll make the gtk-doc changes to for properties and signals to show the annotations.

(In reply to comment #5)
> Sigh. Are the annotation people aware that constantly inventing new stuff that
> conflicts with existing gtk-doc syntax in a creative way is a recipe for
> distaster?  Or is ensuring that gtk-doc will be constantly playing catch-up --
> i.e. most people will see it broken -- a part of some grand plan?

Yes, sorry.  Your criticism is fair.

(In reply to comment #4)

> gobject-instrospection adds the following:
...
> https://live.gnome.org/action/login/GObjectIntrospection/Annotations

So...yeah, these just appeared organically.  The toplevel syntax of:

/**
 * Foo: (element-type int)
 *
 * Some description here.
 */

appeared later (and it's clearly better).

I think though Farstream adding the "block annotations" before the descripton was relying on an unintentional feature of the annotation parser.  All the examples on that page have it after the description.

I think 3 things should happen:

1) Farstream should move its block annotations to the bottom  (or to the top has header annotations)
2) gtk-doc should add in the list of block annotations g-i added
3) We should avoid adding any more block annotations in the future, in favor of header annotations.

I vote for 1) moving the tags as shown in comment #6 and comment #9. I will make a patch to gtk-doc to not silently ignore them.

Created attachment 214195 [details]
GTK-Doc comment block format, as discussed on IRC

Created attachment 214196 [details]
Actual IRC discussion on above GTK-Doc comment block format

Attached the irc discussion from today for reference...

(In reply to comment #10)
> I think 3 things should happen:
> 
> 1) Farstream should move its block annotations to the bottom  (or to the top
> has header annotations)
> 2) gtk-doc should add in the list of block annotations g-i added
> 3) We should avoid adding any more block annotations in the future, in favor of
> header annotations.

(In reply to comment #11)
> I vote for 1) moving the tags as shown in comment #6 and comment #9. I will
> make a patch to gtk-doc to not silently ignore them.

I have the impression point 1) was meant as: Farstream will have to
rewrite their comment blocks as follows to fix the g-i side of this
bug (bug #674658), as we agreed on irc that the comment block description
*has* to be written *before* any tags to be considered valid GTK-Doc (let's
ignore the point whether those tags are supported by GTK-Doc or not
for the moment, they might just as well have been Deprecated: or Returns:
or whatever...):

/**
 * FsSession:codecs-without-config:
 *
 * This is the same list of codecs as #FsSession:codecs
 * without....
 *
 * Type: GLib.List(FsCodec)
 * Transfer: full
 */

ie, point 1) is not about moving the g-i specific tags into
annotations on the identifier, as shown in comment #6 and comment #9
because of the following...

... I think Colin meant that all 3 points will have to be implemented
if we want to prevent breaking g-i annotations that have been supported
(in g-i) and have been used in the wild for 3 years already:
2009-02-20: TAG_ATTRIBUTES added in d5215b23315e7c9c22c6a32218bb6f8027a9dd4c
2009-02-28: TAG_VIRTUAL (currently known as TAG_VFUNC) added in fdbe3cc3e1cfaa546648a76b1dca72beead0b65b
2009-03-19: TAG_RENAME_TO added in 23e6fa6993c046de032598127ea48d4a7ee00935
2010-05-11: TAG_TYPE added in 23e6fa6993c046de032598127ea48d4a7ee00935
etc. The complete list of g-i specific tags is currently as follows:
Virtual, Attributes, Rename to, Type, Unref func, Ref func,
Set value func, Get value func, Transfer, Value

To be clear, g-i doesn't support any of those tags in an "annotation
applied to the identifier" form. The only annotations that can currently be
applied on that level are (skip), (foreign), (constructor) and (method) [1],
which are completely different animals than the tags listed above...

Point 2) argues that GTK-Doc should be extended to support this list of
g-i specific tags, to prevent their text from bleeding into the generated
HTML for example (the point shown in the attached screenshot)

Point 3) then argues that this list of g-i specific tags should no longer
be extended and that any new stuff that needs to apply at the comment block
level should go into the identifier.

Personally, I'd argue against this as it would become a mess having
2 distinct ways of adding data that applies to the comment block level,
knowing full well that these extra organically g-i grown tags are unfortunate,
but unless we want to break existing annotations we're kinda stuck with
them...

[1]  The DockBlock class (in annotationparser.py) has never validated
DockBlock.options (aka annotations) set on a comment block identifier,
but maintransformer.py and transformer.py access "skip", "foreign",
"constructor" and "method" annotations on that level. There's clearly
more room for improvement there too, but that's a different bug for
a different product :)

/**
 * FsSession:codecs-without-config:
 *
 * This is the same list of codecs as #FsSession:codecs
 * without....
 *
 * Type: GLib.List(FsCodec)
 * Transfer: full
 */

is a starting point, but it then will be shown verbatim as part of the main comment block. 

I think we want to break existing annotations. This format was never agreed upon. Lets do some grepping next to figure out how much it has been used.

Most important to me would be to:
* implement support for annotations on identifiers in g-i
* deprecate the top-level tags, ev. offer a tool to migrate

As said I won't add support for the top-level tags in gtk-doc, but if we agree that having annotation on the identifiert is the way to go, I will work on that one.

(In reply to comment #15)
> I think we want to break existing annotations. This format was never agreed
> upon. Lets do some grepping next to figure out how much it has been used.

To be clear, g-i's annotationparser will not be broken by me. It will
continue to support g-i specific top-level "Tag:"s, but complain with a
deprecation warning when such a "Tag:" is encountered, exactly what we
are already doing in other cases of malformed GTK-Doc comment blocks.

It is up to the maintainer of the code containing malformed GTK-Doc
comment blocks to fix whatever is the cause of these warnings when he/she
decides the time has come to do so (and with the case of deprecating
g-i specific top-level "Tag:"s, this means when said maintainer decides
support for older versions of g-i can be dropped).

> Most important to me would be to:
> * implement support for annotations on identifiers in g-i
> * deprecate the top-level tags, ev. offer a tool to migrate

With "annotations on identifiers" and the deprecation of g-i specific
toplevel "Tag:"s in mind, I finally finished adding about 160 test case
comment blocks to g-i yesterday evening and have stumbled across some
issues, both on the g-i and gtk-doc side of things. For gtk-doc:

1) How would the following "Type:" tag be expressed with annotations
   on the identifier:

   /**
    * RegressTestObj:hash-table:
    *
    * Test.
    *
    * Type: GLib.HashTable(utf8,gint8)
    */

   Like this?

   /**
    * RegressTestObj:hash-table:(type GLib.HashTable(utf8,gint8)):
    *
    * Test.
    */

   Looks a bit confusing to me with all those parentheses, no?
   Any suggestions? Note that AnnotationParser.parse_options()
   does not currently work with the above notation :/

2) Stuffing the equivalent annotations for all those g-i specific "Tag:"s
   on the identifier line will most probably lead to overly long
   identifier lines.
   g-i's annotationparser currently emits a warning when annotations span
   multiple lines, but only uses the first line (backward compatible behavior
   with the old parser).
   Note that multi-line annotations have already been spotted -and fixed-
   in the wild on "@parameter"s thanks to said warnings (looks like some
   people can get pretty religious when it comes to coding style, lines
   no longer than 80 columns wide, etc).

   Would it be a good idea to add support for multi-line annotations?

For g-i:
1) Not all g-i specific tags have an annotation counterpart.
   Is it OK to go on and create:

   OPT_VFUNC = 'virtual'
   OPT_RENAME_TO = 'rename-to'
   OPT_UNREF_FUNC = 'unref-func'
   OPT_REF_FUNC = 'ref-func'
   OPT_SET_VALUE_FUNC = 'set-value-func'
   OPT_GET_VALUE_FUNC = 'get-value-func'
   OPT_VALUE = 'value'

   counterparts to TAG_VFUNC, TAG_RENAME_TO, TAG_UNREF_FUNC,
   TAG_REF_FUNC, TAG_SET_VALUE_FUNC, TAG_GET_VALUE_FUNC and
   TAG_VALUE ?

   The idea being that AnnotationParser._parse_comment_block(), on
   encountering a deprecated g-i specific "Tag:" will transform said
   TAG_whatever into the equivalent OPT_whatever and create (or merge)
   the expected objects on DocBlock.options instead of
   DocBlock.tags[TAG_whatever].options

2) The "Attributes:" tag does not cleanly map to the "(attribute)"
   annotation. As in a single "Attributes:" tag can result in multiple
   "(attribute)" annotations on the same identifier. Observe both
   in the following contrived example:

   /**
    * shiny_function:(attribute my.key1 val1) (attribute my.key2 val2):
    *
    * Shiny function doing something.
    *
    * Attributes: (my.key1 val1) (my.key2 val2)
    */

   Also, both do not make use of the capabilities of
   AnnotationParser.parse_options(), which is why I'd like to propose
   we deprecate both options above and replace them with the following:

   /**
    * identifier:(attributes my.key1=val1 my.key2=val2):
    *
    * Shiny function doing something.
    */

Surely more issues will turn up as more tests get added, but this is
already enough to get started I guess...

(In reply to comment #16)
> For gtk-doc:
> 
> 1) How would the following "Type:" tag be expressed with annotations
>    on the identifier:
> 
>    /**
>     * RegressTestObj:hash-table:
>     *
>     * Test.
>     *
>     * Type: GLib.HashTable(utf8,gint8)
>     */

Heh, some of g-i's regression tests use the above, others use:

      * Type: GLib.HashTable<utf8,gint8>

Anybody an idea what that's all about?

(In reply to comment #17)
> (In reply to comment #16)
> > For gtk-doc:
> > 
> > 1) How would the following "Type:" tag be expressed with annotations
> >    on the identifier:
> > 
> >    /**
> >     * RegressTestObj:hash-table:
> >     *
> >     * Test.
> >     *
> >     * Type: GLib.HashTable(utf8,gint8)
> >     */
> 
> Heh, some of g-i's regression tests use the above, others use:
> 
>       * Type: GLib.HashTable<utf8,gint8>
> 
> Anybody an idea what that's all about?

We tried to use the latter format before realizing that of course comments are docbook (XML) fragments, so we can't use < and >.  It was a really stupid mistake =(

(In reply to comment #16)
> (In reply to comment #15)
> > I think we want to break existing annotations. This format was never agreed
> > upon. Lets do some grepping next to figure out how much it has been used.
> 
> To be clear, g-i's annotationparser will not be broken by me.

Note I'm OK with some breaks as long as there's an alternative syntax available in both old and new g-i that also doesn't break gtk-doc (ideally, old and new).

The thing that we want to avoid is making people have to somehow vary their annotation syntax depending on g-i version.

> With "annotations on identifiers" and the deprecation of g-i specific
> toplevel "Tag:"s in mind, I finally finished adding about 160 test case
> comment blocks to g-i yesterday evening

That sounds really great =)

>    /**
>     * RegressTestObj:hash-table:(type GLib.HashTable(utf8,gint8)):
>     *
>     * Test.
>     */

Probably...

>    Looks a bit confusing to me with all those parentheses, no?
>    Any suggestions? Note that AnnotationParser.parse_options()
>    does not currently work with the above notation :/

I don't have a good alternative though.  

> 2) Stuffing the equivalent annotations for all those g-i specific "Tag:"s
>    on the identifier line will most probably lead to overly long
>    identifier lines.
>    g-i's annotationparser currently emits a warning when annotations span
>    multiple lines, but only uses the first line (backward compatible behavior
>    with the old parser).
>    Note that multi-line annotations have already been spotted -and fixed-
>    in the wild on "@parameter"s thanks to said warnings (looks like some
>    people can get pretty religious when it comes to coding style, lines
>    no longer than 80 columns wide, etc).
> 
>    Would it be a good idea to add support for multi-line annotations?

If we're finding a need for that, then maybe we should un-deprecate Tags.

> 1) Not all g-i specific tags have an annotation counterpart.
>    Is it OK to go on and create:

Well, let's first figure out whether we need to keep Tags.
 
> 2) The "Attributes:" tag does not cleanly map to the "(attribute)"
>    annotation. As in a single "Attributes:" tag can result in multiple
>    "(attribute)" annotations on the same identifier. Observe both
>    in the following contrived example:
> 
>    /**
>     * shiny_function:(attribute my.key1 val1) (attribute my.key2 val2):

This would actually have to be (attribute (my.key1 val1) (my.key2 val2))

Colin, Stefan,

The branch at https://github.com/dieterv/gobject-introspection/tree/annotationparser-wave2
has an almost completed implementation of g-i's annotationparser
which can handle the new annotations on identifier as discussed
above but still understands the old g-i specific Tag: notation.
There's still Attributes: to fix and a tool underway that's meant
to rewrite GTK-Doc comment blocks in source files.

Before we go further and possibly merge this into g-i master, are
you aware of any other consumers of GTK-Doc comment blocks that will
need to be adapted to this new notation?

Thanks,
Dieter

(In reply to comment #20)

> 
> Before we go further and possibly merge this into g-i master, are
> you aware of any other consumers of GTK-Doc comment blocks that will
> need to be adapted to this new notation?

As far as I understand it, farstream is one:

http://cgit.collabora.com/git/farstream.git/commit/?id=39450457190675716c8e66700e35b05334a6a1a0

(In reply to comment #21)
> (In reply to comment #20)
> 
> > 
> > Before we go further and possibly merge this into g-i master, are
> > you aware of any other consumers of GTK-Doc comment blocks that will
> > need to be adapted to this new notation?
> 
> As far as I understand it, farstream is one:
> 
> http://cgit.collabora.com/git/farstream.git/commit/?id=39450457190675716c8e66700e35b05334a6a1a0

Nono, I mean comment block parsers like gtkdoc-mkdb and annotationparser.py.
Where does valadoc get's its documentation text from for example? Out of
comment blocks? If yes, it'll probably need to be updated to these new
annotations on the identifier too...

Also, let's get rid of the NEEDINFO status, afaik the requested information has been provided.

Hi Stefan,

(In reply to comment #15)

> As said I won't add support for the top-level tags in gtk-doc, but if we agree
> that having annotation on the identifiert is the way to go, I will work on that
> one.

Looks like we're pretty far along on this one, does it make sense to you to add?

(In reply to comment #22)
> Nono, I mean comment block parsers like gtkdoc-mkdb and annotationparser.py.
> Where does valadoc get's its documentation text from for example? Out of
> comment blocks? If yes, it'll probably need to be updated to these new
> annotations on the identifier too...

18:03 <flo> dieterv: Hey
18:07 <dieterv> flo: hi
18:07 <dieterv> flo: so, I was wondering where valadoc get's is documentation from
18:07 <dieterv> flo: does it parse source files for gtk-doc comment blocks
18:08 <dieterv> flo: or is it using a different source?
18:11 <flo> dieterv: The source depends on your flags. The comments in your vala-sources are the default-source.
18:12 <dieterv> flo: so it never touches gtk-doc comment blocks?
18:12 <flo> no
18:12 <flo> We just generate some of them for gtkdoc and bootstrap it.
18:13 <dieterv> flo: that's great, so it won't be affected by the small format changes wrt gobject-introspection annotations from bug https://bugzilla.gnome.org/show_bug.cgi?id=676133#c22
18:13 <bugbot> Bug 676133: gtk-doc, normal, Normal, ---, gtk-doc-maint, NEW, annotationparser: current code mix tags and description
18:17 <flo> dieterv: Yeah, I think so.
18:17 <dieterv> flo: ok if I paste this on the bug?
18:18 <flo> sure
18:18 <dieterv> flo: great, thanks :)

This is blocking https://bugzilla.gnome.org/show_bug.cgi?id=684319

So reading this again, I'm unsure that the plan to add all new introspection annotations as "firstline tags" is going to work.  We're going to end up with some really terribly long lines. 

Particularly since some of these annotations would be quite nice to have in the gtk-doc, we should really make sure gtk-doc understands them.

A third way which hasn't been discussed is having an entirely separate introspection-specific comment.  It's probably too ugly though:

/**
 * g_foo:
 * @foo: a #GFoo
 *
 * Do the bar on the foo.
 * Returns: %TRUE on success
 */
/* introspection: g_foo
 * Alias: bar
 */
gboolean
g_foo (GFoo *foo)
{
...
}

(In reply to comment #25)
> This is blocking https://bugzilla.gnome.org/show_bug.cgi?id=684319
> 
> So reading this again, I'm unsure that the plan to add all new introspection
> annotations as "firstline tags" is going to work.  We're going to end up with
> some really terribly long lines. 

How about allowing spanning annotation fields over multiple lines? Maybe
not only on the identifier part, but also on parameters and tags?

Something like:
/**
 * shiny_function:(attribute my.key1 val1)
 *                (attribute my.key2 val2):
 *
 * Shiny function doing something.
 */

Or:
/**
 * shiny_function:
 *   (attribute my.key1 val1)
 *   (attribute my.key2 val2):
 *
 * Shiny function doing something.
 */

But not:
/**
 * shiny_function:(attribute my.key1 val1) (attribute
 *                my.key2 val2):
 *
 * Shiny function doing something.
 */

(In reply to comment #26)
> Something like:
> /**
>  * shiny_function:(attribute my.key1 val1)
>  *                (attribute my.key2 val2):
>  *
>  * Shiny function doing something.
>  **/
> 
> Or:
> /**
>  * shiny_function:
>  *   (attribute my.key1 val1)
>  *   (attribute my.key2 val2):
>  *
>  * Shiny function doing something.
>  **/

I like this syntax.

It would expand what is recognised as introspection annotation a bit, requiring that actual documentation does not start with a parenthesised statement followed by a colon, but that's quite reasonable.

(In reply to comment #26)

> How about allowing spanning annotation fields over multiple lines? Maybe
> not only on the identifier part, but also on parameters and tags?
> 
> Something like:
> /**
>  * shiny_function:(attribute my.key1 val1)
>  *                (attribute my.key2 val2):
>  *

Those are just attributes though - we're suggesting doing this for all new toplevel tags right?  So in my case, it'd be:

/**
 * g_resources_register: (transfer full) (other.stuff) (more-stuff)
 *      (alias register)
 * @resources: a #GResources
 *
 * Some description.
 * Returns: stuff
 */

Seems OK.

(In reply to comment #28)
> Those are just attributes though - we're suggesting doing this for all new
> toplevel tags right?  So in my case, it'd be:
> 
> /**
>  * g_resources_register: (transfer full) (other.stuff) (more-stuff)
>  *      (alias register)
>  * @resources: a #GResources
>  *
>  * Some description.
>  * Returns: stuff
>  */
> 
> Seems OK.

No, you forgot the closing colon after (alias register). g-i's
annotationparser would have produced a warning for that...

And yes, allow linebreaks between annotations, so this
would also be valid:

/**
 * g_resources_register:
 *     (transfer full)
 *     (other.stuff)
 *     (more-stuff)
 *     (alias register):
 * @resources: a #GResources
 *
 * Some description.
 * Returns: stuff
 */

Once we get that capability, allowing the same for parameters
and tags would be trivial.

From the gobject-introspection side, the old scanner used to
silently ignore annotations not on the first line, current
code emits a warning (see http://git.gnome.org/browse/gobject-introspection/tree/giscanner/annotationparser.py#n884),
so at least we already know how to detect these.

(In reply to comment #29)
> No, you forgot the closing colon after (alias register). g-i's
> annotationparser would have produced a warning for that...

Apologies for the above incorrect statement Colin, got
carried away a bit...

As described in attachment #214195 [details], there is no requirement
for a colon after the annotations field of an identifier
part. There is however a requirement for a colon after the
annotations field of a parameter or tag part when said
annotations field is followed by a description. Oops :/

So my proposal in comment #26 would lead to ambiguity in the
case of multi-line annotations on identifier parts. For
example, in the following nonsense GTK-Doc comment block,
the "(TODO: write docs)" is clearly supposed and does get
parsed as the comment block description. Even though it
is missing the empty line which should precede the comment
block description, which is a pretty common error.

In short, the previous proposal would erroneously result
in the below being parsed as an annotation instead!

/**
 * g_resources_register:
 * (TODO: write docs...)
 */

So to prevent ambiguity, another idea might be to allow
for newline characters to be escaped by a backslash (or
some other character) within the annotations field of
identifier, parameter and tag parts causing the newline
to be ignored altogether while parsing. Doing so would
allow all of the following:

/**
 * g_resources_register: (transfer full) (other.stuff) (more-stuff) \
 *      (alias register)
 * @resources: a #GResources
 *
 * Some description.
 * Returns: stuff
 */
-> Parses (transfer full), (other.stuff), (more-stuff) and
   (alias register) as g-i annotations

/**
 * g_resources_register: \
 *     (transfer full) \
 *     (other.stuff) \
 *     (more-stuff) \
 *     (alias register)
 * @resources: a #GResources
 *
 * Some description.
 * Returns: stuff
 */
-> Parses (transfer full), (other.stuff), (more-stuff) and
   (alias register) as g-i annotations

/**
 * g_resources_register:
 * (TODO: write docs)
 */
-> Parses (TODO: write docs) as comment block description

/**
 * g_resources_register: \
 *     (attributes \
 *         key1=value1 \
 *         key2 \
 *         key3=value2) \
 *     (transfer full)
 *
 * This is an object used to test annotations.
 */
-> Parses (attributes key1=value1 key2 key3=value2) and
   (transfer full) as g-i annotations

/**
 * annotation_array_length:
 * @list: a #GtkListStore
 * @n_columns: number of columns
 * @types: \
 *    (array length=size) \
 *    (element-type guint8): list of types
 *
 * Document parameter relation for array length.
 */
-> Parses (array length=size) and (element-type guint8)
   as g-i annotations for the @types parameter

I guess some people are naturally going to expect to
also be able to insert an escaped newline
 *    (element-type guint8): list of types
                            ^ here

which is technically already parameter description
territory. So taking this further to also allow an
escaped newline before or after all colon field
separators in addition to within annotation fields
then allows the above to be rewritten as:

/**
 * annotation_array_length:
 * @list: a #GtkListStore
 * @n_columns: \
 *     number of columns
 * @types: \
 *     (array length=size) \
 *     (element-type guint8): \
 *     list of types
 *
 * Document parameter relation for array length.
 */

As usual, the amount of whitespace before the
backslash should not matter. So the above would be
the equivalent of the following kind off extreme
example:

/**
 * annotation_array_length:
 * @list:                     \
 *     a #GtkListStore
 * @n_columns:                \
 *     number of columns
 * @types:                    \
 *     (array length=size)    \
 *     (element-type guint8): \
 *     list of types
 *
 * Document parameter relation for array length.
 */

Which both would be the equivalent of:

/**
 * annotation_array_length:
 * @list: a #GtkListStore
 * @n_columns: number of columns
 * @types: (array length=size) (element-type guint8): list of types
 *
 * Document parameter relation for array length.
 */

Well, I guess you catch my drift. Thoughts?

(In reply to comment #30)
> I guess some people are naturally going to expect to
> also be able to insert an escaped newline...

I must say I expect NOT having to use any escaped newlines in C comments. Ever.

Furthermore, anyone caught doing this

/**
 * g_resources_register:
 * (TODO: write docs...)
 */

should be hanged at dawn on the nearest plane tree.

So, how much real trouble would be to start requiring colons, i.e. fix the syntax to a sane one instead of inventing complex and ugly workarounds?

This is the old problem of gtk-doc again: accepting all kinds of free-form rubbish and then, using complex heuristics, trying to guess what it was supposed to mean.  This way lies madness.

(In reply to comment #31)
>
> I must say I expect NOT having to use any escaped newlines in C comments. Ever.

It is a bit ugly.

> So, how much real trouble would be to start requiring colons, i.e. fix the
> syntax to a sane one instead of inventing complex and ugly workarounds?

Elaborate on "start requiring colons"?  Can you paste an example of your idea?

> This is the old problem of gtk-doc again: accepting all kinds of free-form
> rubbish and then, using complex heuristics, trying to guess what it was
> supposed to mean.  This way lies madness.

Yeah, well, that's what we're stuck with =)

(In reply to comment #31)
> So, how much real trouble would be to start requiring colons, i.e. fix the
> syntax to a sane one instead of inventing complex and ugly workarounds?

Throughout the format, the colon serves as a field separator,
but in your proposal it would double as a sort of "logical line
ends here" token which only has one valid location.

All of the sudden always requiring the annotations field of the
identifier part to be terminated with a colon is also incompatible
with for example the (skip) annotation on the identifier (introduced
in GTK-Doc 2010-04-13 [1]), breaking loads of annotations already in
stable software like GLib and GTK+:

/**
  * NonIntrospectableThing: (skip) < would be broken...

/**
  * NonIntrospectableThing: (skip): < ...by this new requirement

Going further and only requiring it to terminate multi-line annotations
on the identifier would make things even less consistent for GTK-Doc users.

> using complex heuristics, trying to guess what it was
> supposed to mean.  This way lies madness.

The new meaning of the colon proposed would need even more complex
heuristics than a logical/physical line solution as proposed in
comment #30 imho, which already has a well know meaning (C has ";",
Python has "\", M4 also has "\" as you can see in just about any
configure.ac and Makefile.am file, etc.

I agree it might be unexpected for C comment blocks but please
remember we're talking about GTK-Doc comment blocks here, where
due to various reasons conveying structure in a non-ambiguous and
backward compatible way does not leave much room for maneuver...

[1] http://git.gnome.org/browse/gtk-doc/commit/gtkdoc-mkdb.in?id=0a874b3a2d97476caf8100386ceed5070a2d065e

s/but in your proposal/but in the proposal from comment #26/

Sorry about that...

I think the '\' proposal in #comment 30 is not the end of the world. What speaks for this is that we can probably implement is with less heuristics in the code. I also hope that, such long lines are rather the exception.

For this example:
/**
 * g_resources_register:
 * (TODO: write docs...)
 */

we might be able to say that the gi annotation have to start behind the symbol and in this case we have not and thus would parse the next line as a comment. Thus the example would need to be:
/**
 * g_resources_register: (key)
 * (TODO: write docs...)
 */

In any case, there is a change that whatever syntax we invent would have been used before. Technically one could have been writing an example below before gi-annotation were introduced:
/**
 * g_resources_register:
 * @object: (TODO: yada yada): the object
 */

I'll do a bit of grepping in the sources to see if we can find an example where doing the wrapped lines without a '\' would break. If we can't find one, what if we just do it and kindly ask people to cleanup if it caused problems for them.

Where did we leave this?  I'd like to figure this out for real in the 3.10 cycle.

The multiline annotations thing is undecided for now, the rest
is nearing completion on my branch. Had to take a few weeks of
when my son was born but am back working on it since last week :)

As of yesterday g-ir-scanner parses the syntax from comment #6
and emits deprecation warnings for the g-i specific tags
mentioned in comment #4. Changing the title of this bug
report to reflect it's current status...

the multi-line annotations are pretty much required when dealing with new GTypeInstance types, which come with ref-func, unref-func, set-value-func, and get-value-func annotations by default. putting them all on one line makes the comment block look really bad (and hard to edit).

I think we have to go for the '\' syntax :/

> find . -name "*.c" -exec egrep -Hn '\ \*\ *\([a-zA-Z0-9]*:' {} \;
./ibus-1.5.3/src/ibusinputcontext.c:136:     * (Note: The text object is floating, and it will be released after the signal.
./gtk+-2/gtk/gtkwindow.c:6647: * (Note: this is a special-purpose function for the framebuffer port,
./gtk+-2/gtk/gtkwindow.c:6701: * (Note: this is a special-purpose function intended for the framebuffer
./gtk+-2/gtk/gtkwindow.c:7374: * (Note: this is a special-purpose function intended for the
./gtk+-2/gtk/gtkclist.c:6600: * (FIXME: focus the last focused child if visible)
./gtk+-2/gtk/gtkcolorsel.c:1109:       * (https://bugs.freedesktop.org/show_bug.cgi?id=2729)
./gtk+-2/gtk/gtkstyle.c:4810:       * (https://bugs.freedesktop.org/show_bug.cgi?id=2729)
./libxml2-2.8.0/catalog.c:1291:	     * (xml:base handling is automated)
./libxml2-2.8.0/testC14N.c:3: * (http://www.w3.org/TR/2001/REC-xml-c14n-20010315)

We should have picked {} for the annotations ...

/**
 * g_resources_register: {key}
 * @types:
 *     {array length=size}
 *     {element-type guint8}:
 *     list of types
 */

Could only find a few comments that use {} in my jhbuild checkout and thats not a gtkdoc comment.

> find . -name "*.h" -exec egrep -Hn '\ \*\ *\{[a-zA-Z0-9]*:' {} \;
./js17-17.0.0/js/src/vm/Debugger.h:130:     *     {return: value} or {throw: value} - Call unwrapDebuggeeValue to

Created attachment 299363 [details] [review]
giscanner: allow multiline annotations

(In reply to Stefan Sauer (gstreamer, gtkdoc dev) from comment #40)

Using terminology defined here https://git.gnome.org/browse/gobject-introspection/tree/giscanner/annotationparser.py#n25 to explain things.

I've gone through this list, here are my findings:

> > find . -name "*.c" -exec egrep -Hn '\ \*\ *\([a-zA-Z0-9]*:' {} \;
> ./ibus-1.5.3/src/ibusinputcontext.c:136:     * (Note: The text object is
> floating, and it will be released after the signal.
> ./gtk+-2/gtk/gtkwindow.c:6647: * (Note: this is a special-purpose function
> for the framebuffer port,
> ./gtk+-2/gtk/gtkwindow.c:6701: * (Note: this is a special-purpose function
> intended for the framebuffer
> ./gtk+-2/gtk/gtkwindow.c:7374: * (Note: this is a special-purpose function
> intended for the

g-ir-scanner correctly parses these as belonging to the 'comment block description' part.

> ./gtk+-2/gtk/gtkclist.c:6600: * (FIXME: focus the last focused child if
> visible)
> ./gtk+-2/gtk/gtkcolorsel.c:1109:       *
> (https://bugs.freedesktop.org/show_bug.cgi?id=2729)
> ./gtk+-2/gtk/gtkstyle.c:4810:       *
> (https://bugs.freedesktop.org/show_bug.cgi?id=2729)
> ./libxml2-2.8.0/catalog.c:1291:	     * (xml:base handling is automated)
> ./libxml2-2.8.0/testC14N.c:3: *
> (http://www.w3.org/TR/2001/REC-xml-c14n-20010315)

These are not even part of GTK-Doc comment blocks (/* instead of /**) and are
simply ignored by g-ir-scanner.

> I think we have to go for the '\' syntax :/

Well, maybe not... Seeing that there seems to be not a single valid case
here where the proposed syntax from comment # 26 actually breaks something
(and I have a feeling real world cases are going to be rather rare),
I ask you to reconsider that proposal and simply allow identifier`,
`parameter` and `tag` part `annotations` fields to span multiple lines :)

This is the gobject-introspection patch needed to implement just that.

Created attachment 299364 [details] [review]
giscanner: allow multiline annotations

Slightly improved, removes an unneeded dict.update()...

I'll try to come up with a gtkdoc patch that implement what is proposed in comment 26.

commit b73f5a36137a4a4b759b9f037439f8a53210b5f9
Author: Stefan Sauer <ensonic@users.sf.net>
Date:   Fri Apr 17 15:45:26 2015 +0200

    mkdb: add support for multiline symbol annotations
    
    We support the syntax suggested in
    https://bugzilla.gnome.org/show_bug.cgi?id=676133#c26
    Fixes #676133

commit cf80ebf431402624373a4f237f170ac9e31614e9
Author: Stefan Sauer <ensonic@users.sf.net>
Date:   Fri Apr 17 20:38:23 2015 +0200

    docs: Add a section for the gobject annotations.

Review of attachment 299364 [details] [review]:

This looks roughly reasonable to me.   Just so I'm clear on the state, we decided we don't need to use backslashes?

Yes, no backslashes.

(In reply to Colin Walters from comment #47)
> Review of attachment 299364 [details] [review] [review]:
> 
> This looks roughly reasonable to me.   Just so I'm clear on the state, we
> decided we don't need to use backslashes?

Correct, no need for backslashes.

Colin: thanks for the review!
Stefan: thanks for taking care of the GTK-Doc side!

Pushed as 90f69635057171b45cccb785fc1c53ccc992e0c3 and fixed.