After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 307646 - patch adds rss1.1 feed to tinderbox output
patch adds rss1.1 feed to tinderbox output
Status: RESOLVED OBSOLETE
Product: jhbuild
Classification: Infrastructure
Component: general
unspecified
Other Linux
: Normal enhancement
: ---
Assigned To: James Henstridge
Jhbuild QA
Depends on:
Blocks:
 
 
Reported: 2005-06-14 16:52 UTC by Brian Skahan
Modified: 2008-10-18 10:23 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
patch for rss1.1 (8.76 KB, patch)
2005-06-14 16:54 UTC, Brian Skahan
none Details | Review
Similar as above, but with atom support (7.61 KB, patch)
2005-08-05 10:34 UTC, Luis Villa
none Details | Review
new atom patch, still same basic functionality. (7.71 KB, patch)
2005-08-05 10:53 UTC, Luis Villa
none Details | Review
latest take, basically non-suck in conception, though still ugly in practice (8.29 KB, patch)
2005-08-05 15:33 UTC, Luis Villa
none Details | Review
much less ugly, messages now actually semi-informative (8.33 KB, patch)
2005-08-05 16:37 UTC, Luis Villa
none Details | Review
now with actually valid xml! (9.16 KB, patch)
2005-08-06 20:41 UTC, Luis Villa
rejected Details | Review
Cheeseball XML sketch. (380 bytes, text/plain;charset=UTF-8)
2005-08-17 02:10 UTC, Luis Villa
  Details

Description Brian Skahan 2005-06-14 16:52:24 UTC
Attached patch adds an rss1.1 feed to the output directory, adds a commandline
option -u/--url for link to the build logs on the web.
Comment 1 Brian Skahan 2005-06-14 16:54:00 UTC
Created attachment 47762 [details] [review]
patch for rss1.1
Comment 2 Luis Villa 2005-06-19 18:46:39 UTC
Brian- this is a pretty cool idea, I'm surprised I didn't think of it earlier.
thanks for whipping up the patch. Some comments:

With my GNOME tinderbox hat on, I'm wondering if it makes sense to only report
failures in the rss (and maybe the final build status, whether it succeeds or
fails)? I'm building the full gnome tinderbox 4-5 times a day, which means (on a
good day) 500-600 successes- more than I care to cope with in a feed, and which
frankly are mostly uninteresting.

[Reading the patch more]
Ah-ha. So, you're offering a feed entry per build cycle. I guess I'd imagined it
as a feed entry per module. AFAICT, this means that the feed is invalid while
the build is going on- depending on how the tinderbox output is being published
this might be problematic. I'm not sure how to fix that, though, if you want to
stick to the entry per-complete-cycle model, though.

Anyway, thanks a lot for whipping this up- I'll have to see how I can integrate
this into the GNOME tinderbox, once it is on a public machine again.
Comment 3 James Henstridge 2005-06-20 03:16:32 UTC
Is there any particular reason you chose RSS 1.1 over one of the better
supported syndication formats?

http://inamidst.com/rss1.1/ indicates that this format is a draft, so is it
could change before it comes final.
Comment 4 Luis Villa 2005-08-05 10:34:50 UTC
Created attachment 50264 [details] [review]
Similar as above, but with atom support

This is basically the same as the previous patch, but using not quite compliant
atom 1.0. It is missing the <id></id> fields- see:
http://diveintomark.org/archives/2004/05/28/howto-atom-id 

It also removes the per-phase data of the first cut.

That said, this is just for reference; because I still think per-module is
overkill, I'm going to whip up another version that will emit an event on the
start of the build, and on the finish; the start will link to the top-level
index for that build, and the finish will give pass/fail, and if failed, links
to each module that failed.
Comment 5 Luis Villa 2005-08-05 10:39:35 UTC
Hrm. BTW, Brian, the '--url' command you added to jhbuild/commands/tinderbox.py
doesn't seem to work, and I'm not sure why. Poking at that now.
Comment 6 Luis Villa 2005-08-05 10:53:39 UTC
Created attachment 50266 [details] [review]
new atom patch, still same basic functionality.

Found it; missing a 'u:' on line 29 of commands/tinderbox.py. 

This matters because I believe the optimal tinderbox setup is something like
URL/TIMESTAMP/<stuff>, so links should be to the timestamped directory. I'll
probably have to trim the timestamp off for one of the URLs, though.
Comment 7 Luis Villa 2005-08-05 15:32:22 UTC
Hrm. Both of these approaches approach uselessness, because as soon as a build
is finished we nuke the rdf and start all over. So the odds of a newreader
seeing the last few items in the build are slim; the odds of a feed that *only*
has start and finish being seen are even lower. And in the original approach,
there is basically never a valid file to be found, because the file is deleted
as soon as the closing tag is applied.

The patch I'm about to upload takes a different approach:

* if there is no existing feed, write out a new one with bogus entries for a
build launch and a build finish.
* when the build starts, replace the fake build launch entry with a real one
with legit times, data, etc. roughly the same when the build finishes.

So now we're done with a build, and we have a valid and basically informative
feed, that was valid and basically informative all the way through.

At this point, my tinderbox wrapper scripts:
* move the current output directory from ONGOING to LATEST
* copy the feed from LATEST to ONGOING (so we have feed continuity)
* restart the build

So we have a continous feed.

Note that the version I'm about to upload is ugly, but it should serve as proof
of concept. I'll have a better version up in an hour, unless I collapse into a
deep sleep.
Comment 8 Luis Villa 2005-08-05 15:33:56 UTC
Created attachment 50283 [details] [review]
latest take, basically non-suck in conception, though still ugly in practice
Comment 9 Luis Villa 2005-08-05 16:37:29 UTC
Created attachment 50286 [details] [review]
much less ugly, messages now actually semi-informative

This is as far as it is going to get today. Would appreciate you taking a look
and reviewing, James. My intent is that (coupled with the changes I made to
simplify the microtinder bits) I can use this to set up a planet to mimic
having a real, centralized tinder setup. Much easier in the short term than
buildbot support...
Comment 10 Luis Villa 2005-08-06 20:41:30 UTC
Created attachment 50328 [details] [review]
now with actually valid xml!
Comment 11 James Henstridge 2005-08-15 07:02:02 UTC
This seems like the wrong way to produce a feed.

I think the best way to handle it would be to make "jhbuild tinderbox" produce a
machine readable version of the index.html file, and then have a separate
program that could parse a number of these files to produce a feed.

This would also allow for other kinds of summaries, such as the timeline display
Mozilla's tinderbox software produces.
Comment 12 Luis Villa 2005-08-15 12:32:00 UTC
Hrm. You're right, of course. And while we're at it we could rip out the direct
HTML export too, and just replace that with a transform on the same
machine-parseable output.

It sounds like the right thing here is XML, but I know exactly squat about XSLT,
which is the obvious thing to do the transforms, it seems like, so... I might
get around to a patch which generates an XML representation of the code soon,
but actually replacing my atom patch and generating HTML from it might be a ways
off. :) Alternate suggestions for format/transform welcome.
Comment 13 Luis Villa 2005-08-15 12:47:33 UTC
Having read a little on XSLT, this should be easy. I won't really have time
until maybe Thursday to *do* this (exam wednesday) but if it seems like the
right approach to you, James, I'll go ahead and take a whack at converting all
the existing data output into XML/XSLT, and maybe add a couple too.
Comment 14 Luis Villa 2005-08-17 02:10:06 UTC
Created attachment 50822 [details]
Cheeseball XML sketch.
Comment 15 James Henstridge 2005-08-17 04:45:58 UTC
Comment on attachment 50822 [details]
Cheeseball XML sketch. 

Changing content type to text/plain because the file isn't valid XML, causing
mozilla to complain and not display it
Comment 16 James Henstridge 2005-08-17 04:54:47 UTC
Some thoughts:

* should include a unique ID for the build.  This'd be useful for generating an
atom feed from a sequence of builds, since each entry needs a unique ID.

* Be explicit about the time zone of the timestamps.  Should probably record
times in UTC (and include a "Z" on the end to indicate this).  This is important
if you want to provide a combined summary from a number of tinderboxes that are
in different time zones.

* links to the build logs

* It might make sense to record both start and end times for build phases rather
than the start time for each module and the build.  We can get the module start
time by taking the minimum of the start times of its phases and similar for the
end time.

* We might want to allow for longer description of the build status.  At the
same time, we want to know if that description means "success" or "failure".
Comment 17 James Henstridge 2005-08-17 05:08:20 UTC
One reason for recording both start and end times I didn't mention before: if
jhbuild gets support for parallel builds, you can't infer the end time from the
next module's start time.

Also, if download and compile are asynchronous, then there could be a gap
between the download and the build, so you probably want to know how long each
phase took.
Comment 18 Luis Villa 2005-08-18 01:24:35 UTC
As far as links to the build logs go, I was thinking that that was just
something that should be generated from the xml. The rest of the suggestions
make sense to me.
Comment 19 James Henstridge 2005-08-18 05:17:53 UTC
Well, the filename used for the build log is not always equal to the module ID
(e.g. if the module ID contains a '/' character).  Seems better to be explicit here.

It'd only be the relative pathname from the XML file anyway, so would not cause
any problem with moving the build logs around.
Comment 20 James Henstridge 2005-08-23 00:56:04 UTC
Don't know if this interests you, but the feeds you are generating with
MicroTinder are broken in a number of ways:

http://www.feedvalidator.org/check.cgi?url=http://trs80.ucc.asn.au/gnometinderbox/mudhead-LATEST/atom.xml

A few points:
 * <link> element in <feed> should have URL as attribute rather than content
 * no IDs on feed or entries
 * dates badly formed (should use "T" between date/time, must specify time
   zone).
 * content should either be escaped HTML or use the XHTML namespace.  Should
   specify content type
 * missing feed-level <updated> element
Comment 21 Frederic Peters 2008-10-18 10:23:22 UTC
There is now complete buildbot support in jhbuild, it does things properly and provides RSS and Atom feeds.