GNOME Bugzilla – Bug 307646
patch adds rss1.1 feed to tinderbox output
Last modified: 2008-10-18 10:23:22 UTC
Attached patch adds an rss1.1 feed to the output directory, adds a commandline option -u/--url for link to the build logs on the web.
Created attachment 47762 [details] [review] patch for rss1.1
Brian- this is a pretty cool idea, I'm surprised I didn't think of it earlier. thanks for whipping up the patch. Some comments: With my GNOME tinderbox hat on, I'm wondering if it makes sense to only report failures in the rss (and maybe the final build status, whether it succeeds or fails)? I'm building the full gnome tinderbox 4-5 times a day, which means (on a good day) 500-600 successes- more than I care to cope with in a feed, and which frankly are mostly uninteresting. [Reading the patch more] Ah-ha. So, you're offering a feed entry per build cycle. I guess I'd imagined it as a feed entry per module. AFAICT, this means that the feed is invalid while the build is going on- depending on how the tinderbox output is being published this might be problematic. I'm not sure how to fix that, though, if you want to stick to the entry per-complete-cycle model, though. Anyway, thanks a lot for whipping this up- I'll have to see how I can integrate this into the GNOME tinderbox, once it is on a public machine again.
Is there any particular reason you chose RSS 1.1 over one of the better supported syndication formats? http://inamidst.com/rss1.1/ indicates that this format is a draft, so is it could change before it comes final.
Created attachment 50264 [details] [review] Similar as above, but with atom support This is basically the same as the previous patch, but using not quite compliant atom 1.0. It is missing the <id></id> fields- see: http://diveintomark.org/archives/2004/05/28/howto-atom-id It also removes the per-phase data of the first cut. That said, this is just for reference; because I still think per-module is overkill, I'm going to whip up another version that will emit an event on the start of the build, and on the finish; the start will link to the top-level index for that build, and the finish will give pass/fail, and if failed, links to each module that failed.
Hrm. BTW, Brian, the '--url' command you added to jhbuild/commands/tinderbox.py doesn't seem to work, and I'm not sure why. Poking at that now.
Created attachment 50266 [details] [review] new atom patch, still same basic functionality. Found it; missing a 'u:' on line 29 of commands/tinderbox.py. This matters because I believe the optimal tinderbox setup is something like URL/TIMESTAMP/<stuff>, so links should be to the timestamped directory. I'll probably have to trim the timestamp off for one of the URLs, though.
Hrm. Both of these approaches approach uselessness, because as soon as a build is finished we nuke the rdf and start all over. So the odds of a newreader seeing the last few items in the build are slim; the odds of a feed that *only* has start and finish being seen are even lower. And in the original approach, there is basically never a valid file to be found, because the file is deleted as soon as the closing tag is applied. The patch I'm about to upload takes a different approach: * if there is no existing feed, write out a new one with bogus entries for a build launch and a build finish. * when the build starts, replace the fake build launch entry with a real one with legit times, data, etc. roughly the same when the build finishes. So now we're done with a build, and we have a valid and basically informative feed, that was valid and basically informative all the way through. At this point, my tinderbox wrapper scripts: * move the current output directory from ONGOING to LATEST * copy the feed from LATEST to ONGOING (so we have feed continuity) * restart the build So we have a continous feed. Note that the version I'm about to upload is ugly, but it should serve as proof of concept. I'll have a better version up in an hour, unless I collapse into a deep sleep.
Created attachment 50283 [details] [review] latest take, basically non-suck in conception, though still ugly in practice
Created attachment 50286 [details] [review] much less ugly, messages now actually semi-informative This is as far as it is going to get today. Would appreciate you taking a look and reviewing, James. My intent is that (coupled with the changes I made to simplify the microtinder bits) I can use this to set up a planet to mimic having a real, centralized tinder setup. Much easier in the short term than buildbot support...
Created attachment 50328 [details] [review] now with actually valid xml!
This seems like the wrong way to produce a feed. I think the best way to handle it would be to make "jhbuild tinderbox" produce a machine readable version of the index.html file, and then have a separate program that could parse a number of these files to produce a feed. This would also allow for other kinds of summaries, such as the timeline display Mozilla's tinderbox software produces.
Hrm. You're right, of course. And while we're at it we could rip out the direct HTML export too, and just replace that with a transform on the same machine-parseable output. It sounds like the right thing here is XML, but I know exactly squat about XSLT, which is the obvious thing to do the transforms, it seems like, so... I might get around to a patch which generates an XML representation of the code soon, but actually replacing my atom patch and generating HTML from it might be a ways off. :) Alternate suggestions for format/transform welcome.
Having read a little on XSLT, this should be easy. I won't really have time until maybe Thursday to *do* this (exam wednesday) but if it seems like the right approach to you, James, I'll go ahead and take a whack at converting all the existing data output into XML/XSLT, and maybe add a couple too.
Created attachment 50822 [details] Cheeseball XML sketch.
Comment on attachment 50822 [details] Cheeseball XML sketch. Changing content type to text/plain because the file isn't valid XML, causing mozilla to complain and not display it
Some thoughts: * should include a unique ID for the build. This'd be useful for generating an atom feed from a sequence of builds, since each entry needs a unique ID. * Be explicit about the time zone of the timestamps. Should probably record times in UTC (and include a "Z" on the end to indicate this). This is important if you want to provide a combined summary from a number of tinderboxes that are in different time zones. * links to the build logs * It might make sense to record both start and end times for build phases rather than the start time for each module and the build. We can get the module start time by taking the minimum of the start times of its phases and similar for the end time. * We might want to allow for longer description of the build status. At the same time, we want to know if that description means "success" or "failure".
One reason for recording both start and end times I didn't mention before: if jhbuild gets support for parallel builds, you can't infer the end time from the next module's start time. Also, if download and compile are asynchronous, then there could be a gap between the download and the build, so you probably want to know how long each phase took.
As far as links to the build logs go, I was thinking that that was just something that should be generated from the xml. The rest of the suggestions make sense to me.
Well, the filename used for the build log is not always equal to the module ID (e.g. if the module ID contains a '/' character). Seems better to be explicit here. It'd only be the relative pathname from the XML file anyway, so would not cause any problem with moving the build logs around.
Don't know if this interests you, but the feeds you are generating with MicroTinder are broken in a number of ways: http://www.feedvalidator.org/check.cgi?url=http://trs80.ucc.asn.au/gnometinderbox/mudhead-LATEST/atom.xml A few points: * <link> element in <feed> should have URL as attribute rather than content * no IDs on feed or entries * dates badly formed (should use "T" between date/time, must specify time zone). * content should either be escaped HTML or use the XHTML namespace. Should specify content type * missing feed-level <updated> element
There is now complete buildbot support in jhbuild, it does things properly and provides RSS and Atom feeds.