After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 552532 - Please set up a nightly autobuild/publish for gnome-user-docs and gnome-devel-docs
Please set up a nightly autobuild/publish for gnome-user-docs and gnome-devel...
Status: RESOLVED FIXED
Product: website
Classification: Infrastructure
Component: help.gnome.org
current
Other All
: High critical
: ---
Assigned To: GNOME Web maintainers
GNOME Web maintainers
Depends on:
Blocks:
 
 
Reported: 2008-09-16 18:31 UTC by Willie Walker
Modified: 2008-10-25 12:18 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Willie Walker 2008-09-16 18:31:40 UTC
In private e-mail, Frederic Peters mentioned be might be able set up an autobuild system for the gnome-user-docs and gnome-devel-docs modules.

If it helps, my main goal here would be to have a URL we could point people to specifically for the accessibility documentation.  There are three main documents of interest: the users guide, the developers guide, the distribution testing guide.  Vincent Alexander has been working hard on these and currently hosts them at http://snoringbeagle.net/.  This is a temporary solution, however, and we'd like a URL that will remain consistent and available.

For example, just as we have a http://library.gnome.org/users/gnome-access-guide/stable for the users guide, the nightly parallel might be http://library.gnome.org/users/gnome-access-guide/nightly or http://library.gnome.org/users/gnome-access-guide/latest (though the stuff currently at "latest" seems to be the oldest ;-)).
Comment 1 Frederic Peters 2008-09-16 20:38:53 UTC
Writing down how I'd lay things out; and marking as gnome-love to bring new people in.

 - Extend jhbuild so it is possible to create tarballs with "nightly" as version number; for example adding a "makeargs" command line parameter: jhbuild buildone -f --dist --makeargs "VERSION=nightly" gnome-user-docs.
 
 - Extend library.gnome.org (the svn module is named library-web) to look up "somewhere" for the tarball, for every modules it knows about; then process it and recognize its "nightly" version number; not doing any special symlink (stable/devel) trick in that case.
Comment 2 Olav Vitters 2008-09-16 21:57:33 UTC
Removing gnome-love, I don't think this is a gnome-love task.

There are already SVN hooks for both modules and they do install something. Likely somewhere within developer.gnome.org (no time to properly investigate).

Frederic: see ~/hooks under gnomeweb user on window. Hook should:
1. Generate tarball
2. Inform library-web (like already done for release-notes)
Comment 3 Frederic Peters 2008-09-16 22:43:04 UTC
Olav: seems I wrote about my general plan for nightly tarballs (as it won't be possible to have 'make dist' run for every modules on window.gnome.org), not just about gnome-{user,devel}-docs.

For just those two modules using the existing hooks is certainly easier and faster.  Sorry it was not clear at all in my comment.
Comment 4 Frederic Peters 2008-09-26 15:43:39 UTC
Mmm, I wonder why I had this set as NEEDINFO; cancelling that.
Comment 5 Calum Benson 2008-09-26 16:11:03 UTC
Just to add my 2c worth...

For the HIG, I'd love nightly builds of the unstable version to be available (which currently lives in the hig-2-3 branch of gnome-devel-docs).; it's not really needed for the stable version, which is updated about once every four years :)  

Then I could kill the old rebuild-on-checkin version that still lives on developer.gnome.org.

(Is it just me, or does it seem a bit back-to-front to have the stable version of the HIG on trunk, and the unstable version in a branch?  This is the way Shaun imported it for us, so I assume it's the way the other docs work too.  Or maybe it just makes less sense for the HIG because it doesn't really track GNOME releases in any meaningful way.)
Comment 6 Calum Benson 2008-09-30 13:43:00 UTC
Also, the unstable version of the HIG on developer.gnome.org has always had some extra stylesheet magic applied so that the contents of <remark> tags are displayed.  Would be handy if any nightly/unstable builds being hosted on library.gnome.org could do the same.
Comment 7 Murray Cumming 2008-10-01 07:37:21 UTC
Note to self: This is a blocker to removing the HIG from the (must die eventually) developer.gnome.org site.
Comment 8 Frederic Peters 2008-10-13 09:55:58 UTC
Calum, you are right as this oddity (devel version in a branch) makes it difficult to create nightly tarballs in a generic way.

But not a real problem I'll go the generic way for gnome-user-docs and it will be hacky for gnome-devel-docs; could you point me to the extra stylesheet magic?
Comment 9 Frederic Peters 2008-10-13 12:24:03 UTC
2008-10-13  Frederic Peters  <fpeters@0d.be>

        * data/catalog.xml.in, data/xslt/indexes.xsl, src/config.py,
        src/defaults.lgorc, src/lgo.py: added support for publishing
        documentation from nightly tarballs.

We now have to get nightly tarballs in a directory of window.gnome.org.  Olav, I wrote /usr/local/www/gnomeweb/nightly-tarballs/create-nightlies.sh; is this ok to hook it up in the gnomeweb crontab?
Comment 10 Frederic Peters 2008-10-13 14:30:24 UTC
And it is now online, see http://library.gnome.org/devel/hig-book/ and http://library.gnome.org/devel/hig-book/nightly/.

Note there is currently the problem that the title is always taken from the latest version; also there is currently no xsl magic to get a different output for that version.
Comment 11 Frederic Peters 2008-10-13 17:14:38 UTC
2008-10-13  Frederic Peters  <fpeters@0d.be>

        * data/skin/lgo.css, data/xslt/db2html.xsl, src/lgo.py: display and
        style <remark> elements for nightly documents.

It is rebuilding now... (takes long as it rebuilds every gnome-doc-utils documents)
Comment 12 Olav Vitters 2008-10-13 17:34:45 UTC
The script does too much work.

1. Only run autogen etc if svn update results in a new revision.
2. Only set the buildflag if the tarballs have changed
Comment 13 Frederic Peters 2008-10-13 18:21:34 UTC
Olav: I updated it according to your comments.

(btw the rebuild is not yet over)
Comment 14 Frederic Peters 2008-10-14 06:23:00 UTC
Waking up and it's over; there is an example of <remark> at the end of this page: http://library.gnome.org/devel/hig-book/nightly/windows-primary.html.en (perhaps it would benefit some margin).
Comment 15 Murray Cumming 2008-10-14 09:23:35 UTC
Calum, thanks for removing the old unstable HIG stuff from developer.gnome.org.

However, I think this needs to have "Nightly Build: Work In Progress" or suchlike on the title page.
Comment 16 Frederic Peters 2008-10-14 10:22:53 UTC
Do we want such an indication on every pages, or just the title page?  It could for example be added at the top of the sidebar.  Any suggestion?
Comment 17 Frederic Peters 2008-10-14 15:35:18 UTC
Well; I added it to every pages, at the top of the sidebar.  As it is a change to the docbook xsl it is rebuilding tons of documents, just like yesterday.
Comment 18 Frederic Peters 2008-10-15 08:17:06 UTC
Olav; I updated the script to use `svnversion'.  I believe it is now ready.

Calum, Shaun, Willie, Murray: is this ok as is ?
Comment 19 Murray Cumming 2008-10-15 09:07:30 UTC
It looks great to me. Many thanks.
Comment 20 Willie Walker 2008-10-15 12:48:12 UTC
Thanks so much for your work!  I love that we can get nightly builds now.  Two things that I noticed, though:

1) http://library.gnome.org/users/ seems to expose the svn module name whereas it used to show a more human consumable string.

2) http://library.gnome.org/devel/guides -> Accessibility for Developers points to http://developer.gnome.org/projects/gap/guide/gad/, which is pretty old.  It would be great if it could instead point to a build/push of http://svn.gnome.org/svn/gnome-devel-docs/trunk/gadg/.

Thanks again!
Comment 21 Frederic Peters 2008-10-15 13:04:38 UTC
1) Fixed.

2) Shouldn't gnome-devel-docs/gadg/ be updated to match other gnome-doc-utils documents?  It will automatically get on library.gnome.org that way.
Comment 22 Willie Walker 2008-10-15 13:10:52 UTC
(In reply to comment #21)
> 1) Fixed.

Woohoo!  I wait for the next push to check it out.

> 2) Shouldn't gnome-devel-docs/gadg/ be updated to match other gnome-doc-utils
> documents?  It will automatically get on library.gnome.org that way.

Sounds good to me.  What's needed and does Vincent Alexander (the maintainer of gadg) need to do anything?
Comment 23 Shaun McCance 2008-10-15 15:33:09 UTC
Wow, lots of activity here.  Awesome.  Long comment addressing some points:

* Calum, the reason the unstable HIG is in a branch is that it doesn't track Gnome release cycles in any way, and it would be a serious headache to try to manage the not-yet-ready-for-release HIG stuff and the ready-for-release stuff from all the other documents if they're in the same branch.  There's nothing special about branches that makes them only useful for stabe series.  I do branches for experimental development work all the time.

* That said, having the hig-2-3 branch be only the HIG, rather than the whole module, was probably a mistake on my part.  It makes it hard for scripts that expect to see whole modules actually track it.  I can't, for instance, track it with Pulse without considerable work on my part.

* I'm seeing much talk about 'make dist' and autogen.  Are these things really necessary?  I know that for quite a few documents, it's necessary to actually run make to get the files, but I don't think this is the case for any of these documents.  The only reason you might run make is to get the translated documents, but I don't think those are strictly necessary for nightly builds.

* I laid out a proposal for documentation status tracking here:
http://mail.gnome.org/archives/gnome-doc-list/2008-September/msg00075.html
It would be super awesome if the status information could be added to the page for the nightlies.  See for instance the release info here:
http://www.gnome.org/~shaunm/pulse/web/doc/svn.gnome.org/gnome-devel-docs/gdp-style-guide/trunk
I'm willing to do the work for this.  I'd probably just stick it in the nightly build caution box, or right under it.

* Would it be possible to have a page like library.gnome.org/nightly that lists all the nightly-built documents?  The only to find them now is to go through all the documents looking for those with nightly builds.

* Frederic, you said "there is currently the problem that the title is always taken from the latest version".  Does that mean the title on the landing page for a given document is taken from the latest build of the document?  So changing te title in the nightlies would change the title on the landing page?

* Regarding "remark" elements, Calum, I've always wanted to make better use of them.  I'll gladly add support for them in gnome-doc-utils if people are actually using them.  They'd be turned on with a parameter (something like db2html.show_remarks).  Then library's db2html.xsl could set this parameter to the value of $libgo.nightly.  Frederic, if I add stuff to g-d-u for library, how quickly does library get the changes?  Are g-d-u upgrades are manual?
Comment 24 Frederic Peters 2008-10-15 15:44:35 UTC
(In reply to comment #23)

> * I'm seeing much talk about 'make dist' and autogen.  Are these things really
> necessary?  I know that for quite a few documents, it's necessary to actually
> run make to get the files, but I don't think this is the case for any of these
> documents.  The only reason you might run make is to get the translated
> documents, but I don't think those are strictly necessary for nightly builds.

That's it, the sole reason is to get translated documents.  I figured translators would also find useful to get their work-in-progress translation available for easy review.


> * I laid out a proposal for documentation status tracking here:
> http://mail.gnome.org/archives/gnome-doc-list/2008-September/msg00075.html
> It would be super awesome if the status information could be added to the page
> for the nightlies.  See for instance the release info here:
> http://www.gnome.org/~shaunm/pulse/web/doc/svn.gnome.org/gnome-devel-docs/gdp-style-guide/trunk
> I'm willing to do the work for this.  I'd probably just stick it in the nightly
> build caution box, or right under it.

Where does Pulse gets this information?  Certainly library.gnome.org could ask Pulse when building documents but there may be a simpler way.


> * Would it be possible to have a page like library.gnome.org/nightly that lists
> all the nightly-built documents?  The only to find them now is to go through
> all the documents looking for those with nightly builds.

Not a problem.


> * Frederic, you said "there is currently the problem that the title is always
> taken from the latest version".  Does that mean the title on the landing page
> for a given document is taken from the latest build of the document?  So
> changing te title in the nightlies would change the title on the landing page?

Yes, that's the problem; I tried to fix it and got all titles removed (that was the first problem pointed by Willie); I'll nail it soon.


> the value of $libgo.nightly.  Frederic, if I add stuff to g-d-u for library,
> how quickly does library get the changes?  Are g-d-u upgrades are manual?

They must be packaged and installed by Olav (or any other admin, I guess) on window.gnome.org.
Comment 25 Frederic Peters 2008-10-15 15:50:48 UTC
I added bug 556425 and bug 556426 to track some issues.
Comment 26 Frederic Peters 2008-10-16 07:38:34 UTC
> Yes, that's the problem; I tried to fix it and got all titles removed (that was
> the first problem pointed by Willie); I'll nail it soon.

That has also been fixed.
Comment 27 Olav Vitters 2008-10-16 08:25:29 UTC
Regarding new g-d-u on window. Just ping me that the machine should have a new version. Can be installed whenever I have time and as often as needed.
Comment 28 Frederic Peters 2008-10-18 09:36:15 UTC
Olav: I added the entry to gnomeweb crontab, and asked it to mail me the output.
Comment 29 Willie Walker 2008-10-25 11:55:38 UTC
(In reply to comment #20)
> 2) http://library.gnome.org/devel/guides -> Accessibility for Developers points
> to http://developer.gnome.org/projects/gap/guide/gad/, which is pretty old.  It
> would be great if it could instead point to a build/push of
> http://svn.gnome.org/svn/gnome-devel-docs/trunk/gadg/.

Sorry to be a pain, but we still have the old docs up there.  I recall some e-mail flying by somewhere that suggested renaming gadg to a more wordy form, but I'm not sure what the status of that is.

Vincent have you renamed the developer doc files to the more wordy form?  If so, can we get a build set up for it?
Comment 30 Frederic Peters 2008-10-25 12:18:43 UTC
Willie: this is tracked in bug 556500.