GNOME Bugzilla – Bug 512827
WebKit support (or variable Yelp's backend)?
Last modified: 2010-05-10 21:33:36 UTC
Don't you plan to implement support for WebKit? It would be great!
Would love to. Don't suppose you know WebKit APIs enough / know someone who does to implement it? Yelp backend is sufficiently abstracted to allow different backends, it just needs someone to do the work.
Unfortunately, I'm not a developer. :'(
Created attachment 104328 [details] [review] Bad, bad, bad Very, very first implementation. This breaks everything. Don't use is. Just as I'm asking for help and need a storage location.
> and need a storage location. I can provide you some space on the web. Are you interested in?
(In reply to comment #3) > Created an attachment (id=104328) [edit] > Bad, bad, bad > > Very, very first implementation. This breaks everything. Could you use "#if 0 ... #endif" blocks instead of wrapping all the removed lines with /* ... */ Improves readability a lot.. ;)
By the way... Don't the GNOME developers plan to create something like "GNOME-default HTML-rendering engine"? (I'm not a developer.) The idea is: to have all applications with variable HTML-rendering backend. It requires some common library that would work with existing HTML-rendering engines and provide access to them to the applications. In one word: framework. In two words: abstract layer. :-) Advantages: only one library (application developers wouldn't need to implement it themselves), variable backend (user has the possibility of choice), setting on one place.
More info at http://donscorgie.livejournal.com/12716.html.
(In reply to comment #6) > By the way... Don't the GNOME developers plan to create something like > "GNOME-default HTML-rendering engine"? (I'm not a developer.) > > The idea is: to have all applications with variable HTML-rendering backend. It > requires some common library that would work with existing HTML-rendering > engines and provide access to them to the applications. In one word: framework. > In two words: abstract layer. :-) > > Advantages: only one library (application developers wouldn't need to implement > it themselves), variable backend (user has the possibility of choice), setting > on one place. > Hi Martin, I'm the one working on GTK+ Web integration. I was originally hopeful that we could create a kind of generic Web widget, but everything I've seen in the last few months points to abstraction layers severely restricting the capabilities of the content engine (the lowest-common denominator problem). If you try to support Mozilla, WebKit and GtkHTML with the same API, you end up with an un-integrated stack that has inconsistent behaviour. I've been employed to work on a browser abstraction in the past, and I've studied and worked with several others in GNOME. With respect to the authors of things like Yelper, EAL, dh_html, none of these abstractions are elegant enough to become part of a long-term supported browser SDK. At the moment it's looking like we may settle on an external module, eg.: glib-2.0, pango, atk, gtk+-2.0, webkit-1.0 This solves the problem of circular dependencies and gives the opportunity to maintain a more flexible release schedule for the Web component (which has a larger code-base than all the other modules combined.) We saw with Qt that embedding the browser into the toolkit directly causes difficulties and leads what's essentially a fork of the browser engine, something which the GTK+ browser team doesn't have resources to support. So I don't think we will advocate putting the browser directly into the GTK+ module. So I think using WebView as this patch does is a good direction. Whatever happens in Yelp, I recommend only supporting one content engine at any one time. Abstraction layers just aren't worth supporting in this kind of application.
martin, are you yourself working on getting this bug fixed? if not, please do NOT set it to assigned. setting a bug to assigned is the duty of the developers only. thanks.
Andre, Pavel set it to assigned, look at the activity log...
Martin: uhm, i'm sorry. yes, i meant Pavel and not you.
WebKit bug #17353 for the libxml2/xslt issue found by Yelp: http://bugs.webkit.org/show_bug.cgi?id=17353 Bug 17353: XMLTokenizer breaks applications (libxml2 global callbacks)
(In reply to comment #12) > WebKit bug #17353 for the libxml2/xslt issue found by Yelp: > > http://bugs.webkit.org/show_bug.cgi?id=17353 > Bug 17353: XMLTokenizer breaks applications (libxml2 global callbacks) > WebKit Bug #17353 was resolved in WebKit r30236.
Created attachment 109135 [details] [review] Bad, bad, bad 2 fixed yelp-main.c:349: warning: implicit declaration of function 'xsltInit' (added #include <libxslt/xslt.h> ) s/WebKitGtk/webkit-1.0/
Created attachment 109139 [details] [review] Bad, bad, bad 3 (In reply to comment #5) > Could you use "#if 0 ... #endif" blocks instead of wrapping all the removed > lines > with /* ... */ > > Improves readability a lot.. ;) done
Created attachment 109423 [details] [review] Bad, bad, bad 3.5 Diff against current trunk on svn.gnome.org BTW: Yelp developers, is it OK to create a "webkit" branch on svn.gnome.org applying this patch in order to simplify the hacking on this backend and to add a "yelp-webkit" module in jhbuild (we have yet "epiphany-webkit" and "devhelp-webkit")?
Re: comment 16 and patch: I tried this patch with yelp 2.22.1 together with rarian 0.8.0 and webkit r34279. Starting yelp from the command line takes a *long* time until the index opens, but that seems to be because of the man/info pages. Opening a "Help -> Contents" window is quick, but the window title never changes from "Loading...". Help documents open, but links within help documentation do not. Search works, but is slow (man/info again?); links from search results work. "Print This Page" works, but "Print This Document" crashes yelp.
Hi, (In reply to comment #17) > Re: comment 16 and patch: > I tried this patch with yelp 2.22.1 together with rarian 0.8.0 and webkit > r34279. > > Starting yelp from the command line takes a *long* time until the index opens, > but that seems to be because of the man/info pages. > > Opening a "Help -> Contents" window is quick, but the window title never > changes from "Loading...". Help documents open, but links within help > documentation do not. I have an updated patch on my system that fixes the window title stuff and re-adds the scrollbars. It also makes the backend configurable (gecko and webkit). I'll post it here once I've cleaned it up somewhat further and added a few extra things. The links within documents not working is a known issue that I've files with wekbkit [1]. > > Search works, but is slow (man/info again?); links from search results work. > > "Print This Page" works, but "Print This Document" crashes yelp. > Neither of the print options are properly hooked up, so although you'll get the print dialog, you'll never get any output. The "Print this Document" crashing this is a known bug, fixed in 2.23.1. Printing cannot be supported until webkit implements a printing API that doesn't involve javascript voodoo [2]. As for the speed issues, I'm not sure about. Do they go away with warm caches (i.e. the second time you try and run yelp)? Does it appear with a gecko-based yelp, or is it specific to webklit backend? [1] https://bugs.webkit.org/show_bug.cgi?id=19360 [2] https://bugs.webkit.org/show_bug.cgi?id=16493
> I have an updated patch on my system that fixes the window title > stuff and re-adds the scrollbars. It also makes the backend configurable > (gecko and webkit). I'll post it here once I've cleaned it up somewhat > further and added a few extra things. Look forward to it. Are you planning a branch in SVN for this, or is this a goal for 2.24? > The links within documents not working is a known issue that I've files > with wekbkit [1]. Good to know. > Printing cannot be supported until webkit implements a printing API > that doesn't involve javascript voodoo [2]. What about the workaround just mentioned there a few hours ago? > As for the speed issues, I'm not sure about. Do they go away with warm > caches (i.e. the second time you try and run yelp)? Does it appear with > a gecko-based yelp, or is it specific to webklit backend? I'll look into this further and get back later. I doubt it's backend-specific, as it's man(1) that's using the CPU cycles.
Created attachment 112576 [details] [review] Updated patch Updated patch. * Fixes window title * Adds scrollbars * Adds "find" support * Obeys font settings * hacky print support * Allows navigation within docbook documents It's almost ready to go in. There's a few things that are slightly suboptimal: * print preview makes the print dialog go away, which is annoying * Popup menus can't / aren't be customised * font colours aren't being obeyed correctly (high-contrast inverse themes are unusable) * No browsing caret * Jump to anchor-on-page doesn't work, due to another hack needed to allow in-docbook links to work As-is, I'm fairly happy to put in, if others would like to test and comment.
Re: comment 20 and attachment 112576 [details] [review] Thanks for the new patch. I built it with yelp trunk and the same versions of rarian and webkit. > * Fixes window title > * Adds scrollbars > * Adds "find" support Confirmed. > * hacky print support Even "Print This Document..." works. > * Allows navigation within docbook documents Confirmed. > * Jump to anchor-on-page doesn't work, due to another hack needed to > allow in-docbook links to work Confirmed. No news on the webkit bug? > As-is, I'm fairly happy to put in, if others would like to test and comment. I did note that images (e.g. screenshots) in pages did not display. Is that just me? Overall, I'd say to commit so that others will see it and help get it ready for 2.24. /me looking forward to a lizardless GNOME this fall :-)
Don, a couple of replies to your questions: The recommended way to invoke a basic print dialog right now is to invoke JavaScript "print();" as is done in Devhelp and Epiphany. This dialog includes a WYSIWYG print preview feature. https://bugs.webkit.org/show_bug.cgi?id=16493 Bug 16493: [GTK] Provide a public printing API webkit_web_view_execute_script (web_view, "print();"); ^ This isn't perfect, but I wouldn't say it's voodoo either. It does exactly what it says. As far as full printing API goes, we're waiting on feedback from developers on what this should look like. The first proposal was good for many users but I believe too limited for Evolution, which needs to override header and footer rendering in print content. There's not much point in introducing a webkit_web_view_print() with no parameters in the meantime since the JS technique is equivalent. On the subject of print preview, I believe GNOME and GTK+ are moving towards print preview in the print dialog since this is guaranteed to match the print output. The custom print preview capability in GtkMozEmbed was only necessary because Gecko used to implement a non-GtkPrint printing system that generated PS directly. Unless I'm mistaken, there's no need for WebKit to inherit this legacy since we support preview with GtkPrint. On the subject of navigation-requested, I haven't had time to investigate this yet, but my initial impression is that WebKit is doing the right thing. navigation-requested will _only_ fire if the loaded document is going to change. It's not a catch-all link click signal. So the problem boils down to the pathological case where our HTML has internal anchor links to documents that are actually external. I can find some time to figure out a solution if necessary, but if there's any possibility to preprocess and fix up the links with XSL before sending it to the WebView, I'd really recommend doing that rather than trying to support this awkward HTML after the fact. On the subject of merging to SVN, I strongly recommend branching the xulrunner Yelp where it can be maintained if necessary, and landing the WebKit patch without ifdefs on trunk. This is the strategy almost every other application has taken for WebKit migration, and is a huge maintainability win (no ifdefs or abstractions). I'm making this comment based on our real porting experience, not because I have a vested interest in a single-engine solution. Apart from the ifdefs, the patch looks good. At some point (hopefully for 2.24) we'll be exposing GIO accessors in the loader API which will tie in nicely with the recent GIO work in Yelp. +1 for merging without ifdefs, and with the "print();" technique.
I have now committed an initial patch to the "webkit" branch in SVN, removing gecko. This will be broken at the moment as I can't test until I update my webkit properly, so it will probably be broken. I'll attempt to fix this at some point soon. But if people want to try, it's at: http://svn.gnome.org/svn/yelp/branches/webkit
Created attachment 114267 [details] [review] YELP_DEFINES patch Re: comment 23 > http://svn.gnome.org/svn/yelp/branches/webkit Tested r3152 on Cygwin with webkit r35067. I did need a minor patch, attached. Results are similar to the last patch. Navigation between pages works, anchor-on-page still does not work, nor do local images (e.g. screenshots). Both print page and document work.
Don, I've a strange issue in my jhbuild sandbox: when I try to view a manual, the related XML file is loaded in gedit... ???
(In reply to comment #25) > Don, I've a strange issue in my jhbuild sandbox: when I try to view a manual, > the related XML file is loaded in gedit... > > ??? > This sounds like something the gio/soup WebKit backend might do. I'm doing testing using the default (curl) http backend which doesn't exhibit this issue -- can you try with that? The gio/soup backend is probably not ready for real world use just yet.
(In reply to comment #24) > Created an attachment (id=114267) [edit] > YELP_DEFINES patch > > Re: comment 23 > > http://svn.gnome.org/svn/yelp/branches/webkit > > Tested r3152 on Cygwin with webkit r35067. I did need a minor patch, attached. > > Results are similar to the last patch. Navigation between pages works, > anchor-on-page still does not work, nor do local images (e.g. screenshots). > Both print page and document work. > I did some cleanup on the yelp webkit branch -- distcheck passes now. The problem with navigation is down to a quirk in the way yelp generates HTML. yelp gives the WebView a chunk of HTML containing hyperlinks: In some cases the links point to anchors in the currently loaded document, and in other cases they point to another page that needs to be loaded. yelp produces fragment URIs for both of these hyperlink types, eg. <"a href="#section-in-current-page"> and <"a href="#another-page">. WebKit handles these fragment links internally (according to spec) so the 'clicks' never get to yelp. This technique hobbled along successfully with Gecko because yelp was listening for DOM click events, but it doesn't fly with WebKit which has a higher-level API that only emits genuine navigation requests while attempting to deal with fragment URIs internally. I have a patch that changes the logic around to make images and internal hyperlinks work, but it breaks links to other pages. I think the problem lies in db2html.xsl.in which indiscriminately converts all hyperlinks to fragment links: <xsl:template name="db.xref.target"> <xsl:param name="linkend"/> <xsl:value-of select="concat('#', $linkend)"/> </xsl:template> docbook XML has a concept of chunk links, so we can get a little further by generating the links as fragments or queries conditionally: <xsl:template name="db.xref.target"> <xsl:param name="linkend"/> <xsl:param name="is_chunk"/> <xsl:choose> <xsl:when test="$is_chunk"> <xsl:value-of select="concat('?', $linkend)"/> </xsl:when> <xsl:otherwise> <xsl:value-of select="concat('#', $linkend)"/> </xsl:otherwise> </xsl:choose> </xsl:template> Unfortunately, the value of $is_chunk doesn't always seem to be accurate since yelp's concept of chunks are more granular than docbook's. So, with this XSL template (and a couple of minor code changes to support it, will attach to this bug) images work, internal anchor links work and many inter-document links work. The remaining links that don't work seem to be Table Of Content hyperlinks, which are still being generated with fragment syntax even though they're meant to link to another document. I think this should be fairly straightforward to fix if you're familiar with docbook's XSL system, but I don't have the time to dig too deep right now. I'd recommend solving this by fixing the HTML generation to use URI fragments correctly rather than trying to go bug-for-bug compatible with the old Gecko backend. Assistance with the XSL would be welcome -- please check out the patch.
Created attachment 114977 [details] [review] Get images loading (proof of concept) Making use of base_uri
(In reply to comment #26) > > This sounds like something the gio/soup WebKit backend might do. I'm doing > testing using the default (curl) http backend which doesn't exhibit this issue > -- can you try with that? The gio/soup backend is probably not ready for real > world use just yet. > Confirmed, curl backend don't show this issue. I've updated jhbuild in order to not force soup backend. Oh, forgot to mention. A "yelp-webkit" module is available in jhbuild.
(In reply to comment #27) I've checked in fixes based on Alp's patch from comment #28. This does slightly more within the XSL, but gets actual chunk depths from g-d-u to make everything work. I also made info page links work again ;) Based on this, I'd say Webkit backend is ready to go.
Re: comment 30 > I've checked in fixes based on Alp's patch from comment #28. Tested branches/webkit r3163 with webkit-1.0.1 (curl backend). Builds OOTB. I'm still having a problem with links of the kind "Section X.Y -- Subject"; nothing happens when I click on them. AFAICS all other links work. All graphics and screenshots now render on screen, but except for the yellow-box info and notepad icons, they do not print (tested with PDF output backend). BTW, search is still extremely slow, but at least I can quickly load the "Home" page now. (Don't forget there is still unused test_* stuff in src/Makefile.am.)
yselkowitz, The image printing and "Section X.Y" link bugs seem to also be present in the xulrunner-based yelp too so no regressions, but I'll take a look into the first issue soon, looks like an easy one to solve. Don, Nice fix, it's working like a champ. Are we good to get this merged to trunk?
(In reply to comment #31) > Re: comment 30 > > I've checked in fixes based on Alp's patch from comment #28. > All graphics and screenshots now render on screen, but except for the > yellow-box info and notepad icons, they do not print (tested with PDF output > backend). > Image printing fixed in r3165. Let me know if it works for you.
Re: comment 33 > Image printing fixed in r3165. Let me know if it works for you. Tested r3167; screenshots do print. But now I'm getting a lot of pages giving me errors in a red box: This page contains the following errors: error on line 495 at column 36: Encoding error Below is a rendering of the page up to the first error. (Nothing else appears.) But if I print document in such a case, it is printed correctly.
I improved the XSL chunking algorithm slightly. I found a single corner-case remaining, but it's part of a larger bug. > Don, > > Nice fix, it's working like a champ. Are we good to get this merged to trunk? Since GNOME 2.24 isn't going to ship with webkit [1], I'd prefer to leave this on a branch for now. However, as soon as we branch for 2.24, I'll merge this to trunk. Until then, I'll maintain both branches and leave this bug open. [1] http://mail.gnome.org/archives/devel-announce-list/2008-August/msg00001.html
Ping??
Just a heads-up: webkit's recent versions (SONAME 1.0.2 now, I think) dropped the curl backend, leaving only soup.
Just tested the latest version from webkit branch. Seems to work quite well, but all the "About this document" pages display this error: --- This page contains the following errors: error on line 473 at column 69: Encoding error Below is a rendering of the page up to the first error. ---
(In reply to comment #38) > Just tested the latest version from webkit branch. Seems to work quite well, > but all the "About this document" pages display this error: > Not for me, yelp from webkit branch crash on startup. This is the stacktrace (BTW bug-buddy seems unable to get it) (gdb) run Starting program: /opt/gnome2/bin/yelp warning: Lowest section in /usr/lib/libicudata.so.38 is .hash at 0000000000000158 [Thread debugging using libthread_db enabled] [New Thread 0x7f992f96f7e0 (LWP 8356)] [New Thread 0x7f991c912950 (LWP 8359)] [New Thread 0x7f9917fff950 (LWP 8360)] [New Thread 0x7f99177fe950 (LWP 8361)] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7f9917fff950 (LWP 8360)] __xmlParserInputBufferCreateFilename (URI=0x42d228 "/opt/gnome2/share/yelp/man.xml", enc=XML_CHAR_ENCODING_NONE) at xmlIO.c:2472 2472 if (((z_stream *)context)->avail_in > 4) { (gdb) thread apply all bt
+ Trace 216747
Thread 3 (Thread 0x7f9917fff950 (LWP 8360))
Thread 2 (Thread 0x7f991c912950 (LWP 8359))
(In reply to comment #38) > Just tested the latest version from webkit branch. Seems to work quite well, > but all the "About this document" pages display this error: > > --- > This page contains the following errors: > error on line 473 at column 69: Encoding error > Below is a rendering of the page up to the first error. > --- I just pushed 379145060b8fc90fa7c81c0bff20d025e61adc38 to the webkit branch fixing these.
Current versions of Yelp are using WebKit. Closing.