GNOME Bugzilla – Bug 524542
Use CAIR for setting desktop wallpaper
Last modified: 2012-06-20 17:23:45 UTC
Content Aware Image Resizing is a great new algorithm to resize images while avoiding typical scaling artifacts (while possibly introducing different ones). It performs very well for reformatting images to different aspect ratios, for example, which would be very useful in placing desktop wallpapers that don't quite fit weird resolutions such as those found in laptops. It is also sometimes called "seam carving". There's a demo of it here: http://www.youtube.com/watch?v=6NcIJXTlugc And there's an open source library to actually do it on sourceforge: http://sourceforge.net/projects/c-a-i-r/ In my testing, it produces much better results reformatting images for use as wallpapers than any of the included resizing algorithms that Nautilus can use. This, combined with "directory full of image, switch at random" functionality that I hear is coming Real Soon Now(tm) would make GNOME the nicest desktop environment out of the box for using wallpapers. I'm willing to do the coding on this, as it's mostly a matter of incorporating the CAIR code (which is pretty lightweight). However, looking through the Nautilus code, it looks like Nautilus is mostly just using builtin GDK functionality. It strikes me that perhaps this functionality should better be put right in GDK (perhaps something like gdk_pixbuf_new_from_file_at_scale_cair) and then all new GNOME apps can take advantage of it, rather than just Nautilus. This is doubly true since CAIR expects image data in its own internal format to work with, so having it convert to and from the gdkpixmap format interally, hidden from the view of apps using it, is probably best. Plus, it means adding it to nautilus a lot easier, as it'll just take a new named wallpaper type (WPTYPE_CAIR?) and a tiny bit of code in libbackground/applier.c. I'd love to hear some feedback on this, both the value of the functionality in general and my ideas on where the implementation actually needs to go. For instance, the current CAIR implementation uses libpthread to accelerate the resizing process on MP systems. Is this portable enough for GNOME? I'm not sure about what platforms libpthread exists on other than Windows and Linux. Am I totally off-base with the idea that this should go into GDK? Is there a way to make it a plugin to GDK rather than trying to make it core functionality? etc.
Interesting idea, but this should really be filed against libbackground, so I'm moving it there. Hope to see a patch for this!
Except that the rendering stuff is in gnome-desktop. ;-) Anyway, couple of comments: * CAIR seems to be GPL-licensed which means it cannot be linked to any of the (LGPL-licensed) GNOME core libs, including gdk. * even if this was implemented I don't think we'd want to expose backgrounds scaled with CAIR to the user as a new (separate) option since there's no good way to describe them in the UI ("CAIR-scaled"? huh? "scaled 2"? huh? "Improved scaled"? huh, why's the bad scaled option there at all?), so it would need to replace (one of) the existing options. * license issues aside, I don't think GTK/GDK would want to add another dependency for image scaling. But since I'm not a GTK maintainer...
Jens, I bet you're tired of forwarding my bugs that I've put in the wrong place ;). Thanks for fixing this one too.
Good comments, Jens. Here's my thoughts: - Licensing is definitely an issue that I didn't look into well enough. However, I believe it may be possible to convince the author(s) to relicense under LGPL, as from what I've seen they've been good about being flexible to collaborate with other projects (like seam-carving-gui: http://code.google.com/p/seam-carving-gui/). - I think this probably does have to be a separate option (perhaps "content aware scaling"). In most cases it does a better job than regular linear scaling, but there are definitely edge cases in probably 10% of my wallpapers where it does much, much worse. I'd hate to replace the standard scaling algorithm and leave users without recourse in these edge cases. It seems like it would likely need to be documented in the Yelp page for the Appearance panel (perhaps with small image examples comparing the different algorithms) to try and reduce user confusion. - You may be right about GTK/GDK maintainers likely not wanting to add unnecessary extra code. I was unsure about this as well, which is why I wanted to collect feedback from actual GNOME developers before I started hacking code. It seemed to me to be the best place to put it, as it would like nice functionality to have available to any/every GNOME app, rather than just for wallpaper placement. If it's there, I'm sure people will use it. :) Also, I'm still concerned about the portability of the libpthread dep. If anyone knows whether it exists on every platform GNOME exists on, that'd probably be useful information to have. When I was looking at the code, it seemed like libbackground was doing the actual image loading, and using GDK to do the scaling on load. It seems like that's where CAIR would be incorporated, regardless of where the rendering ends up actually happening. Or have I misread the code?
(In reply to comment #3) > Jens, I bet you're tired of forwarding my bugs that I've put in the wrong place > ;). Thanks for fixing this one too. You can make up for it by fixing nautilus. ;-) (In reply to comment #4) > - I think this probably does have to be a separate option (perhaps "content > aware scaling"). In most cases it does a better job than regular linear > scaling, but there are definitely edge cases in probably 10% of my wallpapers > where it does much, much worse. I'm inclined to say CAIR isn't ready or appropriate then. I don't know if we really want to add options that don't work in more than 95% of all cases. > - You may be right about GTK/GDK maintainers likely not wanting to add > unnecessary extra code. I don't think the extra code is the problem. The extra dependency is. > Also, I'm still concerned about the portability of the libpthread dep. If > anyone knows whether it exists on every platform GNOME exists on, that'd > probably be useful information to have. Don't know for sure, but I doubt it. > When I was looking at the code, it seemed like libbackground was doing the > actual image loading, and using GDK to do the scaling on load. It seems like > that's where CAIR would be incorporated, regardless of where the rendering ends > up actually happening. Or have I misread the code? Depends on when you looked at it. There was a bit of reshuffling during 2.21 where parts of that code moved to gnome-desktop.
Sorry, maybe my wording wasn't very clear in my last comment. What I meant to say was that in about 90% of my wallpapers, CAIR scaling does an excellent job. In about 10%, it does a worse job than one of the regular linear scaling modes. Comparatively that's not that bad: in my wallpapers, none of the existing zoom options by themselves do a great job on more than about 60% of my wallpapers due to aspect ratio and background colouring issues. Plus, users (at the moment?) have to set the scaling mode used on a per-wallpaper basis anyway, so when they choose it they can see right away if it doesn't look good. However, your comments about licensing, extra dependency for GTK/GDK, and the possible issues with pthread portability are all well noted. I was looking at the 2.20 codebase, so if things have been reorganized then that would explain it. In terms of extra dependencies being a likely sticking point, I wonder if including the code inline into GDK (assuming licensing issues get worked out) would be any better? The code is reasonably compact (1 source file, 2 headers, compiled size of about 52k using -Os) and is likely also reasonably static at this point, as it just implements an algorithm and doesn't hook into any other libraries. Portability issues are still a problem, but then the code is fixed within the GNOME source tree and won't change often. Personally I find this approach a little sloppier, but it does avoid adding dependencies.
> In terms of extra dependencies being a likely sticking point, I wonder if > including the code inline into GDK (assuming licensing issues get worked out) > would be any better? Unlikely, I'd say. It's too large and functionally too specialized to be of general interest.
Yeah, that's what I figured.
I will close this, as there were no code contributions made, and say that I'd rather see something like this integrated into GIMP or a content-creation application than in our system settings.