GNOME Bugzilla – Bug 363793
Add the ability to change the color of the flat review rectangle
Last modified: 2008-09-11 15:32:07 UTC
Being able to customize the color of the flat review rectangle would be helpful for users with low vision. (Suggested at Boston Orca users group)
This one is fairly easy to do at one level and more involved at another: 1/ The easy approach would be to just have a color setting in settings.py and then adjust drawOutline() in util.py to use it. If we go for color, we can't call: graphics_context.set_function(gtk.gdk.INVERT) 2/ The more involved approach would be putting up an accessible color wheel in the Orca preferences and using that to set the color. There might be a middle ground where we allow the user to adjust three sliders for red, green abd blue and generate the new color on the fly as they are doing so. That shouldn't be too hard.
See also semi-related bug #464734.
Created attachment 102532 [details] [review] Patch to goof with Joanie and I got to talking about the COMPOSITE extension, highlight windows, and other stuff today. So, that piqued my interest and I decided to hack a little bit. This patch is merely a goof so I wouldn't lose what I learned. What it attempts to do is create an undecorated top level window with an opacity value. The idea is that we'd just move/resize this window over the thing we want to highlight. We could also set the opacity, the color, and even draw into the window if we wanted to. In order for the opacity to work, a composite manager needs to be running. I tried running this in conjunction with Orca using gnome-mag in full screen mode, but the opacity still didn't seem to work. Got me. I haven't tried to debug it. In addition, while the top level window that is getting created doesn't grab keyboard focus, it does end up swallowing mouse clicks, such as when we press KP_Divide to force a click on a flat review item. So, a way to ignore/forward mouse events to the window underneath would need to be devised. Ah well...it's a thought and maybe a start. BTW, there's a line in there that's commented out that should be uncommented if you want to go back to the normal "xor a rectangle" method we have in place now: + #orca_state.highlightWindow = -1 + pass Another thought Joanie had was to just use gnome-mag all the time and use a zoomer for the highlighting window. I didn't try that.
First coarse pass at GNOME 2.24 planning.
Just tried this patch. For me, it was putting up a "pink" rectangle okay, but it looked like the x/y offset's were for the desktop, not for the window that currently had focus. Shouldn't be too hard to adjust that if this is an approach we want to take.
Created attachment 112494 [details] [review] Revision #1. Well the problem of it not working when the window being flat reviewed wasn't in the top-left of the screen (see comment #5) has gone away, so I've just tidied up the work that Will did (see comment #3). There are now two new settings in settings.py: # If we can do compositing, these are the red, green, blue values to use # to highlight the current item in flat review mode. # They are values between 0 and 65535 (0xFFFF), with 65535 (0xFFFF) # indicating full intensitiy # highlightingColor = [ 0xFFFF, 0x0, 0x0 ] # If we can do compositing, this is the opacity value to use to highlight # the current item in flat review mode. # This is a value between 0 and 1, with opacity 0 being fully transparent # and 1 fully opaque # highlightingOpacity = 0.1 Like several other settings in that file, they are not exposed in the Orca Preferences GUI, but they can be nicely overridden in your ~/.orca/orca-customizations.py. For example: orca.settings.highlightingColor = [ 0x0, 0x0, 0xFFFF ] orca.settings.highlightingOpacity = 0.3 When I'm using the compiz window manager (i.e. compositing is available), then it nicely highlights the current item under flat review. If I'm using metacity (no composite support), then it nicely reverts to using an xor'ed rectangle. Questions: 1/ Is this sufficient for this flat review bug? Note that there is a similar separate bug: "464734 - Provide some kind of visual feedback for the item with focus" http://bugzilla.gnome.org/show_bug.cgi?id=464734 Handling that should be separate from this issue. 2/ If it is sufficient, what should the default colors and opacity values be?
This is pretty cool, Rich. I can see it potentially being of benefit for users with learning disabilities too. I'm afraid I have questions of my own, however: :-) "Highlighting" text by covering it up with a semi-transparent window decreases contrast. In order to make the rectangle easier to see, one needs to make it less transparent and in doing so decreases the contrast further. Therefore: 1. Would it be possible to add a border to the window? 2. How hard would it be to grab the text being reviewed and display it in the new window? (i.e. could we completely cover up the thing being reviewed with better contrasting text?) And a bug: As an experiment, I modified my orca-customizations.py to use your example settings (blue rectangle, 30%) and my app-settings/gedit.py to use cyan, 30%. If I switch between a gedit document and a Writer document and flat review within each, whatever rectangle color I get first seems to persist.
Thanks for your feedback. > 1. Would it be possible to add a border to the window? It probably would, but after playing with this for 2-3 hours now, I don't know how. I think we need to get a more savvy Gtk+ person involved with this. > 2. How hard would it be to grab the text being reviewed and display > it in the new window? (i.e. could we completely cover up the thing > being reviewed with better contrasting text?) For somebody who knows what they are doing I suspect it would be trivial. > And a bug: > > As an experiment, I modified my orca-customizations.py to use your example > settings (blue rectangle, 30%) and my app-settings/gedit.py to use cyan, 30%. > If I switch between a gedit document and a Writer document and flat review > within each, whatever rectangle color I get first seems to persist. That would appear to be a separate problem. I can run Orca without this patch and achieve the same affect with two xor'ed rectangles. I suggest that this should be filed as a separate bug and fixed separately.
I misunderstood Joanie's bug in comment #7. With those two lines, it certainly won't work correctly. Instead, you will need to add the following four lines: import orca.orca_state orca.orca_state.highlightWindow = None orca.settings.highlightingColor = [ 0x0, 0xFFFF, 0xFFFF ] orca.settings.highlightingOpacity = 0.5 (adjusting the highlightingColor and highlightingOpacity values to taste).
Aha! Thanks Rich. That solves half the problem. Now my flat review rectangle in OpenOffice is blue and in Gedit it's cyan. But once it becomes cyan, it stays cyan (i.e. when I go back to OpenOffice where the rectangle color is governed by my orca-customizations.py). I tried adding import orca.orca_state orca.orca_state.highlightWindow = None to my orca-customizations.py but it doesn't seem to impact anything. Is there a corresponding change I need to make to cause the blue rectangle to come back?
orca-customizations.py is only read once at startup time. So I can think of two ways to resolve this: 1/ Tell the user's that they can only override this once. Therefore they will be forced to have only one color for this flat review highlight window. I'm not sure how many Orca users we have that hand-edit their ~/.orca/app-settings/... script files (or even know they can). 2/ Tackle the bigger issue. When we get a new window:activate event, and we look for a new script and a new potential set of script settings to load, always reread orca-customizations.py. This would then fix this problem (assuming the four lines in comment #9 were in the orca-customizations.py file. It would always probably allow us to do other clever things on a per-app basis as well. Will, what would you like to see done here? Thanks.
(In reply to comment #11) > orca-customizations.py is only read once at startup time. > > So I can think of two ways to resolve this: > > 1/ Tell the user's that they can only override this once. Therefore > they will be forced to have only one color for this flat review > highlight window. I'm not sure how many Orca users we have that > hand-edit their ~/.orca/app-settings/... script files (or even > know they can). > > 2/ Tackle the bigger issue. When we get a new window:activate event, > and we look for a new script and a new potential set of script > settings to load, always reread orca-customizations.py. This would > then fix this problem (assuming the four lines in comment #9 were > in the orca-customizations.py file. It would always probably allow > us to do other clever things on a per-app basis as well. > > Will, what would you like to see done here? For now, let's shoot for #1. At some point, we're going to need to refactor how we handle settings -- the global approach we're doing now has worked for a long time, but we are starting to see some benefits of potentially doing things in a class-based way, where each script instance gets its own settings class instance. The time to do that refactor might be around the same time we look at pulling the presentation manager abstraction and making locus of focus be a per script thing. Furthermore, I still view this flat review rectangle color something that is more of an experiment and something that we can use to draw out more user requirements in this space. As such, until we really get a better understanding of what low vision and cognitive impairment users need, I'm hesitant to have us invest a lot of time in this, especially when we have lots of other bugs that fall in the 'higher use case' category. Thanks!
> For now, let's shoot for #1. Okay, thanks. I'll do a little more experimenting, and seeif it's possibly to put a canvas (or something) in this highlightWindow, that will allow us to draw a border around the currently showing text.
(In reply to comment #6) > Created an attachment (id=112494) [edit] > Revision #1. This patch works great for me, and is something that we could use to open a discussion with the low vision and learning disability folks to see what they really want. Right now, it's just a matter of when we want to start that discussion (i.e., make sure we can follow through with the work).
Give me another day or two. I'm working on trying to draw a colored rectangle (default black) around the area too (if compositing allowed).
Created attachment 113303 [details] [review] Revision #2. Here's a second attempt at this. The problem here is that the window opacity value also affects how the outline rectangle is drawn, and that's causing it to not be shown very clearly. I thought the solution here might have been to have two composited windows, one for the highlight color and one for the outline color, each one with different opacities, but I suspect whatever is on top, is going to blot out the one underneath. I think a better solution here is to have a floating window, but with no opacity. The highlight color would be a delicate pastel shade; the outline color would be black and we'd also have another color to draw the actual text in. This would also be drawn into that floating window. All three colors would have default values in settings.py which the user could override in their customizations file. This would involve adjusting the drawOutline() method(s) to have an extra parameter which is the text string to be flat reviewed. This is probably a good idea anyhow, if we are thinking of doing this for the focused item as well. More tomorrow.
Created attachment 113304 [details] Sample Python example that draws lines, rectangles and text into a drawing area. I found this in a www.pygtk.org tutorial. The #START ... #END sections were where I modified the default code to try to add opacity. Hopefully this will be useful when I add in displaying the text into the "floating" window.
Created attachment 113311 [details] [review] Revision #3 Well after spending most of the evening trying to come up with something that would color the background and draw text into a non-opaque floating window, and finding out that that introduced even more problems, I suddenly realized that we could vastly improve things by just: 1/ Setting a much higher opacity value. 2/ Starting with a much lighter highlight color. 3/ Adding in an expose handler to correctly redraw the flat review rectangle when we got expose events. This is starting to look good. What we now need to do is tweak the three settings.py values: highlightOpacity, highlightColor and outlineColor, to get the best possible default contrast. Suggestions welcome.
Created attachment 113550 [details] [review] Revision #4. Joanie and I have been working on improving this. We now have a flat review rectangle where you can set: 1/ the highlight (background) color. 2/ the outline color. 3/ the text color. 4/ the width of the outline (in pixels). There is no opacity value any more. Things still to do: 1/ We are displaying text in the flat review rectangle inappropriately. The algorithm needs to be refined, but I think we only want to put text in there if queryText() returns a non-zero length string that is visible, for the current object under flat review. 2/ Now that we aren't using compositing to get window opacity, maybe this will all just work with metacity. Need to check.
Things still to do: 3/ Pick good default values for the four new settings.
I'll play with this some. I think we want to keep the opacity -- I think that (what we had earlier) will appeal to users with learning disabilities who are not visually impaired. We should add that option back and only draw the text if the window is not transparent.
Created attachment 113703 [details] [review] Revision #5. Joanie has been working on this over the weekend (thanks!). It's starting to look very nice. Here are a few comments from her: Disclaimers: * Still a work in progress * Needs comments/documentation * Needs cleaning up. This is still in the "get it working" phase Addresses: * The disappearing rectangle * Rectangle appearing in the window list of the bottom panel * Writing text we shouldn't ("label" in the Orca main window) * Covering up non-text objects * Implementation of a fully opaque outline independent of the highlight * Improving contrast given a semi-transparent window (see below) * Moving all (or most) of the rectangle code into its class so that the default script doesn't have to know how it works. * Probably other stuff, but I'm forgetting now :-) The Contrast Issue: Try this patch as is in Gedit. In particular, change the following settings: highlightOpacity = 0.4 redrawText = False Review text in a document. Then set redrawText to True and try again. It doesn't take much of a highlight to diminish the contrast of the text being viewed. :-( Redrawing the text in cases where we can obtain the attributes helps quite a bit, I think. In order to redraw the text in a semi-transparent window, however, we need to know the critical attributes of all of the text being reviewed. For instance, in a Writer document, if a given line has text that is bold and not bold, we need to know about both or the text we draw will not quite line up with what is visible under the semi-transparent highlight and things will look even worse than they do with the poor contrast. Right now this patch doesn't have the smarts to parse the whole string for attributes. Instead, it grabs the attributeRun and checks to see if the offsets suggest there's mixed formatting in the given string. If so, it declares the formatting non-uniform and doesn't bother trying to redraw it. Because of the contrast issues, I'd like to try to remedy that. ---- I've moved the HighlightWindow class out into a new file. When I went to pylint it, I found there was also a context = self.getFlatReviewContext() missing in the panBrailleRight() in default.py, so I've added it in. That's got to have fixed something. :-) Will, can you give this a try and see where you'd like us to go from here? I.e. is it time to get input from the low-vision people on the Orca mailing list? Thanks.
(In reply to comment #22) > Created an attachment (id=113703) [edit] > Revision #5. Interesting! > * Moving all (or most) of the rectangle code into its class so that > the default script doesn't have to know how it works. I needed to get rid of the "()" in "class HighlightWindow():" for this to work with Python 2.4. > * Probably other stuff, but I'm forgetting now :-) I also found these: * Mouse clicks don't work since they seem to be swallowed by the highlight window. * The text doesn't always necessarily match what's on the screen. In particular: 1) In the pidgin main window, the "Chats" node that organizes the active chat rooms comes up in a strange spot. Probably an AT-SPI implementation issue, though. 2) Spoken/generated text appears (e.g., navigate to a checkbox) The other thing that is interesting is whether we really want to re-render the text or not. If we decide to run down the path of zoomer-like lenses over things, then I suspect we probably do want to do the re-rendering. > Will, can you give this a try and see where you'd like us to go from here? > I.e. is it time to get input from the low-vision people on the Orca mailing > list? I think it probably is time to let users try this, but it would be nice to get the mouse click issue resolved first. Without it fixed, flat review users will not be able to use the keypad to click on the flat review object. Note also that we will need to really coordinate hard with Kristian since he will be working on a highlighting API. I suspect this and his approach will overlap. So...at worst, I expect this patch to be something to toss to users as a means to help foster discussion for requirements gathering.
Created attachment 114129 [details] [review] Revision #6 * Back to one rectangle * Simpler code * Better highlighting (I think) * Deals with the click and key issues (I think) * Added underline and wedge as options (wedge still needs work) For the life of me, I cannot figure out the overlapping issue. Try this: 1. Flat review the Orca Main Window by word, moving from Preferences to Quit. 2. Review the current line For some reason we're getting the borders from the Quit button. You can see something similar when flat reviewing up by line in gnome-terminal. I have tried everything one can do to pixbufs, drawing areas, gtk windows, and gdk windows, including clearing, redrawing, hiding, iconifying, destroying, setting masks, etc. Basically, if it was anywhere in the gtk or gdk docs, or is a service of the local voodoo priest, I've tried it. (Who woulda thunk a single, skinny, headless chicken could bleed THAT much??) Anyhoo, Rich, go for it. ;-)
Thanks. Just tried this. It's got potential. There is another problem though. Start Orca and flat review the Orca main window by character. As you move right, the new flat review rectangle has part of the border of the previous character running vertically down the middle. Maybe this is exactly the problem you are describing in comment #24 and I didn't understand.
> Maybe this is exactly the problem you are describing in > comment #24 and I didn't understand. It is indeed :-(
For flat reviewing by characters, I believe the problem is that the second and subsequent times, you are taking a new pixbuf of the underlying character, but you have the red frame outline from the highlight window for the adjacent character as well. If you set the decorationStyle to HIGHLIGHT_DECORATION_LINE you'll see that the problem goes away. What you might want to do is to snap a pixbuf without that frame (maybe hide the flat review window before grabbing the new pixbuf), or initially take a pixbuf of the whole application window once, and then just grab a sub-portion for the area you are interested from that window pixbuf each time you need one.
Thanks for looking at this! > If you set the decorationStyle to HIGHLIGHT_DECORATION_LINE you'll > see that the problem goes away. Yup. > What you might want to do is to snap a pixbuf without that frame > (maybe hide the flat review window before grabbing the new pixbuf), I tried that -- see comment #24. Does that work for you? And if so, what's the exactly magical lines of code and where do they belong. > or initially take a pixbuf of the whole application window once, > and then just grab a sub-portion for the area you are interested > from that window pixbuf each time you need one. But what if the content changes? Say the user is in a mail client and a new message has come in?
> I tried that -- see comment #24. Does that work for you? I didn't try that. It was clear from your comment that you had explicitely tried that. I was simply making a suggestion. > But what if the content changes? Say the user is in a mail client > and a new message has come in? Yup. And what if the window you are flat reviewing is partially obscured or has components that are hidden? All are things that need to be dealt with.
Created attachment 114190 [details] [review] Revision 6a. Just so it doesn't get lost, here's a version of Joanie's last patch that makes some slight changes to the getPixbuf() method in highlightwindow.py: It hides the flat review rectangle, flushes the event queue and sleeps for 0.3 of a second before snapping a copy of the root window at the desired location and reshowing the flat review window. What we really need is a version of this which doesn't require the "time.sleep(0.3)". I thought that using: _display = gtk.gdk.display_get_default() _display.sync() instead of gtk.gdk.flush() might have done it, but it's not so.
Removing the testing required for now. There's still issues with the above. I had an idea regarding how it might be resolved, but it didn't pan out. :-( Assigning this one to me, but keeping it's "low" priority in mind.
This *might* merit a separate bug, but I want to add a quick note here saying we also need to take magnification into account. There are a couple issues: 1) In simple XOR/outline mode, when we draw a rectangle, it is not magnified. Instead, the rectangle is drawn where the object would be on the screen had things not been magnified. 2) In fancy highlight window mode, the rectangle is drawn in the right spot and is magnified appropriately (yeah!), but the next is illegible.
Created attachment 118436 [details] [review] Patch to do just outlining using windows This patch uses multiple windows to accomplish just outlining (i.e., no highlighting is done). It seems to work well in magnified and unmagnified mode, and pylints well. The complete gtk-demo regression tests are running right now, but a spot check of the ones that do flat review seems to indicate it will test fine. The current default is red wedgie mode. Sounds painful. The main issue with the multiple window approach is that Compiz's shading of windows ends up looking a little weird when we use four windows to draw a box around the obj. I might try to see if I can use the shape_combine_mask to create a hollow "see through" window with a rectangular border.
Created attachment 118455 [details] [review] Patch to use a single "hollow" window for the outline rectangle This seems to do rather well, and I optimized it to provide performance that matches the XOR rectangle we currently have. The only issue I'm seeing now is that there's an odd interaction with Compiz where the outline window will not pop to the top if you Alt+Tab to another window and start flat reviewing there. Instead, it moves all the way to the bottom of the stacking order. Very strange. So...Rich, Joanie, and I have all give this beast the good old college try. I'm wondering if we should just cease work on it until the composite-based highlighting API is ready from the GOPA magnification task?
Created attachment 118462 [details] [review] One more try to eliminate the "fly under the window" problem This patch uses the set_user_time trick that Joanie had in a previous patch. Seems to eliminate the hiding window trick nicely. So...this patch is ready for review. There are still some quirks, but it's looking ready to check in: 1) The outlines are a bit jumpy. It's because I don't hide/show them. I found hide/show or iconify/deiconify made things pretty unperformant when Compiz was involved. 2) The outlines don't appear on the Quit confirmation dialog. I suspect this is because it also does the set_user_time trick and interferes with what we do. Joanie, can you give this one a shot?
Still playing with it, but overall I think it's pretty cool! Initial thoughts: 1. I'm not accustomed to the wedge growing/shrinking like that. It typically is the same size -- and really it's mostly used in association with the caret (which is admittedly not what we're doing here). My temptation might be to either center it or place it at the beginning of the item of interest. 2. When you flat review to a blank line the "rectangle" disappears completely. You don't know where you are until you move again. What about placing it at the start of the newline? 3. I wonder if there's a "hint" we could give Compiz to suggest that it not fly windows in and out? 4. There are times where the rectangle is covering up a portion of the thing being reviewed or seems to be "off" a bit. Of course, it's that way without your patch. :-) In the context of "it's a debugging tool we didn't turn off" I think this is no big deal. I'm not sure if we should try to adjust the position before suggesting the user community take it for a spin, or if we put it out there with "known issue" statement.
Created attachment 118495 [details] [review] Address some of the issues Joanie mentioned > 1. I'm not accustomed to the wedge growing/shrinking like that. I made a change to keep it the same size and centered. Kind of weirder than the changing wedge size to me, but I'm OK with it and we can fix it later if the thousands of users who have requested wedges complain. ;-) > 2. When you flat review to a blank line the "rectangle" disappears completely. Fixed -- it's now the way it used to be with the XOR'd rectangle. > 4. There are times where the rectangle is covering up a portion of the thing > being reviewed or seems to be "off" a bit. Of course, it's that way without > your patch. :-) In the context of "it's a debugging tool we didn't turn off" I > think this is no big deal. I believe it might be a text/object extents bug. Do you have specific examples? OK - executive decision time. As is, is this better than the XOR solution we have in place today? If the answer even wreaks of a small hint of "well...if we made this little tweak", I'm going to push this bug out to FUTURE and we will stay with the XOR solution for 2.24.
Joanie and I talked about this in IRC and agree that the patch is better than the current XOR solution. It also solves the problem of being able to change the color of the rectangle (this bug ;-)). We're also pretty exhausted by this thing. So, I'm closing it as FIXED. If users request the ability to highlight the rectangular region of interest instead of just outlining it, we can add that as an RFE.