After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 309015 - Evince needs accessibility support
Evince needs accessibility support
Status: RESOLVED FIXED
Product: evince
Classification: Core
Component: general
unspecified
Other Linux
: Normal normal
: ---
Assigned To: Evince Maintainers
Evince Maintainers
Depends on:
Blocks:
 
 
Reported: 2005-06-25 16:16 UTC by Jonathan Blandford
Modified: 2011-04-19 15:39 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Tar.gz with patch and new files (5.41 KB, application/x-gzip)
2005-07-25 21:33 UTC, Nickolay V. Shmyrev
  Details
EvDocumentAccessibleInterface (6.70 KB, patch)
2010-05-20 16:35 UTC, Daniel Garcia
reviewed Details | Review
pdf backend implements EvDocumentAccessibleInterface (2.71 KB, patch)
2010-05-20 16:37 UTC, Daniel Garcia
none Details | Review
atk interface implementation (46.03 KB, patch)
2010-05-27 14:50 UTC, Daniel Garcia
none Details | Review
atk interface implementation (47.42 KB, patch)
2010-06-01 14:33 UTC, Daniel Garcia
none Details | Review
AtkInterface implementation (48.29 KB, patch)
2010-06-07 11:49 UTC, Daniel Garcia
none Details | Review
AtkInterface Implementation (30.91 KB, patch)
2010-06-15 14:26 UTC, Daniel Garcia
needs-work Details | Review
AtkInterface Implementation (160.50 KB, patch)
2010-06-23 15:53 UTC, Daniel Garcia
none Details | Review
caching GtkTextBuffer in EvViewAccessible (10.05 KB, patch)
2010-06-29 11:03 UTC, Daniel Garcia
needs-work Details | Review
caching GtkTextBuffer in EvViewAccessible (10.15 KB, patch)
2010-06-29 14:47 UTC, Daniel Garcia
committed Details | Review
Caret mode (13.44 KB, patch)
2010-07-16 10:46 UTC, Daniel Garcia
none Details | Review
Patch to fix Orca stopping when changing pages (while on caret mode) (4.46 KB, patch)
2010-07-27 09:37 UTC, Max Nicosia
none Details | Review
Patch to fix cursor getting out of Sync with Orca (2.61 KB, patch)
2010-08-03 07:57 UTC, Max Nicosia
none Details | Review
Ensures that loaded page gains focus so Orca can read it (1.34 KB, patch)
2010-08-04 09:20 UTC, Max Nicosia
none Details | Review

Description Jonathan Blandford 2005-06-25 16:16:33 UTC
We should add an ATK implementation to EvView.
Comment 1 Nickolay V. Shmyrev 2005-07-25 21:33:56 UTC
Created attachment 49749 [details]
Tar.gz with patch and new files

Initial patch. Comments are welcome
Comment 2 Marco Pesenti Gritti 2005-07-26 12:58:39 UTC
Leaving this to Jhonatan, not sure if it will have to wait after freeze or we
risk it now...
Comment 3 Jonathan Blandford 2005-07-26 13:39:17 UTC
Definitely post freeze.  There's a lot more work to be done.
Comment 4 Marco Pesenti Gritti 2005-08-31 18:01:11 UTC
We branched, time to look in this! :)
Comment 5 Nickolay V. Shmyrev 2005-10-29 09:05:15 UTC
Comment on attachment 49749 [details]
Tar.gz with patch and new files

This patch committed.
Comment 6 Willie Walker 2006-07-17 16:23:20 UTC
Hi All:  I'm curious what the status of this bug is.  For GNOME 2.16, we'd really like to see compelling access to PDF documents for people with visual impairments, and Evince seems like it could probably a good partner in this effort.  The Orca screen reader (http://live.gnome.org/Orca) is good test platform for this, and we're happy to help you get going with it.  We're also happy to test any fixes to Evince as well.  :-)
Comment 7 Nickolay V. Shmyrev 2006-07-17 16:31:57 UTC
Currently we lack AtkText implementation, that's the main point. I think it will be substantial effort and it will require some poppler rework as well. As far as I know nobody is working on it, so patches are welcome :)
Comment 8 Willie Walker 2008-02-25 21:27:06 UTC
Hi All:

Good news -- we may get support for this task!  While the exact details are to be worked out, we need at least the following:

* Full keyboard navigation must be supported. The user must 
  be able to navigate around (Left/Right/Up/Down, 
  Ctrl+Left/Right/Up/Down, Home/End, Ctrl+Home/End, 
  Shift+nav_keys to select, etc) in the document content
  as well as use the keyboard to select and copy document 
  content. 

  NOTE: The F7 key is used as a means to enable/disable 
  caret navigation, but it may just make sense to always have 
  caret navigation enabled for evince.

* As the caret is moved, accurate AT-SPI "object:text-caret-moved" 
  events must be issued to reflect the new caret position in the 
  document.

* As selections are made, accurate "object:text-selection-changed"
  events must be issued to reflect the current state of selection.

* As focus changes from the document content to the menu bars (and 
  other objects in the UI), accurate and appropriate AT-SPI "focus:" 
  and "object:state-changed:focused" events must be issued to reflect 
  the new object with focus.

* The AT-SPI Accessibility_Text interface must be fully supported for 
  the document content. 

Testing should be performed with at least PDF documents (including PDF documents with forms and annotations) Accerciser, Orca, and GOK. Preferably, feedback from end users should be solicited and incorporated as well.  Success criteria includes compelling access by Orca and GOK users.
Comment 9 Willie Walker 2008-02-26 00:20:44 UTC
Please also add to this support for Accessibility_Hypertext and Accessibility_Hyperlink.

(In reply to comment #8)
> Hi All:
> 
> Good news -- we may get support for this task!  While the exact details are to
> be worked out, we need at least the following:
> 
> * Full keyboard navigation must be supported. The user must 
>   be able to navigate around (Left/Right/Up/Down, 
>   Ctrl+Left/Right/Up/Down, Home/End, Ctrl+Home/End, 
>   Shift+nav_keys to select, etc) in the document content
>   as well as use the keyboard to select and copy document 
>   content. 
> 
>   NOTE: The F7 key is used as a means to enable/disable 
>   caret navigation, but it may just make sense to always have 
>   caret navigation enabled for evince.
> 
> * As the caret is moved, accurate AT-SPI "object:text-caret-moved" 
>   events must be issued to reflect the new caret position in the 
>   document.
> 
> * As selections are made, accurate "object:text-selection-changed"
>   events must be issued to reflect the current state of selection.
> 
> * As focus changes from the document content to the menu bars (and 
>   other objects in the UI), accurate and appropriate AT-SPI "focus:" 
>   and "object:state-changed:focused" events must be issued to reflect 
>   the new object with focus.
> 
> * The AT-SPI Accessibility_Text interface must be fully supported for 
>   the document content. 
> 
> Testing should be performed with at least PDF documents (including PDF
> documents with forms and annotations) Accerciser, Orca, and GOK. Preferably,
> feedback from end users should be solicited and incorporated as well.  Success
> criteria includes compelling access by Orca and GOK users.
> 

Comment 10 Nickolay V. Shmyrev 2008-02-26 17:15:34 UTC
Thanks amazing, but who will work on it :) The code will require deep poppler changes at least.
Comment 11 Willie Walker 2008-02-26 18:10:35 UTC
(In reply to comment #10)
> Thanks amazing, but who will work on it :) The code will require deep poppler
> changes at least.

We may be able to find someone, and we may also have people who can serve as mentors.  If this were the case, would you be willing to entertain the notion of reviewing and accepting patches?
Comment 12 Nickolay V. Shmyrev 2008-02-26 18:12:56 UTC
Sure
Comment 13 Willie Walker 2008-02-27 15:24:13 UTC
OK - it's official.  :-)  Please refer to the "TASK: Evince Accessibility" section of http://live.gnome.org/Accessibility/GOPA for detailed information on the task.
Comment 14 Carlos Garcia Campos 2008-02-27 18:09:44 UTC
Ok, accesibility is now in our RoadMap for 2.24 (http://live.gnome.org/Evince/Roadmap) :-)

Nickolay, what changes would be needed in the poppler side? 
Comment 15 Nickolay V. Shmyrev 2008-02-27 18:13:35 UTC
I think it would be nice to get full text layout in some form similar to tex boxes from poppler side and use it in selection rendering and accessibility
Comment 16 Rafael Rocha 2008-04-07 23:37:36 UTC
Hello! We are two brazilian students in our last year of Computer Science at Senac Unviersity in São Paulo. Danilo Oliveira (daniloao@gmail.com)and Rafael Rocha (rafaelcrocha@gmail.com). To conclude our studies we must make a final job. We have decided to work with accessibility for free software. One job that is a strong candidate to us is to work in improving Evince Accessibility. Now, we are deciding between this and trying to make a brazilian portuguese voice (our mother language) for festival. So we would like to know a little bit more of what is needed in evince so we can see if we have the proper skills to do it. We have some experience programming with C, Python and Java. We are users also of Gnu/Linux for some years. We already have glanced the ATK library and we have seen some examples of code. We would like to learn more about what is needed so we can make our decision. We are open to work with more people as a team. 
Comment 17 Nickolay V. Shmyrev 2008-04-08 04:59:07 UTC
You are welcome, Raphael. Join #evince channel on irc.gnome.org and we'll discuss details. Brasilian Portuguese voice is also a very interesting proposal :)
Comment 18 Willie Walker 2008-04-08 11:29:28 UTC
Hi Raphael:

I'd definitely recommend this task over a new voice for Festival.  There's already reportedly some Brazilian Portuguese voices for MBROLA, and eSpeak also reports to have support for Brazilian Portuguese.  I also have first hand experience creating new voices in FestVox/Festival, and I can report that it can take a very short time to make a bad voice, and a very long time to make a good voice.  The work involves finding good voice talent as well as very careful alignment of the data.

The Evince Accessibility task is very important, and will also be a great challenge.  In addition, you will learn a lot of skills that can help you in future jobs.  :-)
Comment 19 César Octavio López Natarén 2008-07-18 06:07:05 UTC
What's the status of this report? Is anybody working on it?
Comment 20 Nickolay V. Shmyrev 2008-07-18 06:49:05 UTC
The task is still open, nobody working on it.
Comment 21 Rafael Rocha 2008-08-14 23:13:09 UTC
Hello,
I am taking back the task after 1 month winter vacations when I had no time... In the last semester, I produced some beginners documentation about accessibility and evince in portuguese. Most of all translation. Now I have one semester to work on the code and see if I can be of any help. I would apreciate some instruction from someone who know more about evince structure. I understood that the task is abou implementing ATK Interfaces, and I produce some dummy code with calls to the accessibility library. 
My email, as I said before is rafaelcrocha@gmail.com
Comment 22 Brian Cameron 2008-08-18 21:00:57 UTC
I was responsible for implementing much of the existing AtkText interfaces for all GTK+ text-based widgets.  If you use custom text widgets in Evince, then you may need to write your own AtkText implementation for your widgets.  Note the GailTextUtil interface in the libgail-util library.  This provides common 
functions needed by various text widgets.

  http://library.gnome.org/devel/gail-libgail-util/stable/

I think this might be useful to review if you need to implement any AtkText interfaces for your custom widgets.  I'd also obviously refer to the libgail code which implements the various GTK+ text widgets since it is a good example of how to implement these interfaces.

If you need any help using the library, or writing code that implements AtkText, or need other help, please feel free to pose any questions.  I'm on the cc:list of this bug report and will respond to anything you might ask.

Comment 23 Li Yuan 2008-08-19 02:24:36 UTC
Evolution's e-text is an example of accessibility implementation of custom text widget. Just note that e-text doesn't depend on libgail-util because gail was a separate library and Evolution doesn't want to depend on it. Now gail is in gtk+, so feel free to call libgail-util functions.

Any questions, feel free to send mail to me or add comments on the bug.
Comment 24 Rafael Rocha 2008-09-26 01:15:23 UTC
Thanks Li Yuan and Brian Cameron for the support offered. Next week will be crucial as I will go to the implementation. I have been working around with Gail and it is very useful. To bad evince doesn't use GTK+ and I can't do the direct bindings. 
Previosuly I've chated with Nicolay and he suggested me three steps to follow:

1) Replace poppler_page, poppler_selection and poppler_link with a single poppler_markup that should contain a) letters combined in words b) their coordinates c) links if required

2) use this poppler_markup to implement AtkText

3) use it in rendering selections and in text search

I had already done something very simple, altough not very useful inside poppler_page interface, in the poppler_page_get method I could print the result string. I thought about, instead of printing it, sending it to my ATkText. But this, altough could impress my teacher and guarente my grade would be useful for the comunity and for everyone who relly on acessibility.

So, I will taker a look next week at the examples provided and try to see ehat I can.
I don't know if I should quit nikolays idea of unificating those tree interfaces (seems to big work for the little time I have) into poppler_markup. What you think?  Maybe if I could get some help...

Is there an IRC channel where I can find you all? #evince at gnome.org is always dead...

Thanks and sorry for the time taken to reply.

Rafael
Comment 25 Rafael Rocha 2008-09-26 01:22:44 UTC
correction: But
this, altough could impress my teacher and guarente my grade wouldn't be useful
for the comunity and for everyone who relly on acessibility.
Comment 26 Brian Cameron 2008-09-26 17:16:02 UTC
If your questions are about how to implement ATK interfaces, then I recommend using the #a11y channel at irc.gimp.org.  The people on the #a11y channel are more likely to be familiar with accessibility issues and ATK than people on #evince.  My handle on IRC is "yippi", so you can look for me there.  Note that next week is a public holiday in China, so Lin probably won't be available to help until after October 5th.  If you have specific questions, I also recommend you send Lin and myself an email if you don't find us on IRC.

Note that some AtkText interfaces are a bit complicated to implement.  Does the evince/poppler code use pango?  If so, then you can probably make use of the libgailutil library which has some common functions used by various text widgets.
Refer to GTK+ in the modules/other/gail/libgail-util directory.  These functions 
hide some of the uglyness of interacting with pango so this code doesn't need to be duplicated in more than one place.  

If you find yourself copying code from gail into evince/poppler, then it might make sense to instead make common functions in libgail-util and share the code instead of copy/pasting.

Good luck!
Comment 27 Rafael Rocha 2008-10-15 00:28:10 UTC
Hi, can someone tell me what the ev-view-accessible does? I found this file, but no reference to any of its methods in the rest of the code? Was it an unsucesful implementation of a11y to Evince?
Comment 28 Nickolay V. Shmyrev 2008-10-15 07:03:31 UTC
It's the implementation of AtkAction accessibility interface.
Comment 29 Ray Wang 2009-03-28 15:36:13 UTC
what is this bug's current status? 
I could see Evince's each child from Accerciser, and does it mean that Evince is accessible now?
Comment 30 Nickolay V. Shmyrev 2009-03-28 15:45:19 UTC
Ray: EvView needs to implement AtkText
Comment 31 Rafael Rocha 2009-03-28 22:23:51 UTC
Anyone willing to take this project, I can help you understand glib, atk, at-spi and start from the point I left. I could not produce a lot of code, but I have written a lot of documentation (in portuguese though)... At the end I could make orca read an pdf in evince, but "cheating" a little. I registered in the ev-view a gtk-text widget (that has already text accessibility implemented) and I pasted the lines being selected on it, and this widget took the rest of the job (and evince had one more window)... I graduate and I will not take this project alone because of lack of time, but I still want to help...
Comment 32 Sam 2009-04-05 05:05:52 UTC
I was in talks with William Walker regarding this project.
So, I will let you know once I start working on  this, and your help will be greatly appreciated :)
Comment 33 Willie Walker 2009-04-06 23:49:06 UTC
(In reply to comment #32)
> I was in talks with William Walker regarding this project.
> So, I will let you know once I start working on  this, and your help will be
> greatly appreciated :)

Many thanks Sam!  Note that GOPA has ended and we're still not sure about sources of funding for this.  However, your contributions as a community member would be greatly welcomed, and this work could easily serve as an example of to put in your portfolio of work.
Comment 34 Rafael Rocha 2009-08-25 22:06:37 UTC
Anyone still on that? I have some time I could help...
Comment 35 Willie Walker 2009-08-25 22:12:44 UTC
(In reply to comment #34)
> Anyone still on that? I have some time I could help...

It's stagnant and no work has been done yet, so any help would be AWESOME!
Comment 36 Rafael Rocha 2009-08-25 22:20:45 UTC
Hi,

As you can see from previous post, during my last graduation semester, I took this task as my end course project. But it took me so much time to understand glib, at-spi, that I could not produce anything usefull. I had only one night per week to work on this, and I had to produce a lot of documentation, UML and all this stuff, for me a waste of time (nothing new for english speakers, but I wrote those in portuguese...)
So, I am here... I am not a good hacker though, so I need someone else to help... My partner may also take some time on this... I want to help also, because I am not working directly with programming (more sysadmin), so I don't wan't to forget all I've learned...
I will appear in IRC next week...
Comment 37 David Bolter 2009-08-25 22:25:08 UTC
Hi Behdad... do you have any cycles for mentoring here?
Comment 38 Behdad Esfahbod 2009-08-26 20:14:03 UTC
Yes, I'll blog about it soon.
Comment 39 Sam 2009-08-26 20:28:03 UTC
It is nice to see more people interested in contributing to this project.

Willie and Behdad,I got tied up with school stuff until end of next week.
I shall get back to you regarding getting some work done very soon.

Rafael , I will be more than happy to have your help too and I shall see you on the IRC.
Comment 40 Juanjo Marín 2010-03-25 22:29:11 UTC
I'd like to know the current status of this bug. 

Sam, are you working on this ?

If not, I think it could be a good idea to give a detailed to-do list of what it needs to be done.

I have interest on working on it and there are people on the Evince list who asked about this [1].

[1] http://mail.gnome.org/archives/evince-list/2010-March/msg00030.html

PS: yippi is usually on the irc to offer his help about atk :)
Comment 41 Sam 2010-03-27 11:02:04 UTC
Juanjo, very nice to know that you are interested to work on it.
I got to apologize bug for the terribly delayed response regarding my work or any progress on this bug.
I have only had a brief look at this bug for a week or two in the months of September and October '09.
Later on, I was off this bug due to personal problems. 
I tried to make some progress from where I left off this on this bug since mid-March'10.
Due to my current circumstances, I will need more time to comprehend and acquire the appropriate knowledge to make progress on this bug.

I shall provide the detailed to-do list of what needs to be done in a day or two. I have emailed Behdad, and Brian regarding this as well.
Let me get back to you about this after I get their response regarding my mail.

Thanks once again for showing interest to work on this bug. Keep in touch.



(In reply to comment #40)
> I'd like to know the current status of this bug. 
> 
> Sam, are you working on this ?
> 
> If not, I think it could be a good idea to give a detailed to-do list of what
> it needs to be done.
> 
> I have interest on working on it and there are people on the Evince list who
> asked about this [1].
> 
> [1] http://mail.gnome.org/archives/evince-list/2010-March/msg00030.html
> 
> PS: yippi is usually on the irc to offer his help about atk :)
Comment 42 Sam 2010-03-29 04:20:06 UTC
Hope you got my email.
You can share it with anyone(mailing list or so) 
and let me know your thoughts.

Hopefully Rafael will have time to chip in as well.
(In reply to comment #40)
> I'd like to know the current status of this bug. 
> 
> Sam, are you working on this ?
> 
> If not, I think it could be a good idea to give a detailed to-do list of what
> it needs to be done.
> 
> I have interest on working on it and there are people on the Evince list who
> asked about this [1].
> 
> [1] http://mail.gnome.org/archives/evince-list/2010-March/msg00030.html
> 
> PS: yippi is usually on the irc to offer his help about atk :)
Comment 43 Juanjo Marín 2010-03-31 10:58:11 UTC
Sam, I've received your emails. I don't any experience at all with a11y, so if you think you can do the work, for me it is ok. Surely I'll spend more time than you fixing this bug. 

I think it is a good idea to do a summary of what have been done and what should be done. The maintainters are the ones who knows better the code and we can ask them for some guidance. I guess there is some work to do on poppler before to work on evince.

There is another person, Javed Shaikh who has interest on helping on this, and you mention Rafael too. So if we define some task we can help you if you want.
Comment 44 Rafael Rocha 2010-03-31 12:09:19 UTC
Hi, I am here. Having a TODO list I can contribute. Do you wan't to arrange a meeting in IRC?
Comment 45 Sam 2010-03-31 13:37:38 UTC
Good to hear that you have received my emails.
I have only had time to read and comprehend relevant material mentioned in 
the email, i.e., studying or taking time to briefly study the gaim stuff 
as well as Evince and Poppler codebases 
just to get in touch and more FAMILIAR with it, as I had a long break after the months of Sept / Oct '09.

Due to my current circumstances, I will not have time immediately to make
any progress.

Please feel free to take over  the work and focus on write up of the 
proposal as mentioned in the emails. 
It looks like Rafael would like to help with the To-Do list as well.

I'm glad that Javed Shaikh likes to help with this task.
Please forward the same information to him, and inform him to add himself 
to this bug for further notifications / progress with regards to this 
task.

Yes, Rafael's interest to help out can be seen in the comments on the bug.
I think he will be able to participate or meet regarding this discussion in the IRC as well.

Hope this helps.

(In reply to comment #43)
> Sam, I've received your emails. I don't any experience at all with a11y, so if
> you think you can do the work, for me it is ok. Surely I'll spend more time
> than you fixing this bug. 
> 
> I think it is a good idea to do a summary of what have been done and what
> should be done. The maintainters are the ones who knows better the code and we
> can ask them for some guidance. I guess there is some work to do on poppler
> before to work on evince.
> 
> There is another person, Javed Shaikh who has interest on helping on this, and
> you mention Rafael too. So if we define some task we can help you if you want.
Comment 46 Javed Shaikh 2010-04-01 05:45:28 UTC
Hi Nickolay, Sam, Willie, Juonjo, Rafael, Carlos and others
As Juonjo know that I am serious about the evince accessibility and wish to work sincerely on this field. But I don't know how to start and proceed. I am working on my own grounds. But to accelerate the task I need your help. Here is what my guessed to do list, plz comment on this list.

1)ATK implementation on evince source code.
2)Poppler enhancement for text extraction from a pdf file.
3)Like pidgin, gedit etc writing a script for evince in ORCA.
4)The most important implementing a atk bridge to carry information from gtk widgets of evince to AT-SPI which in turn transfer information to assistive technology.
Comment 47 Juanjo Marín 2010-04-02 10:26:28 UTC
Hi, yesterday I found that Consortium Fernando de los Rios for the
Knowledge and Information Society [1], supported by the Regional
Government of Andalusia in Spain, has awarded some contracts to some
local companies for working on a11y, including a lot for adding a11y support to Evince. 

So, I think it would better wait for them and help them if they need it. They haven't put in contact with us yet, but I guess they will do soon as long as they sign the contract.

I quickly translated the details of this lot:

LOT 3: IMPROVE THE ACCESSIBILITY OF Evince, THE READER THE PDF READER OF GNOME, FOR THE GUADALINFO ACCESSIBLE PROJECT

The purpose of this set is to improve accessibility of Evince using the screen reader ORCA.

It is intended to allow the use of such application by blind users, so it follows the same order in which they are printed on screen. The changes made will have to have enough support from prestigious developers to stay in the free software community once they are released.

TECHNICAL DESCRIPTION

Regardless of what later detailed in this document and its Annexes, the successful bidder of this lot must conduct at least and in general, the following tasks:

1. Improved accessibility for Evince using the ORCA screen reader. Explicitly it is required to use ORCA for making Evince accesible. Moreover, it is required get speeches from the text content of PDF files rendered by Evince using this reader screen.

2. Improved functionalities in Evince
2.a Order the text content of PDF files. You will need to improve the readability of the text content of PDF files arranged in two columns or tables, so the text can be exported to the clipboard in the same order as they should be read, thereby preventing them from mixing parts from different sentences, headers, columns and/or rows.
2.b Copy the text to the clipboard properly arranged to be read for third GNOME applications.
2.c The speeches by ORCA should be as well properly arranged. In general, you must allow the speech of text content from PDF files generated by the government, and in particular those from the Official Journal of the Government of Andalusia (BOJA). This functionality will be verified by:
2.c.1. The speech by ORCA of a randomly selected text from the last BOJA available in PDF format. http://www.juntadeandalucia.es/BOJA 
2.c.2. Copy to the clipboard and into the OpenOffice.org, Thunderbird and Firefox (Gail). 

3. Creating multimedia documentation, at least with the following contents
3.a Spanish-language user and reference documentation of Evince with special indication of their usage under Guadalinex V6.
3.b Self-paced tutorial about the usage of Evince with ORCA. All documentation must be produced in accessible formats and supports free distribution licenses. 

4. Software changes in Evince under this contract should gain acceptance to be pushed to the official repositories of GNOME so they will be available in the future for all Linux distributions that use the GNOME desktop.  

More information of these contracts, see: 
     http://live.gnome.org/Guadalinfo_accesible
Comment 48 Daniel Garcia 2010-05-20 16:35:01 UTC
Created attachment 161566 [details] [review]
EvDocumentAccessibleInterface
Comment 49 Daniel Garcia 2010-05-20 16:37:53 UTC
Created attachment 161567 [details] [review]
pdf backend implements EvDocumentAccessibleInterface
Comment 50 Daniel Garcia 2010-05-20 16:38:50 UTC
As Juanjo said in the previous comment the Consortium Fernando de los Rios is funding this work and we are doing it.

I've made two changes to add some a11y support to Evince.

I tested it with Orca and it "reads" the text. There still a lot to be done but this is the beginning. I want to share my patches as soon as possible to get some feedback.

To implement the a11y layer I belive it is useful to have a new interface that every backend should implement. That's the purpose for the EvDocumentAccessibleInteface.

The main idea is to provide all methods that should be implemented by each backend in that interface and use these methods to implement the libview/ev-view-accessible.c functions. Right now the interface consist
in just one method but is expected to grow.

The second patch implements that interface for the pdf backend using existing poppler function 'poppler_page_get_text'. The offset parameter is ignore for now.

We'll keep implenting the AtkText interface in the next days.
Comment 51 Daniel Garcia 2010-05-20 16:42:33 UTC
We have also a branch in github [1] if you prefer branchs to patch files.

[1] http://github.com/danigm/evince/tree/a11y
Comment 52 Carlos Garcia Campos 2010-05-21 07:48:41 UTC
Review of attachment 161566 [details] [review]:

what other methods will this interface have? I'm not sure we need an Accessible interface, but maybe a Text interface. In order to get the whole text of a page we can use the selection interface by using the page bounding box as selection rectangle, I know it's not very intuitive, though. But in any case what we need is not only the text, but also the layout and I think such information might also be useful for other things than Accessibility.

Another problem I see with the patch is that every time ev_view_accessible_get_text() is called we ask the backend for the text. Backends should always be used with the document lock held which ins this case would block the ui. We could get the text layout information by using the PageData job (that runs in a thread with the document lock held) and cache the result in EvPageCache.
Comment 53 Daniel Garcia 2010-05-25 09:00:00 UTC
I have talked with Carlos and we are refactoring the code to put it in the right place in Evince
Comment 54 Daniel Garcia 2010-05-27 14:50:11 UTC
Created attachment 162109 [details] [review]
atk interface implementation

These patchs implements the atk interface in evince in a better way that previous patchs. It depends on poppler bug https://bugs.freedesktop.org/show_bug.cgi?id=28276

That works with orca, but exploring text line by line don't work very well because orca gets the scrollbar with text. And if you put sidebar, orca get confused, and mix lines with sidebar and scrollbar.
Comment 55 Daniel Garcia 2010-06-01 14:33:28 UTC
Created attachment 162460 [details] [review]
atk interface implementation

I made some changes in the patch, fixing in some cases segfault.
Comment 56 Daniel Garcia 2010-06-07 11:49:57 UTC
Created attachment 162924 [details] [review]
AtkInterface implementation

Using prepend instead of append commit added. I changed the glist append and used prepend and reverse for eficience.
Comment 57 Daniel Garcia 2010-06-15 14:26:38 UTC
Created attachment 163681 [details] [review]
AtkInterface Implementation

I changed the implementation of AtkInterface using the new poppler patch to get the text layout:

https://bugs.freedesktop.org/show_bug.cgi?id=28276#c4
Comment 58 Carlos Garcia Campos 2010-06-17 13:25:32 UTC
Review of attachment 163681 [details] [review]:

Hi Daniel, thanks again for the patch. Instead of using the selection interface we could add a new interface EvDocumentText with the mthods get_text() and get_text_layout() and move ev_selection_get_selection_map() renamed as ev_document_text_get_text_mapping(). This patch could be splitted into 4:

 1. libdocument changes
 2. backend changes
 3. libview changes
 4. atk implementation


More comments inline

::: backend/pdf/ev-poppler.cc
@@ +1985,3 @@
+	*n_areas = pn_areas;
+
+	*areas = g_new (EvRectangle, *n_areas);

make sure *n_areas > 0 before trying to allocate memory. You can return here, indeed if get_text_layout returns FALSE.

@@ +1987,3 @@
+	*areas = g_new (EvRectangle, *n_areas);
+	for (i = 0; i < *n_areas; i++)
+	{

Evince doesn't use the same codying style than poppler, move '{ ' to the end of previous line

@@ +1997,3 @@
+
+	g_free (pareas);
+

This is one case where we can just use the PopplerRectangle as EvRectangles since they are compatible, so that we can reuse the memory allocated by poppler. This function might be something like:

return poppler_page_get_text_layout (poppler_page, areas, n_areas);

::: configure.ac
@@ +143,2 @@
 PKG_CHECK_MODULES(LIBDOCUMENT, gtk+-2.0 >= $GTK_REQUIRED gio-2.0 >= $GLIB_REQUIRED)
+PKG_CHECK_MODULES(LIBVIEW, gtk+-2.0 >= $GTK_REQUIRED gail gthread-2.0 gio-2.0 >= $GLIB_REQUIRED)

Instead of adding gail dependency, just copy gailtextutil.c and gailtextutil.h from gail sources into cut-n-paste directory. 

We need to check whether new poppler api is available to be able to enable/disable it at compile time. See the other examples poppler_page_render, poppler_page_get_image and poppler_annot_file_attachment_get_attachment

::: libdocument/ev-selection.c
@@ +89,3 @@
 }
+
+GtkTextBuffer *

I've realized that the only thing we do in the backends is insert the text into the GtkTextBuffer, I think it's better to just return the gchar *, and create the TextBuffer in ev-view-accessible

::: libview/ev-jobs.c
@@ +594,3 @@
 	ev_page = ev_document_get_page (job->document, job_pd->page);
 
 	if ((job_pd->flags & EV_PAGE_DATA_INCLUDE_TEXT) && EV_IS_SELECTION (job->document))

I think we should have 3 flags: INCLUDE_TEXT_MAPPING, INCLUDE_TEXT_LAYOUT and INCLUDE_TEXT and LAYOUT and TEXT should be enabled only when accesibility is enabled, I don't know whether we need to change something in ev-page-cache to enable/disable flags once the cache has been created.

::: libview/ev-page-cache.c
@@ +41,3 @@
+	GdkRegion     *text_mapping;
+	EvRectangle   *page_text_layout;
+	guint          page_text_layout_length;

This is page cache, the page prefix in var names is redundant, use just text_mapping, text_layout, etc.

::: libview/ev-view.h
@@ +101,3 @@
+                                           GdkRectangle *page_area,
+                                           GtkBorder    *border);
+

Do not make this function public this way. Either rename it to something like ev_view_get_page_extents() or keep it private but available from ev-view-accesible by moving the prototype to ev-view-private.h
Comment 59 Daniel Garcia 2010-06-23 15:53:14 UTC
Created attachment 164411 [details] [review]
AtkInterface Implementation

This patch implements all changes proposed by Carlos in this comment https://bugzilla.gnome.org/show_bug.cgi?id=309015#c58
Comment 60 Carlos Garcia Campos 2010-06-26 16:48:43 UTC
Good work, thank you very much. I've applied it with some minor changes into git master. It still needs more work though:

 - It doesn't seem to work well when the sidebar is open
 - Figure out how to enable/disable text and text-layout in EvPageCache dependeing on whether a11y is enabled or not
 - Cache the GtkTextBuffer in ev-accessible instead o create/destroy it all the time.
 - Figure out how to use the EvDocumentModel in ev-accessible instead of acessing EvView attrs directly

Thanks
Comment 61 Daniel Garcia 2010-06-29 11:03:55 UTC
Created attachment 164879 [details] [review]
caching GtkTextBuffer in EvViewAccessible

Caching GtkTextBuffer in ev-accessible instead of create/destroy it all the time.
Comment 62 Carlos Garcia Campos 2010-06-29 12:19:04 UTC
Review of attachment 164879 [details] [review]:

::: libview/ev-view-accessible.c
@@ +64,3 @@
 	GtkScrollType idle_scroll;	 
+	GtkTextBuffer *buffer;
+	guint current_page;

current_page should be initialized to something like -1, so it should be gint instead.

@@ +107,3 @@
+  }
+
+  EvViewAccessiblePriv* priv = EV_VIEW_ACCESSIBLE_GET_PRIVATE (accessible);

Please, move the declaration to the vars declaration block

@@ +111,3 @@
+    return priv->buffer;
+  }
+

if priv->current_page is different from view->current_page we have to update priv->current_page
Comment 63 Daniel Garcia 2010-06-29 14:47:59 UTC
Created attachment 164896 [details] [review]
caching GtkTextBuffer in EvViewAccessible
Comment 64 Carlos Garcia Campos 2010-06-29 15:54:26 UTC
Review of attachment 164896 [details] [review]:

Pushed to git master with just a minor change. Thanks!
Comment 65 Max Nicosia 2010-07-12 12:00:43 UTC
Hey there, 

I have done a few tests on evince 2.31.4.1 with poppler 0.14, particularly on the accessibility functions. (Compiled with jhbuild with help of Daniel)

The tests have been done on the following pdfs, I aimed at having a selection of standard formats that are encountered on the Internet: 

1)http://netseminar.stanford.edu/seminars/Cloud.pdf (Lecture notes, presentation style)
2)http://www.bbctraining.com/pdfs/newsstyleguide.pdf (random pdf with text in non standard columns) 
3)http://www.cs.st-andrews.ac.uk/~mirco/papers/wosn10.pdf (Typical research papers)
4)http://www.cs.st-andrews.ac.uk/~mirco/papers/simplex10.pdf (Same as above)

I will concentrate in describing the bugs / odd behaviour that I found instead of saying if a test was passed or not. Also I believe it is worth mentioning that I am testing as to ensure that a blind user is able to use the read out functions effectively. 

1) 

-) Flat review seems to be the only way that allows to navigate through text. However when you reach the end of what fits in the page, it is not possible to scroll the page to continue reading, the red square gets stuck  at the end.
-) The red square disappears or gets completely out of sync if the page is moved with page up or down. 
-) The reading by line seems to be affected by the zooming factor. Full screen makes the flat review reading more effective but also turns the red pointer invisible. 
-) Reading text does not work at all in presentation mode (F5), orca can only read "unknown" and can also spell it. 
-) Using the dual option, traps the pointer in a page and does not allow to move to the next one. 

2) 

-) On flat review it is not possible to continue reading pass what fits on the screen, ie. it is not possible to scroll to continue reading. If the page is moved with the mouse or page up, down the pointer gets completely out of sync and either reads from wherever it landed on or continues from the beginning of the page where it was. 
-) With this document, evince reads across the lines without recognising the column structure.  

3) & 4)

-) Read line actually selects more than one line at a time, but seems to read from the second line onwards till the end of the red box. 
-) Evince seems to have a problem reading lines that have been indented, ie the start of a paragraph. In the case of both of this pdf documents, the first line (indented) is skipped, the second line is then read till the end of the square. (This behaviour occurs after once you enter the normal text, that is after the abstract section). The effect continues for at least 1 or 2 lines, then it resets itself. 
-) Evince recognises that there are columns, but I can't see any way to continue reading (flat review) once I reached the end of the column. However if I tab from the menu to the text and let Evince read the whole page it recognises the columns. 
-) If you click several times on the text of the other column is it possible to get flat review to read it in the same way as it reads the first column, but the red square gets trapped in that column. By the same procedure one can switch back to the other column. 
-) If the document is zoomed in, in such a way that only one column is visible on the screen, flat review works fine till you reach the end of what is on the screen. Then it gets stuck. 

A few of the bugs described above are present on all the documents, so I have repeated them. 
There are some more general bugs that seem to be persistent on all as well: 

While navigating through the document, the terminal outputs the following error: 
 
** (evince:3448): CRITICAL **: atk_object_initialize: assertion `ATK_IS_OBJECT (accessible)' failed

When closing Evince the last output is: 

** (evince:3448): CRITICAL **: atk_object_initialize: assertion `ATK_IS_OBJECT (accessible)' failed

I have only done the tests that I thought would cover most of the functionality that a blind person would expect, however I doubt that they are exhaustive so if anyone would like to suggest more tests or would like me to describe some problems in more detail, please let me know. (Also happy to do some extra tests that might not be just related to accessibility).

At the moment I don't know how the accessibility functions have been implemented, but if I get a bit of a description, I can start looking into it and see if I can fix some of the problems. 

Any feedback welcome. 

Cheers
Max
Comment 66 Daniel Garcia 2010-07-16 10:46:24 UTC
Created attachment 166012 [details] [review]
Caret mode

This patch add caret mode, like in firefox you can navigate throught the document using cursors. With that mode, orca can read where you are, line by line, word by word or char by char.

Features *not* implemented in this patch, that will be implemented:

 * move the cursor by clicking in a word.
 * selecting with cursor.
Comment 67 Behdad Esfahbod 2010-07-19 20:17:08 UTC
Great work Daniel!
Comment 68 Max Nicosia 2010-07-27 09:37:16 UTC
Created attachment 166633 [details] [review]
Patch to fix Orca stopping when changing pages (while on caret mode)

This patch ensures that Orca continues reading when the cursor enters the previous/next pages while on caret mode. 

*Still to fix*
Cursor gets out of sync wrt to Orca (caret mode)
Comment 69 Max Nicosia 2010-08-03 07:57:35 UTC
Created attachment 167030 [details] [review]
Patch to fix cursor getting out of Sync with Orca

Fixes cursor getting out of sync wrt to Orca, while on Caret mode. 

Provided that the document was rendered correctly by poppler.
Comment 70 Max Nicosia 2010-08-04 09:20:03 UTC
Created attachment 167101 [details] [review]
Ensures that loaded page gains focus so Orca can read it

This patch ensures that newly loaded pages obtain focus, so Orca can start reading them.
Comment 71 Jose Vilmar Estacio de Souza 2010-09-30 10:15:10 UTC
Hi all, the patches are applied in evince 2.32.0?
I am trying to read a pdf file without success, maybe I am doing something wrong.
Thanks.
Comment 72 Daniel Garcia 2010-09-30 19:35:55 UTC
(In reply to comment #71)
> Hi all, the patches are applied in evince 2.32.0?
> I am trying to read a pdf file without success, maybe I am doing something
> wrong.
> Thanks.

You need poppler 0.14 or newer.
Comment 73 Jose Vilmar Estacio de Souza 2010-09-30 23:30:29 UTC
(In reply to comment #72)
 
> You need poppler 0.14 or newer.

According to the output of the statement apt-cache show libpoppler6, I've the following installed:
Version: 0.14.2.is.0.14.1-0ubuntu1

I am trying ubuntu 10.10
Comment 74 Daniel Garcia 2010-10-01 08:53:39 UTC
(In reply to comment #73)
> (In reply to comment #72)
> 
> > You need poppler 0.14 or newer.
> 
> According to the output of the statement apt-cache show libpoppler6, I've the
> following installed:
> Version: 0.14.2.is.0.14.1-0ubuntu1
> 
> I am trying ubuntu 10.10

I've looked and you don't need poppler 0.14, you need at least 0.16 what isn't in ubuntu yet.

PD: I'll close the bug because a11y is implemented, for new stuff related to a11y like improvement, we should create new bugs.