GNOME Bugzilla – Bug 456438
Search support for DL
Last modified: 2018-05-22 12:06:15 UTC
Please describe the problem: Hey all. I've implemented a search functionality in DL. You can see it in action at: http://translate.fedoraproject.org/search/ I'm opening this bug to better review it before putting it in trunk. Steps to reproduce: 1. 2. 3. Actual results: Expected results: Does this happen every time? Other information:
Created attachment 91704 [details] [review] Updates index page and .htaccess for search page existence
Two more files are needed apparently, search.py and templates/search.tmpl. Both can be found at: http://cvs.fedoraproject.org/viewcvs/web/flpweb/?root=l10n This patch adds another dependency for damned lies, the translate toolkit (if we put it in the trunk the REQUIREMENTS file needs updating too).
Is the dependency on translate toolkit just used for the pogrep function? Wouldn't it be possible to use msggrep calls to achieve the same functionality, hence preventing a new dependency? What's the rationale?
Created attachment 91750 [details] implementation with msggrep Right, translate toolkit is only used for pogrep. The rationale was that I didn't find a way for msggrep to omit creating a PO header in the output and that toolkit has a python API, giving us the ability to do more things. For example, msggrep prints out stuff that it shouldn't in some occasions (like warnings). Nevertheless, I implemented it with msggrep as well. Attaching patch. There are some problems currently: 1. When I deploy it at publictest4.fedora.redhat.com I get the following warning: msggrep: warning: Locale charset "ANSI_X3.4-1968" is different from input file charset "UTF-8". Output of 'msggrep' might be incorrect. Possible workarounds are: - Set LC_ALL to a locale with encoding UTF-8. - Convert the translation catalog to ASCII using 'msgconv', then apply 'msggrep', then convert back to UTF-8 using 'msgconv'. 2. Warnings are passed as normal output and are triggered as false matches. 3. Highlighting doesn't work for case-insensitive non-ascii matches
*** Bug 397606 has been marked as a duplicate of this bug. ***
Any news on this? Would be great having this for gtranslator.
Frankly, I'm reluctant to add new functionalities to DL. I think the future is really using Transifex, which has this already.
It's feasible to use transifex right now? I mean, if it's mature enough we can make the switch after 2.24 (to ease the pain for translators) or just install ASAP in a parallel location to test everything out during the new cycle.
From the discussions @ GUADEC, I think we should be ready to start playing with Transifex on our infrastructure. A parallel deployment should take place, since Tx still needs Damned Lies for the statistics. There's a branch that does statistics, we just need to get it in mainline (hopefully soonish).
Now that we have Vertimus doesn't change your thoughts Claudio? If Transifex will be merged (after 2.26?) and it has already search capabilities maybe is enough. Another option is use the open-tran.eu API: http://open-tran.eu/dev.html http://open-tran.eu/RPC2 Gtranslator have a plugin for it also.
Users should be given the ability to search the PO files on the server. I'm not sure if translate-toolkit would be sufficiently fast for such a complex task. There are some more efficient methods for fulltext search such as Lucene, solr and Xapian. http://code.google.com/p/djapian/ open-tran.eu should be an additional feature, if needed.
From what I see on djapian, it's only for running searches within what is already on django's db, so we should first add all translations into the database, which I don't think has much benefits apart from bloating up the database... I'm not really a fan of the idea of getting searches within damned-lies, I would rather make it more easy to extract po files from it (if it is not easy enough) and then maybe help hacking on open-tran.eu or whichever other search tool out there to make it so that there's an option to show only results for GNOME related modules for example.
Let me just point at http://pylyglot.org/ which also allows searching in translations and which has python code available at https://github.com/omaciel/pylyglot
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/damned-lies/issues/4.