After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 456438 - Search support for DL
Search support for DL
Status: RESOLVED OBSOLETE
Product: damned-lies
Classification: Infrastructure
Component: general
unspecified
Other All
: Normal enhancement
: ---
Assigned To: damned-lies Maintainer(s)
damned-lies Maintainer(s)
: 397606 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2007-07-13 00:43 UTC by Dimitris Glezos
Modified: 2018-05-22 12:06 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Updates index page and .htaccess for search page existence (1.43 KB, patch)
2007-07-13 00:44 UTC, Dimitris Glezos
none Details | Review
implementation with msggrep (6.10 KB, text/x-python)
2007-07-13 19:36 UTC, Dimitris Glezos
  Details

Description Dimitris Glezos 2007-07-13 00:43:38 UTC
Please describe the problem:
Hey all.

I've implemented a search functionality in DL. You can see it in action at:

  http://translate.fedoraproject.org/search/

I'm opening this bug to better review it before putting it in trunk.

Steps to reproduce:
1. 
2. 
3. 


Actual results:


Expected results:


Does this happen every time?


Other information:
Comment 1 Dimitris Glezos 2007-07-13 00:44:20 UTC
Created attachment 91704 [details] [review]
Updates index page and .htaccess for search page existence
Comment 2 Dimitris Glezos 2007-07-13 00:46:23 UTC
Two more files are needed apparently, search.py and templates/search.tmpl. Both can be found at:

http://cvs.fedoraproject.org/viewcvs/web/flpweb/?root=l10n

This patch adds another dependency for damned lies, the translate toolkit (if we put it in the trunk the REQUIREMENTS file needs updating too).
Comment 3 Claude Paroz 2007-07-13 09:27:59 UTC
Is the dependency on translate toolkit just used for the pogrep function? Wouldn't it be possible to use msggrep calls to achieve the same functionality, hence preventing a new dependency? What's the rationale?
Comment 4 Dimitris Glezos 2007-07-13 19:36:15 UTC
Created attachment 91750 [details]
implementation with msggrep

Right, translate toolkit is only used for pogrep. The rationale was that I didn't find a way for msggrep to omit creating a PO header in the output and that toolkit has a python API, giving us the ability to do more things. For example, msggrep prints out stuff that it shouldn't in some occasions (like warnings).

Nevertheless, I implemented it with msggrep as well. Attaching patch.

There are some problems currently:

 1. When I deploy it at publictest4.fedora.redhat.com I get the following warning:

    msggrep: warning: Locale charset "ANSI_X3.4-1968" is different from
                  input file charset "UTF-8".
                  Output of 'msggrep' might be incorrect.
                  Possible workarounds are:
                  - Set LC_ALL to a locale with encoding UTF-8.
                  - Convert the translation catalog to ASCII using 'msgconv',
                    then apply 'msggrep',
                    then convert back to UTF-8 using 'msgconv'.

 2. Warnings are passed as normal output and are triggered as false matches.

 3. Highlighting doesn't work for case-insensitive non-ascii matches
Comment 5 Gil Forcada 2007-07-25 14:11:26 UTC
*** Bug 397606 has been marked as a duplicate of this bug. ***
Comment 6 Ignacio Casal Quinteiro (nacho) 2008-08-21 08:19:22 UTC
Any news on this? Would be great having this for gtranslator.
Comment 7 Claude Paroz 2008-08-22 06:31:33 UTC
Frankly, I'm reluctant to add new functionalities to DL. I think the future is really using Transifex, which has this already.
Comment 8 Gil Forcada 2008-08-22 21:54:29 UTC
It's feasible to use transifex right now?

I mean, if it's mature enough we can make the switch after 2.24 (to ease the pain for translators) or just install ASAP in a parallel location to test everything out during the new cycle.
Comment 9 Dimitris Glezos 2008-08-25 11:41:01 UTC
From the discussions @ GUADEC, I think we should be ready to start playing with Transifex on our infrastructure.

A parallel deployment should take place, since Tx still needs Damned Lies for the statistics. There's a branch that does statistics, we just need to get it in mainline (hopefully soonish).
Comment 10 Gil Forcada 2009-01-10 15:19:45 UTC
Now that we have Vertimus doesn't change your thoughts Claudio?

If Transifex will be merged (after 2.26?) and it has already search capabilities maybe is enough.

Another option is use the open-tran.eu API:
http://open-tran.eu/dev.html
http://open-tran.eu/RPC2

Gtranslator have a plugin for it also.
Comment 11 Dimitris Glezos 2009-01-10 16:57:25 UTC
Users should be given the ability to search the PO files on the server. I'm not sure if translate-toolkit would be sufficiently fast for such a complex task. There are some more efficient methods for fulltext search such as Lucene, solr and Xapian.

  http://code.google.com/p/djapian/

open-tran.eu should be an additional feature, if needed.
Comment 12 Gil Forcada 2012-10-16 22:42:28 UTC
From what I see on djapian, it's only for running searches within what is already on django's db, so we should first add all translations into the database, which I don't think has much benefits apart from bloating up the database...

I'm not really a fan of the idea of getting searches within damned-lies, I would rather make it more easy to extract po files from it (if it is not easy enough) and then maybe help hacking on open-tran.eu or whichever other search tool out there to make it so that there's an option to show only results for GNOME related modules for example.
Comment 13 Alexandre Franke 2012-10-17 09:52:45 UTC
Let me just point at http://pylyglot.org/ which also allows searching in translations and which has python code available at https://github.com/omaciel/pylyglot
Comment 14 GNOME Infrastructure Team 2018-05-22 12:06:15 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to GNOME's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/damned-lies/issues/4.