I think that this idea is interesting, and might want to work on it sometime soon. However, I have a question.

Full-text searching in files is something I've tried before and I had difficulty with it. Most in-built string searching algorithms require you to read the entire contents of a file into a string, which can be very memory intensive for large files. Reading a few bytes at a time might also not work because our search window might be broken between a potential result.

There are several options which are even better than GNU grep for searching text. Two of which I'm aware of are 'ack' and 'silversearcher-ag'.

Would this plugin require an in-built text search, or can it rely on one of the available tools? Based on input parameters configured by the user (search phrase, target directory, paths and filetypes to exclude, etc) the plugin can use any one of the tools available on the system: silversearcher, ack, git-grep and finally grep. This way, it will work out of the box on most *nix systems, and users can install specialized tools to improve performance.

The results are read back from the tool that was chosen and displayed back in gedit. This lets us focus on the primary experience, rather than reinventing the wheel.

However, there is a big downside to invoking an external tool - and that is that the output format of one of the tools can change in the future, which will break the plugin in strange ways. Then the output parser will have to accommodate older and newer-versions.

So is relying on an external tool to do the text searching a worthwhile pursuit?

Anirudh,

I agree that reading an entire file into memory to search it isn't a great idea.  However, you could certainly implement this by reading and searching one line at a time from each file.  Most libraries that read files provide a function for reading a single line, and a line should always easily fit into memory (unless you attempt to read a line from a binary file, but a full-text search plugin should detect and avoid those).

Altenatively, you could use an external tool.  I think it would be reasonable to assume that GNU grep will be present on the system.  GNU grep has been stable and available for a long time, so I don't think you need to worry about its output format changing.  I'd be reluctant to use other tools such as ack since they may not be present on all systems.

I think an option to use git grep might be useful too, actually.  I suppose if you wanted you could make the plugin general enough to use other tools such as ack, but I think that would probably be a feature for advanced users only.

Regarding reading files into memory, I would suggest memory-mapped I/O.  That allows a library to read in as much of the file it needs at any time, and unused pages can be freed in the background by the operating system.

We looked into this problem earlier with Garret Reiger.  He thought (and I agree) that the GLib regex classes would work fine.  They're backed by the PCRE library (http://pcre.org/) which is widely used and stable.

I agree that using 'grep' is a better option than relying on any of these other tools as it's reasonably good performance. Plus, the main benefit of these tools is that they ignore directories like .git and binary files - something that the gedit plugin can do.

I would like to approach the problem in this manner:

I will build an abstraction to search a folder using grep. I will also build a small suite of acceptance tests. I will then use this abstraction inside the gedit plugin to search and display results in the UI. This will enable me to focus more on the UI. Then, based on the feedback, I can modify it to use memory-mapped searching if required. And because of the tests, I can reliably swap the search backends.

I will be starting tomorrow. If you have any suggestions on how the workflow should be, I'm all ears. Also, I am writing it in python and using the "snippets" plugin as a reference implementation on how to structure the files, and how to abstract the UI using Gtk.Builder. If there is a better one I can use as a reference, please let me know.

Have you tried Sagasu? It is in Debian.

http://perso.b2b2c.ca/sarrazip/dev/sagasu.html

@Shawn: I just tried Sagasu and it seems reasonable, though its user interface feels a bit technical (it has items "Dir recursion depth" and "Exclude CVS dirs" and "Exclude symlinked dirs") and a bit dated (it still uses GTK 2, and mentions CVS!)  In any case, I think there's a strong need for a search tool that's integrated with gedit.

@Anirudh: I'm glad to hear you're interested in working on this.  Your basic plan sounds reasonable to me.  You might want to look at some of the existing gedit plugins for searching in files.  In particular, the one I use most often is Advanced Find (http://code.google.com/p/advanced-find/).  I think its interface is too complicated, but I like the way it presents search results to the user.  It's written in Python, so you might be able to reuse some of its code (since it's released under the GPL and so is gedit).  In fact, one possible implementation path would be to fork the Advanced Find plugin and attempt to clean up its interface to be nice enough to be accepted as a core gedit plugin.  I filed a bunch of user interface bugs with the Advanced Find author, but he ignored most of them or marked them as WontFix.  In particular, these bugs were filed by me:

http://code.google.com/p/advanced-find/issues/list?can=1&q=reporter%3Aadam%2Cmedovina.org

Or you could start with a new implementation.  It's up to you.

Adam, thanks for the pointer.

I did take a pretty detailed look at that plugin. It mostly seems good except for the fact that searching seems to be happening on the UI thread leading to issues like [1].

Since this also uses grep internally I will consider it as some validation that 'grep' is good enough for a basic search system.

[1] - https://code.google.com/p/advanced-find/issues/detail?id=118

I have finished an initial version of the find-in-files plugin. I am quite happy with the implementation and I feel it's almost ready for everyday use but needs some polishing and testing for edge cases.

You can see screenshots:

Search Window: http://i.imgur.com/SMTBG25.png
Search Results: http://i.imgur.com/hJLBTdQ.png

You can get the plugin here: https://bitbucket.org/skyronic/findfiles

To activate it you have to go to Search > Find in Files

I will work on integrating it with gedit-plugins or gedit assuming I get a go-ahead from the gedit developers.

Where I need help:

1. What to call the plugin, and what terms and language to use for the UI?
2. How to improve the UI to make it work according to GNOME's Human Interface Guidelines.

Also, any suggestions to improve the workflow are welcome.

Anirudh, this looks like a reasonable start.  Here's a first round of feedback.  If you'd like to address these issues then I'd be happy to review again.  Many of these points are related to the GNOME interface guidelines as you suggested.

I still suspect that an easier implementation path might be to start with the existing Advanced Find plugin I mentioned and clean up its interface (and move searching into a background thread), but that's completely up to you.

- When I first tried to run gedit with your plugin following your instructions, I saw

Traceback (most recent call last):

+ Trace 232896

I debugged a bit, and found that SearchUIManager.get_search_widget was looking for findinfiles.ui at

  /home/adam/.local/share/gedit/plugins/findinfiles/findinfiles.ui

even though it was actually at

  /home/adam/.local/share/gedit/plugins/findfiles/findinfiles.ui

since your git repo is called "findfiles".  I had to rename the findfiles directory to findinfiles in order to get the plugin to run.

- According to the GNOME interface guidelines, field labels use sentence capitalization:

  https://developer.gnome.org/hig-book/3.10/design-text-labels.html.en#layout-capitalization
  
  So in your interface, "Search Phrase:" should be "Search phrase:", and "Match Case", "Single Word", "Regular Expression" and "More Options" should similarly be recapitalized.
 
- The Folder label needs a colon. 
 
- In your comment at https://bugzilla.gnome.org/show_bug.cgi?id=696859, you wrote "To activate it you have to go to Search > Find in Files".  But even when your plugin is active, I don't see a Find in Files item in the Search menu.  (I'm using gedit 3.8.3 with GTK 3.10.6, by the way).  I think there should be an item like this, with keyboard accelerator Ctrl+Shift+F.

- After I've searched, I see no way to go from the search results back to the search interface.

- Every time you run grep, you write 'Running command: ...' to the console.  A production plugin should not ever write output unless an error occurs.  Instead you could write this to the GNOME debug log using g_debug (or its Python equivalent), or simply comment out this line in your code so you can reactivate it when you need it.

- Start gedit with your plugin enabled, close the Untitled Document which appears by default, then attempt to search using your plugin.  You'll see this error on the console:

Traceback (most recent call last):
  File "/home/adam/.local/share/gedit/plugins/findinfiles/searchui.py", line 36, in on_start_clicked
    self.searchParamsBox.hide()
AttributeError: 'SearchUIManager' object has no attribute 'searchParamsBox'

- The Find button should be the default button in your search interface.  That means that it should appear slightly different than the other buttons (for an example of this, choose File->Open and notice that the default button Open looks different than Cancel), and that I should be able to press Enter anywhere in the search area to activate a search.

- The Find button should be disabled until I've entered at least one character of a search phrase.

- Each field label in your search interface should have a keyboard mnemonic, which lets me jump to that field by pressing Alt plus a letter.  This is standard for GNOME dialogs and even though your interface isn't a dialog, it would make sense here too.

- In search results, each occurrence of the search term should appear in bold or otherwise be highlighted (just like in the Advanced Find plugin, for example).

- In search results, currently I have to expand each filename individually to see all the matches in that file.  That's a bother.  Instead, I think all matches should initially be expanded automatically.

- After I search, your interface shows "Search results for: Search Term".  Instead of "Search Term", you should display the actual text the user searched for.

- Your plugin's tab has the label "Search".  I think this is too generic.  "Find in Files" would be a reasonable name, I think.

- When I double click a search result, you open the file and navigate to the line containing the match.  That's good.  In addition, you should automatically select the matching text on that line (again, like Advanced Find).

Adam,

Thank you for such a detailed and thorough response. I have made almost all the changes that you have requested. I am awaiting the next round of feedback from you and other members of the community.

I would like to clarify that I chose to not fork and improve advanced-find is because there are some sections I felt will be difficult to maintain.

There are some choices in advanced-find (like having "find all in document" functionality) which makes the code a little more complex and hence less maintainable. I decided to keep a strong focus on searching files in a directory and the amount of code to do things like aggregating results and handling selections on the results list can happen in a fraction of the code of advanced-find.

> since your git repo is called "findfiles".  I had to rename the findfiles
> directory to findinfiles in order to get the plugin to run.

Ouch, I have changed the name of the repo. Thanks for not preventing this from letting you try it :)

The new repo is: https://bitbucket.org/skyronic/findinfiles

> - According to the GNOME interface guidelines, field labels use sentence
> capitalization:

I have fixed this.

> - The Folder label needs a colon. 

Fixed

> (I'm using gedit 3.8.3 with GTK 3.10.6, by the way).  I think there should be
> an item like this, with keyboard accelerator Ctrl+Shift+F.

I forgot to push the commit with the menu item. It's my fault. But it's there in the repository now. You should be able to find the menu item. I will also add a keyboard accelerator soon.

> - After I've searched, I see no way to go from the search results back to the
> search interface.

Added a back button. It works now.

> - Every time you run grep, you write 'Running command: ...' to the console.  A

Removed.

> AttributeError: 'SearchUIManager' object has no attribute 'searchParamsBox'

I could not find out how to fix this. It's very strange. I will have to investigate it more later.

> The Find button should be the default button [..snip..] 
> I should be able to press Enter in the search area to activate a search.

Fixed.

>  The Find button should be disabled until I've entered at least one character
> of a search phrase.

Fixed.

> Each field label in your search interface should have a keyboard mnemonic,

All labels and fields have a keyboard mnemonic now.

> In search results, each occurrence of the search term should appear in bold
> or otherwise be highlighted

Highlighted the results in bold.

> In search results, all matches should initially be expanded automatically.

Done

> After I search, your interface shows "Search results for: Search Term". 
> Instead of "Search Term", you should display the actual text.

Oops, bug on my part. Fixed

> Your plugin's tab has the label "Search".  I think this is too generic. 
> "Find in Files" would be a reasonable name, I think.

Changed

> When I double click a search result, you open the file and navigate to the
> line containing the match.  That's good.  In addition, you should automatically
> select the matching text on that line (again, like Advanced Find).

Added this functionality.

@Adam - can you provide another round of feedback on the updated version of the plugin? I can only work on weekends, and I would like make more improvements on this 2 days from now. But I'm out of ideas on what would make sense next...

@Anirudh: thanks for your latest code iteration.  It's been a busy week, but I'll be able to give you another round of feedback tomorrow (Saturday).

Of course, one item I know you'll want to work on is the AttributeError described above.  By the way, it seems to occur if I have closed any document in the current gedit session.

OK - here's some more feedback:

- Your plugin description as it appears in the Plugins tab is "A plugin to find in files".  The words "A plugin to" are unnecessary, since everything in this list is a plugin.

- You've added a menu command "Find in Files".  This should be "Find in Files...", since the command requires more input from the user before it can be carried out.

- As I mentioned in my last review, the "Find in Files" command should have a keyboard accelerator.  I recommend Ctrl+Shift+F.

- The "Find in Files" command should also have a keyboard mnemonic (i.e. a letter that appears underlined and that I can press with Alt to choose the command from an open menu).

- Run gedit from a terminal.  Open Edit->Preferences->Plugins.  Clear the checkbox next to Find in Files, the check it again.  You'll see this on the console:

  (gedit:32426): Gtk-WARNING **: Inserting action group 'FindInFilesPluginActions' into UI manager which already has a group with this name

  That should not happen.

- When I invoke Search->Find in Files, the keyboard focus should be placed in the "Search phrase" box so I can begin typing a search phrase immediately.

- If I select some text and invoke Search->Find in Files, then the "Search phrase" field should automatically be filled with the selected text.

- Currently the Folder field defaults to my home directory.  If the file browser sidebar is open, I think that Folder should automatically be set to the current directory in the sidebar.

- You have a checkbox "Whole Word".  As I mentioned field labels use sentence capitalization, so this should be "Whole word".

- When I double click a search result, you automatically select the entire line containing the match.  Instead, only the matching text itself should be selected.

- When I double click a search result, if the file it contains is already open then you select the containing line, but if the file was not already open then you open it but nothing is selected.  The selection should always occur whether or not you're opening the file.

- A search might take many seconds to complete.  When a search is in progress, there should be some indication of that fact such as a spinner.  There should also be some way to cancel a search in progress.

- When no search results are found, I think there should be some indication of that such as a message "No matches found".

A couple more points:

- If a search is in progress, and I press Back and start a new search, then a second grep process will start running as the same time as the first.  I think the first search should be cancelled as soon as I press Back.

- If a search is in progress and I exit gedit, then grep keeps running in the background.  The plugin should kill the grep process in this case.

Hi Adam,

I have pushed some improvements:

1. When a search is started, a spinner is displayed with the option to cancel the search. This is automatically hidden at the end.
2. Added a Ctrl+Shift+F hotkey for it.
3. Added a 'no match found' message
4. Selected text is automatically entered into the search entry.
5. Search entry gets focus by default.

There are still some pending issues:

1. The error when you activate the plugin multiple times.
2. Even when cancel or back button is pressed, the grep process continues.
3. When you click on a row to open a file the first time, the line isn't highlighted. This is a very strange error and I couldn't understand why it isn't working. I will need to consult with the gedit developers for help regarding this.
4. Some miscellaneous exceptions.

I am very inexperienced with threading and the grep-cancellation feature will require inter-thread communication and I have never done this before. So it will be a little while before I manage to get the grep cancellation working great.

Anirudh, thanks for the update.  It's nice to read about the further progress you've made.  Good luck working through the bugs and threading issues you listed above, and I'll look forward to your next iteration!

Adam,

I have pushed a new version:

1. Pressing cancel or quitting gedit will cancel the grep process.
2. Fixed the exception when no tab is open.
3. Clicking on a search result highlights the specific word in the line as you suggested.
4. The plugin now removes all menu items and keybindings, and cleans up the panel automatically when deactivated. (and fixes the gtk warning you saw)

Known issues:

1. When a search result of an un-opened tab is activated, it doesn't highlight the line. I need the help of gedit developers to figure this out as I don't know how it could be happening.
2. The default directory is still the home folder. This is because I couldn't find any way to detect if the 'filebrowser' plugin is open and determine the folder that's open. I am looking for a workaround for this.
3. The plugin doesn't search .git, .svn, etc folders by default but it needs a preferences button to disable this.

Created attachment 264312 [details] [review]
patch to add the findinfiles plugin to gedit

This is a patch that adds the plugin and required makefile changes to be built with gedit from master.

Adam,

I have fixed the new-tab highlight issue, and even a new tab will have the proper highlight. Turns out the issue was that the gedit tabs are loaded asynchronously, and when I request for the highlight, the tab isn't ready yet, so I had to wait for them to load. I listened to the 'loaded' signal as suggested by jessevdk.

I have pushed the latest revisions, but I was able to find a few minor bugs which I will be fixing along with your next round of suggestions which I hope can be soon because I will be pretty busy next week and would like to have this stable by then.

Adam,

I'm waiting for your next round of feedback. Hope to make it production-ready this weekend.

Anirudh,

ok - thanks for your continuing work on this.  Here's some more feedback.

These issues from my previous feedback remain unresolved:

- You've added a menu command "Find in Files".  This should be "Find in Files...", since the command requires more input from the user before it can be carried out.

- If the file browser sidebar is open, I think that Folder should automatically be set to the
current directory in the sidebar.  You said you didn't know how to detect the directory that the file browser is showing.  You could look at the source to the external tools plugin, which sets GEDIT_FILE_BROWSER_ROOT to that value.  See the function file_browser_root in https://git.gnome.org/browse/gedit/plain/plugins/externaltools/tools/functions.py .

- I should be able to press Enter anywhere in the search area to activate a search.  Currently this only works when I press Enter in the 'Search phrase:' field.  (Perhaps it would be easier to implement this if you turn your search interface into a dialog, since then you could use the standard GTK mechanism for making a button the default - I'm not sure if that will work outside a dialog).

In addition:

- You open the search dialog if I type Shift+Ctrl+F, but it also opens if I type only Ctrl+F or Shift+F!  Also, the accelerator should appear beside the Find in Files command in the Search menu, but it does not.  From a glance at your code it appears you've implemented this by explicitly checking for a Gdk.KEY_F event.  That's not the way that accelerators are normally implemented in a GTK program.  If you're using a GtkActionGroup you would normally pass an accelerator to a GtkActionEntry passed to gtk_action_group_add_actions.  See

  https://developer.gnome.org/gtk3/stable/GtkActionGroup.html#GtkActionEntry

- Currently I must double click each search result to open it.  I think a single click would be more convenient.

- You have checkboxes labeled "Case sensitive", "Whole word" and "Regular expression".  Note that gedit's built-in incremental search has the options "Match Case", "Match Entire Word Only" and "Match as Regular Expression".  (You can see these options by pressing Ctrl+F to search, then clicking the magnifying glass icon in the search box.)  It might be better to use the exact phrases gedit uses (with sentence capitalization) for the sake of consistency.

- Suppose that I search for "xyz" when "Case sensitive" is clear, so that it matches "XYZ".  If I double click a search result containing "XYZ" then that word should automatically be selected, but it is not.

- After a search, it would be nice if I could press Esc to activate the Back button.

- There seems to be no way to access the Cancel Search button using the keyboard.  At the very least it should have a keyboard mnemonic.  It would be also be nice if I could press Esc to cancel a search.

- When there is only a single match in a file you display "1 matches".  This should be "1 match".

- Even when the "regular expression" box is unchecked you launch grep with --regexp.  This doesn't seem right.

- Set the Folder: field to the findinfiles source directory, check "Regular expression" and set "Search phrase:" to

    vers*
  
  Now press Find to search.  gedit will hang, and the console will show

===  
  Exception in thread Thread-1:
Traceback (most recent call last):

+ Trace 232949

===

- Perform a search that will run for a while, then cancel it.  Now exit gedit.  You'll see this on the console:

===
Traceback (most recent call last):
  File "/home/adam/.local/share/gedit/plugins/findinfiles/findinfiles/__init__.py", line 38, in do_deactivate
    self.search_manager.raw_cancel_search()
  File "/home/adam/.local/share/gedit/plugins/findinfiles/findinfiles/searchui.py", line 57, in raw_cancel_search
    self.backend.cancel_search()
  File "/home/adam/.local/share/gedit/plugins/findinfiles/findinfiles/searchbackend.py", line 18, in cancel_search
    self.grep_thread.cancel_search()
  File "/home/adam/.local/share/gedit/plugins/findinfiles/findinfiles/greprunner.py", line 98, in cancel_search
    self.proc.terminate()
  File "/usr/lib/python3.3/subprocess.py", line 1758, in terminate
    self.send_signal(signal.SIGTERM)
  File "/usr/lib/python3.3/subprocess.py", line 1753, in send_signal
    os.kill(self.pid, sig)
ProcessLookupError: [Errno 3] No such process
===

- Open Find in Files, click the Folder: button and choose Music from the list.  Now enter a search phrase such as "xyz" and press Find.  The plugin will launch grep and will search in your home directory, not in the Music directory.

- Click the Folder: button, then choose 'Other...'.  The resulting dialog box will say "Select a File".  I think this should be "Select a Folder".

- Create a folder in your home directory called "test".  Open Find in Files, click the Folder: button and choose 'Other...'.  In the resulting file browser dialog, navigate to your home folder, single click the directory 'test' without entering it and press Open.  The Folder: field will now read 'test'.  Enter a search phrase and press Find.  Now the plugin will launch grep searching in your home directory, not in 'test'.

- I think the plugin should not search in gedit backup files (i.e. files whose names end with '~'), at least not by default.

- Perform a search that will run for a while, then press Back without cancelling the search.  Now the 'grep' process will still be running (you can see it if you run 'ps').  I think the plugin should kill the grep process in this situation.

- In your search interface it's not clear what I can enter in the Include: or Exclude: boxes.  Can I specify multiple patterns (such as '*.c *.h')?  If so, should they be separated by spaces, commas, or something else?  I think you should add tooltips or other documentation that clarifies this.

Anirudh, you haven't posted here in a while.  Just so we know, are you still working on this?

Hi Adam,

I intend to fix all your requests over the next 2-4 days. I am a little busy with my day job but I intend to proceed with this and have it ready for master after 1 or 2 more iterations (provided the gedit devs agree)

Anirudh: once again, are you planning to continue work on this?

Anirudh: Are you planning to continue work on this? If not, would you mind I I would take over your code and update it until it's ready?

Mail from Anirudh, just for the record:
"Hi Robert,

Thanks for getting in touch. I've actually fixed a lot of issues and had kept it ready for updating after a lot of time, but unfortunately my computer failed completely, and had to go back to using my old laptop.

Now while I had all the code backed up to a private git repo, I am having a lot of trouble simply getting gedit to build from master (because I intend to push this as a patch to gedit master or gedit-plugins master). jhbuild isn't really working too well and I have to spend a little more time fixing it.

But yes, I intend to fix it up and push it asap. I will let you know if I'm not able to, and will license all the code under GPL so you can continue work on it (Currently I haven't specified a license for it so until I do it's not suitable for merging into gedit).

Thanks,
Anirudh"

I have fixed most of the issues for gedit 3.8, and confirmed that it still works with gedit 3.10 but with gedit master (the one with the minimalist UI) everything has stopped working. I can't see a menu item, or do I see a 

It took me a lot of time to get gedit to compile from master (my primary workstation is Ubuntu, and it became impossible to use jhbuild on that and I had to install fedora instead)

I can verify that the plugin is active. Even other plugins like python console no longer work. Have there been some changes to the plugin API?

Yes, see:
https://wiki.gnome.org/Apps/Gedit/ThirdPartyPlugins-v3.12

Firstly, I have ported all the changes required to work with gedit master.

(In reply to comment #22)
> - You've added a menu command "Find in Files".  This should be "Find in
> Files...", since the command requires more input from the user before it can be
> carried out.

Fixed.


> - If the file browser sidebar is open, I think that Folder should automatically
> be set to the
> current directory in the sidebar.  You said you didn't know how to detect the
> directory that the file browser is showing.  You could look at the source to
> the external tools plugin, which sets GEDIT_FILE_BROWSER_ROOT to that value. 
> See the function file_browser_root in
> https://git.gnome.org/browse/gedit/plain/plugins/externaltools/tools/functions.py
> .

Fixed

> - I should be able to press Enter anywhere in the search area to activate a
> search.  Currently this only works when I press Enter in the 'Search phrase:'
> field.  (Perhaps it would be easier to implement this if you turn your search
> interface into a dialog, since then you could use the standard GTK mechanism
> for making a button the default - I'm not sure if that will work outside a
> dialog).

I feel making it a dialong will break the flow. I have manually listened to the enter key on text inputs (using enter key on checkboxes turn them on/off)

So marking this as fixed.

> 
> In addition:
> 
> - You open the search dialog if I type Shift+Ctrl+F, but it also opens if I
> type only Ctrl+F or Shift+F!

Fixed.

> Also, the accelerator should appear beside the
> Find in Files command in the Search menu, but it does not.  From a glance at
> your code it appears you've implemented this by explicitly checking for a
> Gdk.KEY_F event.  That's not the way that accelerators are normally implemented
> in a GTK program.  If you're using a GtkActionGroup you would normally pass an
> accelerator to a GtkActionEntry passed to gtk_action_group_add_actions.  See
> 
>   https://developer.gnome.org/gtk3/stable/GtkActionGroup.html#GtkActionEntry
> 
> - Currently I must double click each search result to open it.  I think a
> single click would be more convenient.

Done.

> 
> - You have checkboxes labeled "Case sensitive", "Whole word" and "Regular
> expression".  Note that gedit's built-in incremental search has the options
> "Match Case", "Match Entire Word Only" and "Match as Regular Expression".  (You
> can see these options by pressing Ctrl+F to search, then clicking the
> magnifying glass icon in the search box.)  It might be better to use the exact
> phrases gedit uses (with sentence capitalization) for the sake of consistency.

Done.
> 
> - Suppose that I search for "xyz" when "Case sensitive" is clear, so that it
> matches "XYZ".  If I double click a search result containing "XYZ" then that
> word should automatically be selected, but it is not.

Fixed.

> 
> - After a search, it would be nice if I could press Esc to activate the Back
> button.

Done.

> 
> - There seems to be no way to access the Cancel Search button using the
> keyboard.  At the very least it should have a keyboard mnemonic.  It would be
> also be nice if I could press Esc to cancel a search.

Done.

> - When there is only a single match in a file you display "1 matches".  This
> should be "1 match".

Done.

> 
> - Even when the "regular expression" box is unchecked you launch grep with
> --regexp.  This doesn't seem right.

Fixed. Now regexp is handling perfectly

> 
> - Set the Folder: field to the findinfiles source directory, check "Regular
> expression" and set "Search phrase:" to
> 
>     vers*
> 
>   Now press Find to search.  gedit will hang, and the console will show

Fixed.

> - Open Find in Files, click the Folder: button and choose Music from the list. 
> Now enter a search phrase such as "xyz" and press Find.  The plugin will launch
> grep and will search in your home directory, not in the Music directory.

> - Click the Folder: button, then choose 'Other...'.  The resulting dialog box
> will say "Select a File".  I think this should be "Select a Folder".

I don't have control over this. This seems to be a gtk issue

> - I think the plugin should not search in gedit backup files (i.e. files whose
> names end with '~'), at least not by default.

Ok. I'm looking into this.

> - Perform a search that will run for a while, then press Back without
> cancelling the search.  Now the 'grep' process will still be running (you can
> see it if you run 'ps').  I think the plugin should kill the grep process in
> this situation.

I made some changes, it should kill the process but I can't reproduce this 
issue. please see if it still persists.

> - In your search interface it's not clear what I can enter in the Include: or
> Exclude: boxes.  Can I specify multiple patterns (such as '*.c *.h')?  If so,
> should they be separated by spaces, commas, or something else?  I think you
> should add tooltips or other documentation that clarifies this.

Ok. how do you think I should document this? tooltips

Anirudh, thanks for the update.  Would you like to attach a patch with your latest work?

Regarding your last question ("how do you think I should document this"), I think tooltips could be okay if the description is small - otherwise either a help page for your plugin (preferably in Mallard format to integrate with the existing gedit documentation) or a wiki page would be fine.

@Adam Dingle: I've found the latest code in Anirudh's bitbucket repo, until he attaches a full patch, you could use it too for testing: https://bitbucket.org/skyronic/findinfiles

Created attachment 278929 [details] [review]
Patch to add the findinfiles plugin to gedit master

Hi Adam, Robert,

I have attached a patch that applies cleanly to master at this time of writing and has a functional plugin. I have only used a tooltip and will write mallard documentation after the plugin has been accepted.

I've tested the plugin quite thoroughly and even dog-fooded it to do some refactoring for another project, and it worked quite well without any noticeable memory leaks. Let me know if there's anything else I can do to improve it before it gets merged with master.

Also, I had a lot of trouble getting the makefiles right, and it was simply by trial-and-error and copy-paste that I got it working, and even then I'm not 100% sure that it's actually going to work properly in all scenarios.

While I'm quite confident about the plugin itself I would appreciate a review of the makefile.

Hi Robert, Adam,

I just wanted to check if you had some time to look into this patch and whether it can be merged in to gedit. I've been using it for my own work for a little while and it seems to work pretty good.

The plugin can anyway be linked in the wiki, with a git repository hosted on github or gitorious for example.

<rant>
Python code is more difficult to maintain: no compilation, so no compilation warnings when using deprecated APIs. When doing some refactoring in gedit core, some functions may be removed, and grepping the Python code is more difficult than C. With a C function we have the namespace, the class name, and the method name (e.g. gtk_widget_show()). In Python, grepping a short method name like "show", "load" or "save" gives lots of false positives. I don't know why so many programmers like Python, it's primarily a scripting language, after all.
</rant>

Hi Sebastian,

Does linking the plugin on the wiki count as completion of this bug? If I can get a confirmation of this from either the bug reporter (Adam) or a gedit developer (You or Robert) I will go ahead and do it. But I was honestly hoping to get this merged into gedit master.

<response-to-rant>
I personally share your views on the benefits of statically typed languages in general, because of type safety and tooling benefits. But I primarily chose python because I'm more familiar with it than C and don't trust myself to do memory management properly. Plus, the repl is great for experimentation with unfamiliar APIs.

Although now in retrospect I could've chosen Vala.
</respone-to-rant>

The feature is useful for lots of people, so it makes sense to have it in the gedit-plugins module, and there are other Python plugins there. But if it is included, we expect bug fixes and maintenance work in the future. If the plugin is broken and nobody is working on it, we can remove the plugin in the worst case, and put it on github so someone can take the relay.

Sebastian,

I'm willing to maintain it, and I feel it's quite stable as a plugin. The only thing is I will fix a bug if someone cc'es me on a bug report. I don't want to monitor all the gedit bugs to check which ones are relevant here.

Adam, Robert,

I'm still waiting for your feedback on the last version of my patch. Can we declare this stable enough for production?

Anirudh, thanks for your latest patch, and sorry for the delay in reviewing.  I tried it out on Ubuntu 14.04 with GTK 3.13.3 and gedit from git master.  I didn't look at the source code.  Some feedback:

- The menu item "Find in files..." should be capitalized as "Find in Files..."

- As soon as I open the Find in Files pane, the console shows

sys:1: Warning: g_object_unref: assertion 'G_IS_OBJECT (object)' failed

- In the Find in Files pane, when the Folder is '(None)' the Find button should be grayed out.

- As I mentioned before, if the file browser sidebar is open, I think that Folder should automatically be set to the current directory in the sidebar.  That doesn't seem to be working: if I have the file browser set to ~/src, then when I open Find in File the current directory is '(None)'.

- Many searches fail to complete.  For example, search the gedit source directory for the string 'foo'.  The Find in Files pane displays "Searching" with a progress spinner that spins forever.  'ps' reveals that this process is running:

   grep -r -I --exclude-dir=.git --color=never --line-number --ignore-case --regexp=foo /home/adam/src/gedit

If I run that grep command from the command line, it completes instantly.  So the problem is not that grep is taking a long time to complete - instead, something must be going wrong in the reading of grep's output.

- I'd be inclined to display the search output in a fixed-width font for consistency with how text is displayed in gedit windows normally.

That's all for now.  If you'd like to address the issues above, I'd be happy to take another look.

Adam,

I've fixed allthe cosmetic changes. However I simply cannot replicate the issue with the hung git process. I tried replacing it on a variety of use cases.

My only alternative is to add a timeout to the grep process, but this will not work for 100% of scenarios and can have some false positives of grep processes that are still running.

Anirudh, it's good to hear you've made further progress.  Would you like to attach your latest work here?  If you do that, I could check whether I still see the hung grep process issue.  Thanks!

Created attachment 288820 [details] [review]
Searchmonkey Find in Files Integration in Gedit

The patch adds support for Find in Files to Gedit, through the integration of searchmonkey, not adding any dependencies. Neither searchmonkey nor any additional library are necessary.

Volodymyr,

I tried applying your patch to gedit master, but it wouldn't apply:

$ git apply 0001-SearchMonkey-Integrated-in-Gedit.patch
0001-SearchMonkey-Integrated-in-Gedit.patch:107: trailing whitespace.
	gedit-ff-systemio.c		
0001-SearchMonkey-Integrated-in-Gedit.patch:128: trailing whitespace.
void 
0001-SearchMonkey-Integrated-in-Gedit.patch:200: trailing whitespace.
    
0001-SearchMonkey-Integrated-in-Gedit.patch:435: trailing whitespace.
  
0001-SearchMonkey-Integrated-in-Gedit.patch:442: trailing whitespace.
    
error: patch failed: gedit/Makefile.am:124
error: gedit/Makefile.am: patch does not apply
error: patch failed: gedit/gedit-commands-search.c:562
error: gedit/gedit-commands-search.c: patch does not apply
...

Are you on some older version of gedit?  If so, can you rebase your patch onto the current master?  Thanks!

Hello,
I've started an implementation using native code instead of spawning an external helper.
The results are satisfying as of now (should be comparable to those of 'ag' or 'ack'), but there's definitely some place for optimization.
I'm currently trying to put together a nice UI but I'm no UX master, so I'd like to hear from someone more knowledgeable :) 
Also, do you need both keyword and regex ?

Thanks for the progress report.  Regular expression searches would certainly be nice, though I think they aren't essential for a first implementation.

In the end I managed to shove in Regular Expressions too and some other niceties, the UI tried to adhere to the latest gnome design guidelines (but I'm open to suggestions).
I've set up a github repo at https://github.com/LemonBoy/gedit-nemo
Consider it as a working demo, I just need to wire up some UI elements and split the monolithic file into smaller ones :)

LemonBoy, I just cloned your github repository and tried to build your plugin.  I got this:

cp: cannot stat ‘gedit-nemo.plugin’: No such file or directory
Makefile:2: recipe for target 'all' failed
make: *** [all] Error 1

Did you forget to add that file to the repo?  If you can add it I'll try again.

LemonBoy, thanks for adding gedit-nemo.plugin to the repo - now I was able to build and install.  I tried your plugin with gedit 3.14.3 built on Ubuntu 15.04.  In that version I don't see the option to enable the bottom panel at all - as far as I can tell there is only a side panel.  When I enable your plugin, its user interface appears in a separate window.

What version of gedit are you using for development?  I'm pretty sure the gedit maintainers will only be interested in a plugin that works with gedit master.

My fault, the latest commit should fix this problem (with a rather ugly workaround, the panel doesn't get shown when there are no visible child, hence the problem you've noticed)

OK - I just tried the latest build.  I'm seeing some surprising behavior, but I'm not in a classic GNOME environment: I'm running Ubuntu 15.04 with Unity.  So perhaps some of this is due to my own setup - I'm not sure.  Anyway, here's what I'm seeing:

- The "Find in files" dialog appears as soon as I start gedit.  I would expect that it wouldn't appear until I choose the "Search in files" menu item.

- When I do choose "Search in files", the "Find in files" dialog does not reappear.  Also, I see "Hello :D" on the command line; that looks like debugging code that should be taken out sooner or later.

- The "Find in files" dialog looks like a top-level window, not a dialog.  In my Ubuntu environment it has maximize, minimize and close buttons; I would expect it would have only a close button.  It has toolbar buttons, which are unusual in a dialog.  I'd expect this dialog to look more like gedit's Replace dialog, for example.

- Dialog titles and menu items generally use "header capitalization" in GNOME: every word is capitalized.  See the GNOME interface guidelines.  So I'd expect the dialog title to be "Find in Files", not "Find in files".

- Your menu item "Search in files" opens a dialog "Find in files".  Generally the menu command and dialog title are supposed to match, so you should pick one of these two wordings and use it everywhere.

- In the dialog you have labels "Find :" and "Where :" with a space between the word and the colon.  In GNOME dialogs there is not normally a space there; it should be "Find:".

- The label "It just works" is cute, but looks out of place in a GNOME dialog.  I doubt the gedit maintainers will want to keep that.

- When I perform a search, a new tab appears in the bottom pane, but is not focused: I have to click the tab to see its contents.  Instead it should get focus immediately.

- When I double click a search result in the bottom pane, nothing happens.  Instead I would expect this to open the corresponding file and move to the line with the search match.

Thanks for your comments, most of the code relative to the UI is full of debug silliness that is going to get removed once I settle with a definitive design.
I didn't catch the dialog being a top-level window since I don't have a Window Manager at all, I'll promptly change that.
My main concern right now is finding the right compromise between "easyness" and flexibility when specifying where to search: ideally you should let the user search into the open files, into an arbitrary directory (maybe specify some wildcards too ?)
A good source of inspiration here is Sublime Text [1][2]
The current approach is the most simplistic one, just ask the user for a single folder and for the search keyword and he's ready to go. The options are all hidden in the gear menu since I assumed those aren't modified often (maybe it's better to have those under the keyword entry box, don't really know).
I'd love to get some feedbacks on this matter :)

[1] http://104.131.103.243/wp-content/uploads/2014/09/SublimeFilterOptions.png
[2] http://www.macdrifter.com/uploads/2012/07/Screen%20Shot%2020120707_093219.jpg

@lemonboy: OK - I just pulled your latest changes and tried again.  Previously I hadn't noticed the search options (case sensitive, etc.) because they are behind what you call the "gear menu", though on my machine it looks like a "hamburger menu", i.e. three horizontal lines.  Again, I think it's odd to have a toolbar like this in a GNOME dialog.  I'd suggest putting the search options in the dialog below the "Where" field.  If you don't want them to be visible all the time, perhaps you could have an expander triangle that shows/hides them.

The latest commit should fix all the issues reported (Please see 'Bug 745253' for reference, the fix has already landed in master).
I'm currently adding some more features that have been suggested by the gedit developers themselves such as remembering the latest search location and a history of recent searches.

lemonboy, I tried your latest version.  It looks like you've made some good progress.  Here's some more feedback:

- The "Search in files" menu item should have an ellipsis since its command requires more input before it can be carried out.  Also, "files" should be capitalized: "Search in Files..."

- I personally think "Find in Files" would be a better name than "Search in Files", for consistency with the existing "Find..." and "Find and Replace..." menu items.

- Search in Files should have a keyboard shortcut.  (If it already has one, I can't find it.)  I'd suggest Shift+Ctrl+F.

- The labels in the Search in Files dialog should all have keyboard mnemonics so they can be activated via the keyboard.  In other words, when I press and hold the Alt key in the dialog I should see one letter underlined in each label, just like in the Find/Replace dialog, for instance.

- In the dialog, once again there should not be spaces before colons.  In other words, "Search for :" should be "Search for:".

- If the file browser plugin is active, then I think the default search directory should be the current file browser root.

- The bottom pane should show the text of each matching line.  I don't think there's any need to display the column number of the match.

- If there are several matches on a single line, I think the bottom pane should show that line only once.

- I think the bottom pane should show paths relative to the search directory.  In other words, if I'm searching in /home/adam/src/gedit then instead of '/home/adam/src/gedit/m4/threadlib.m4' it should just display 'm4/threadlib.m4'.

- When no matches are found, I think the bottom pane should display "No matches found", "Done" or something to that effect.  Currently it remains entirely empty, so I don't know whether the search is still running or not.

- If I select some text in a gedit window and then invoke Search in Files, the selected text should appear automatically in the 'Search for:' field.

I'll try to answer to all the points in the same order:

+ Done.

+ There's a subtle but substantial difference between the two wordings. [1]

+ Done.

+ Done.

+ Done (sorry for not doing this earlier).

+ Don't know if it's possible to talk to other plugins, but will consider this only if it doesn't require much code.

+ I've made some tests with the matching line shown in a column but it does look horrible and is very unhelpful.

+ Each match deserves it's own entry, if you click on one you gedit open the file and positions the cursor over the selected word.

+ Done (but not yet committed)

+ The search is completed once the stop button that pops up on the bottom-right corner disappears.

Thanks again for the useful feedback :)

[1] http://stackoverflow.com/questions/480811/semantic-difference-between-find-and-search

lemonboy, thanks for your latest changes.

- You added an ellipsis to "Search in files...", but the word "files" is still in lowercase.  Menu commands in GNOME programs are supposed to use header capitalization, so this should be "Search in Files...".

- Regarding "search" versus "find", the Stack Overflow page you referenced has a couple of different opinions.  In my opinion the most important consideration here is consistency with the Find... command, which looks for text in the current buffer.  Your functionality is similar but operates on a set of files.  So I still vote for Find.  I'd be curious to hear (and would be happy to defer to) the gedit developers' opinion about this.

- Both the Quick Open and External Tools plugins reference the file browser root, so you should be able to do the same.  Have a look at externaltools/tools/functions.py and quickopen/quickopen/__init__.py in the gedit source tree.

- I still strongly believe that each matching line should appear in the bottom pane.  Without that I would definitely not use this plugin: it's just too burdensome to have to click on each result in turn to see it in context.  Previous find in files plugins for gedit have worked exactly that way, by the way: see

http://oliver.github.io/gedit-file-search/

and

https://code.google.com/p/advanced-find/

- Similarly, I still think each matching line should appear just once.  grep works that way on the command line and I think it's the most natural way to present the search results.

- Thanks for pointing out that the search is complete once the stop button vanishes, but I don't think that's evident enough.  I still think you should display some text such as "No matches found" or "Done".

Hopefully this will be the last commit :) I've actually implemented all the features you've requested.

Lemonboy, thanks for this latest revision and for implementing many of my suggestions.  A few more comments:

- When I build your code, I see this warning:

  nemo-search-job.vala:288.21-288.56: warning: local variable `match_length' declared but never used
      var match_length = match_to - match_from;

- The first time I open the Search in Files dialog, I see this on the console:

   sys:1: Warning: g_object_unref: assertion 'G_IS_OBJECT (object)' failed

- In the results pane, it would be nice to visually distinguish the filenames from the lines you display.  The filenames could be bold or in a different color, for example.

- In my opinion the column with the line number of each match is visually distracting. I personally would prefer output that has line numbers at the left, like this (if I searched for "priority"):

gedit/gedit-small-button.c  (2 hits)
  60:          GTK_STYLE_PROVIDER_PRIORITY_APPLICATION);
  113:         GTK_STYLE_PROVIDER_PRIORITY_APPLICATION);
  
gedit/gedit-document.c  (1 hit)
  1160:              G_PRIORITY_DEFAULT,

and so on.

- It would certainly be nice to highlight the text I searched for both in the results pane and in the main window if feasible.

- In my opinion it's awkward that in the results pane all the triangles beside filenames are closed by default.  That means I have to click each triangle in turn to see the list of matches in that file.  The problem is that often I want to visually scan through all the search results to look at all the lines at once.  I think they should all be open by default.

- The message "No result found" sounds a little unnatural.  I'd say "No results found" instead.

Also, I think the gedit developers won't consider taking this unless you turn it into a patch to either the gedit or gedit-plugins source tree (depending on which source tarball this should go into) in git format-patch format.  I don't know whether you want to take that on now, but I think you'll have to sooner or later.  I also doubt they'll want to the plugin to be called "Nemo" - for consistency with the other gedit plugins they'll want it to be called "Find in Files" or something like that, I believe.

The previous version highlighted the selected match once clicked, but that's not possible anymore since each result stands for one line (no, there's no API to highlight arbitrary portions of text)

I couldn't come up with a nice and clean way to highlight the needle in the TreeView results, this can be added at a later time.

All your other nitpicks have been addressed, I hope we've finally reached the end.

Over and out

Lemonboy, thanks for your latest changes.  This is starting to look pretty nice.

- Use your plugin to search for any string, then right click in the output panel and choose Close.  Now perform another search.  The output panel will not reappear.  (The same thing happens if I search for the same string twice in a row.)

- Currently I must double click any line in the search results to navigate to it in the main window.  I think a single click would be more pleasant for the user - the double click just seems unnecessary.

- When a document has a single match, you display "1 hits".  That should be "1 hit".

- Regarding highlighting each match in the main window, you wrote that "there's no API to highlight arbitrary portions of text".  You could simply select the matching text, or just select the entire line.  I think either of those would make it much easier for the user to see the text that was matched.

- In the gedit source tree, search for the regular expression "celor??".  You'll see that the matches include this line from po/eo.po:

  msgstr "_Forigendaj celoj:"

  That should not match this regular expression.

Actually I take back my comment about the regular expression "celor??".  I realized that ? matches zero or one occurrences of whatever precedes it, so "celor??" should actually match "celo" as found in the "msgstr" line above.

+ The panel was intended to hold a single tab for each plugin.
The current use is a clear abuse of the intended usage and I'm trying to coordinate a solution with the upstream.

+ Single click grabs the focus, let's try to adhere to the Rule of Least Surprise (moreover, it'd be hard to grab the focus without opening a undesired file).
Anyway you can also use your keyboard to scroll the list and then open the selected file by pressing enter.

+ Multiple selections aren't possible and, in my opinion, highlighting the whole line is no better than what it does now.

+ I went nuts trying to understand what the problem was :)

Why not just use a single tab for this plugin and reuse it for every new search?  I think that would be completely fine, and it sounds like it would make the implementation easier.

I would still prefer a single click for opening files and I still think that highlighting the whole matching line would be nice, but if you disagree we can always see what the gedit maintainers think.  As I mentioned before, you'll probably need to transform your changes into a patch to gedit (or gedit-plugins) before they'll look at your code anyway.

The latest commits should fix all the remaining issues :)

Lemonboy, thanks for your latest changes.  I tried your latest version and it looks OK.  I would still prefer a couple of user interface differences as described above (single clicking, whole line highlighting) but I think this is basically usable, so I'll let the gedit maintainers review from here (I tried your plugin but haven't looked at your code).  Thanks again for your work on this.

By the way, I would actively prefer a single tab that gets reused for each search rather than a separate tab per search, since I virtually never want to look at two different searches and it's a pain to have to close a tab after every search to prevent lots of them from piling up.  Again, apparently you feel differently so we'll see what the gedit maintainers have to say.

The plugin has just been merged into the official gedit-plugins repository and has been blessed by the gedit developers.

The commit:
https://git.gnome.org/browse/gedit-plugins/commit/?id=b93aaed962ed42a7d896191add5f9b8bd2daa4df