GNOME Bugzilla – Bug 172099
Performances: it takes too long to open big files
Last modified: 2020-11-24 09:59:44 UTC
Let's take an example: I have 20+ Mb log files to analise, opening them with gedit takes so much. Over 3 minutes for a single file, when e.g. with UltraEdit it takes just 10 seconds or so. This is an important feature to fix to improve gedit performances with big and also smaller files. Other information: Take a comparison with Anjuta/Scintilla, it's definitely quickier in opening big files than gedit. my linux box: kernel 2.6.10, 2GHz processor, 398 Mb ram, 1Gb swap. gedit 2.8.3
How do you measure how much time gedit spent opening the file? Are you able to read/navigate the file while gedit is "loading" it?
Hi, I noticed it wasn't a problem of a big-buffer loading and displaying time, but a problem with the scrollbar. I've got this big text file to open max@darkslack:/tmp$ ls -alh big_file -rw-r--r-- 1 max users 26M Jul 22 18:16 big_file on a gnome-terminal tab I launch top [so I can monitor the cpu-time usage], in another tab gedit big_file. Process starts up, and text is displayed. All the 26MB are displayed I suppose, coz if I ctrl+end I reach the end of the loaded file with a previously marked line "hey this is the end of the file". CPU usage is: about 30% gedit, 60% X Switching back on gedit window: I note that if I move the right scrollbar to the middle of its height [as if I have to read something in the middle of the file] it moves up first faster than slower [as if the file would be still loading]. After a 3-4 minutes the cpu comes to the normal values and the scrollbar stops doing crazy things.
which theme are you using? can you try with different themes, especially with the old but reliable themes like the "simple" etc
Hi, I've tried with "Simple", "Mist, "Nuvola" [everytime logging out from X session and relogging] but I see the same behaviour on gedit. My default theme is "Dropline Fun", which comes with dropline-slackware gnome. Don't you see anything abnormal opening big files? I don't have particular processes cpu-consuming opened. I'm on gedit 2.10.3 and slack 10.1 right now. When I posted this bug I was with the 2.8.3 and with a slackware 10.
This is something that would be interesting to investigate further, if it's really the scrollbar, it may well be a GtkTextView issue. Massimo, do I ask too much if I ask to give cvs HEAD (or 2.13.0) a try? We changed the way local files are loaded and at least from the gedit point of view things should be way faster. Running sysprof would also be very interesting. I'll try to that myself.
Created attachment 56022 [details] sysprof sreenie Quick sysprof run. A quick try didn't turn up any scrollbar misbehavior, it looks like all the time is spent in pango, layouting the text in TextView. It was a qick measure though, if someone wants to try to profile more accurately it would be great.
btw, for a 15 MB file I have around here I get, cvs HEAD takes about 12 seconds to load (though strangely enough the cpu stays busy for a while even after finishing load)
paollo, could you please attach the sysprof output file ?
sure. I throw it away, but I can easily remake it. Give me five minutes :)
Created attachment 56033 [details] fileopen.sysprof.tar.gz here it is (.tar.gz)
btw, I don't see a reson to spam usability people here, removing them from CC.
Hi, I downloaded and compiled from cvs gedit 2.13.0, got also gtksourceview 1.2.0. This time I have a file 21M Dec 15 20:47 big_file.txt and I tried to open it: it gives me the same problems, cpu stands at 100% for too much and the scrollbar is still crazy. I can give you a sysprof profile tomorrow in the afternoon, i hope :) btw: you can let me in cc if you want, I would like to squash this bug :)
CPU standing at 100% for a few time is not a problem if you can navigate and edit the file. It is due to GtkTextView rendering text in the backround (to confirm it, please try to reproduce it using the test-text program in gtk+/tests). DO you have the crazy behavior for the scroolbar even if you don't touch gedit while the file is loading? Can you reproduce the same behavior using the texttext program?
Created attachment 56072 [details] sysprof output for gedit opening a big file. The tested file was 39 MB.
Answering to comment #13: 1. yes, using the GtkTextView rendering text in gtk+/tests/testtext gives the same behaviour: the buffer is loaded quickly but the scrollbar stands moving for a lot of time. 2. No, the scrollbar remains stable if I don't touch gedit. So at this point it doesn't seem to be a gedit bug, but a GtkTextView problem... I tried to open the big_file with emacs and it doesn't suffer of anything, nor the scrollbar problem nor the cpu time one. It's instantaneous.
uhm... Massimo's sysprof output doesn't seem particularly enlighting (btw, the attachemnt is tar.bz2 if someone has difficulties in opening it). Massimo are you running sysprof on an install without debugging symbols? I don't think that comparing to emacs is fair, emacs uses a different data structure to hold the text, with is pro and cons and also doesn't use pango to draw the text. As long as the text is readable/writable right away, I think it's normal that GtkTextView keeps the cpu busy for a while since it is laying out the text in an asyncronous manner. So I guess that what remains to understand is the scrollbar behavior... once again it's normal that the scrollbar keeps being resized while the text is laying out since gtk doesn't know a priory the height of the text until it finishes laying out, however 1) I understand it can be annoying (if someone has better ideas...) 2) It can very well be doing something stupid which causes the scrollbar to be redrawed/moved to often Btw, Massimo feel free to drop by in #gedit (and #gnome-it ;) on irc.gnome.org if you want to chat about this issue without going through the bugzilla overhead
hmm last message 2005-12-16 15:14 UTC, its 2007 and I still see this bug
I have to confirm this (good old) bug.I must deal with big txt files too. At the very begining, this bug made me switch to kde.I gave to gnome 2.18 a try yesterday, had the same bug. The same solution applies. I'm so sorry to throw away (each) gnome (version) because of this bug. 46Mo text file is opened with kwrite in 9s. Gedit is taking 3min and 30s! This bug is critical.
Hi Massimo! Could you try to run valgrind --tool=callgrind on the test/testtext program of GTK+ which this large file? (And install debugging symbols of Gtk+, glib, pango). Maybe this gives some clue. Thanks!
The GtkTextView widget is based on the Tk widget, right? Maybe this is related then: http://www.tcl.tk/cgi-bin/tct/tip/155.html
*** Bug 489356 has been marked as a duplicate of this bug. ***
*** Bug 496434 has been marked as a duplicate of this bug. ***
*** Bug 486393 has been marked as a duplicate of this bug. ***
*** Bug 525964 has been marked as a duplicate of this bug. ***
I'm unable to open up SQL dumps in gedit because it seems to eat all of the RAM I have available (3GB?!), and then it just seems to hang there and fill up my swap file. The large files I'm dealing with are a bit more than 26MB. (Try up to 768MB.) I think in the case of large files, gedit should ask whether it should enable syntax highlighting, or disable it. It seems to me like that would shave down some time, but that still does not help me enough to be able to open large files in gedit. Also, I see no point to loading the entire thing in the RAM right away.
Personally I have to deal daily with text files that are even several gigabytes in size. I am completely unable to use gedit because of this bug, and have to resort to other editors. The problem for me is not CPU or memory usage. It is that I most usually start working from the end of the file so I would have to wait for even 30+ minutes for the file to load completely. (Just wanted to tell you guys that there are more than just the earlier commenters that are affected.)
*** Bug 566590 has been marked as a duplicate of this bug. ***
(In reply to comment #26) > Personally I have to deal daily with text files that are even several gigabytes > in size. I am completely unable to use gedit because of this bug, and have to > resort to other editors. The problem for me is not CPU or memory usage. It is > that I most usually start working from the end of the file so I would have to > wait for even 30+ minutes for the file to load completely. > > (Just wanted to tell you guys that there are more than just the earlier > commenters that are affected.) > yeah, I've definitely switched to alternatives editors. This bug makes gedit unusable on most text files I open.
This problem of gedit taking a very long time to open and meander around files in the tens of megabytes or larger is confirmed in Ubuntu 10.04: lsb_release -rd Description: Ubuntu 10.04.1 LTS Release: 10.04 apt-cache policy gedit gedit: Installed: 2.30.3-0ubuntu0.1 Candidate: 2.30.3-0ubuntu0.1 Version table: *** 2.30.3-0ubuntu0.1 0 500 http://us.archive.ubuntu.com/ubuntu/ lucid-updates/main Packages 100 /var/lib/dpkg/status 2.30.0git20100413-0ubuntu1 0 500 http://us.archive.ubuntu.com/ubuntu/ lucid/main Packages I am using a Toshiba Satellite L305D-S5934 laptop, seems not a hardware resource issue. Downstream bug may be found at: https://bugs.launchpad.net/ubuntu/+source/gedit/+bug/620780
I have this bug for a long time (currently I use gedit 2.30.3 on Gnome 2.32.0). gedit is very slow to open (not so) big plain text files of approximately 5 or 6 MB. gedit freezes for a while if I try to move the right scrollbar.
Created attachment 206010 [details] Text file that has very long rows and causes gedit to grind to a halt when viewed with word-wrap off View this text file in gedit with word-wrap off and it causes gedit to hang for 30+ seconds repeatedly.
Created attachment 206026 [details] attachment attempt 2: zip file of text file example that has very long rows. This 5MB text file causes gedit massive delay problems when word wrap is off This is a correction to the previous attachment attempt. This text file is only 5MB, but has very long lines (approximately a 200000 characters per line). If it is viewed in gedit 2.30.4, with word-wrap OFF, then scrolling around the text file causes gedit to become unresponsive for 10-30 seconds each time. I am using a ubuntu 11.4, 64 bit, with 11GB ram, so I think a 5MB text-file of this kind should not cause these problems. Looking at the other comments in this thread, common elements seem to be the long lines, and word-wrap off.
This certainly seems to be related to bug #127840 (gtktextview uses huge amounts of memory on large files), if not an outright duplicate.
Is this fixed? Any plans to workaround or remove these limitations?
(In reply to comment #34) > Is this fixed? Any plans to workaround or remove these limitations? yes, the workaround is: use UltraEdit editor. There's no hope to see improvements on Gedit on this task, unfortunately.
Though, we'd love to see alternate, faster GTK widget. It's the rendering that poses the problem.
*** Bug 722060 has been marked as a duplicate of this bug. ***
*** Bug 737342 has been marked as a duplicate of this bug. ***
(In reply to comment #33) > This certainly seems to be related to bug #127840 (gtktextview uses huge > amounts of memory on large files), if not an outright duplicate. the memory usage in gedit (tested with 3.12) is the same as in geany, the embarrassing difference is that geany load a 100MB file in less than a second while gedit take several minutes (with 100% cpu usage). I think the severity of this bug reported should not be "Normal"
Is someone working on this? I am interested in working on this bug. Can someone in the gedit maintainer list reply to give an overview about this problem? Its been 9 years since the ticket was opened.
hi, The problem is gtktextbuffer itself which must be totally re-coded and probably in the same process, it make sense to re-code all the text stack, so many months of hard work. First, a overview of what we need and it's general design ( with comparaison with the concurrents ) must be done, a new API must be created etc.... It's more a work for gtk+ 4.0
I'm sure performances can be improved without rewriting entirely GtkTextView :-) The first thing is to write benchmarks. After that, the bug #724247 would be a good starting point, to disable other features when a file is loading. Currently features like bracket matching is still enabled during a file loading. Another thing that should be disabled is updating the cursor position in the gedit statusbar. When loading a big file you'll see the "Ln" and "Col" being updated continuously. Also, it's interesting to see the difference between using GtkSourceFileLoader and the "quick and dirty" code that loads a file into an entire string and use gtk_text_buffer_set_text() with that string.
*** Bug 743415 has been marked as a duplicate of this bug. ***
*** Bug 783235 has been marked as a duplicate of this bug. ***
Created attachment 357514 [details] [review] possible patch for lazy initializing the GtkSourceMap
Hi, I investigated this bug and I found 2 big time consumers: 1) The gedit-view-frame contains a GtkSourceMap. This map is connected to the main view even if it's not visible (https://github.com/GNOME/gedit/blob/master/gedit/resources/ui/gedit-view-frame.ui#L34 -> which leads to the execution of the following line https://github.com/GNOME/gtksourceview/blob/master/gtksourceview/gtksourcemap.c#L1232). The text rendering seems to be done in the same way for the main view and for the GtkSourceMap. So this doubles the processing time even if the GtkSourceMap is never shown. A solution would be to lazy initialize the GtkSourceMap (I attached a possible patch for that). 2) There are 2 important async operations: - loading the file from the disk and building the GtkTextBTree - rendering the GtkTextBTree After the file has been completely loaded, Gedit executes: https://github.com/GNOME/gedit/blob/master/gedit/gedit-tab.c#L507 which leads to https://github.com/GNOME/gtk/blob/master/gtk/gtktextview.c#L2927 which invalidates the entire tree that has been built so far: https://github.com/GNOME/gtk/blob/master/gtk/gtktextlayout.c#L360 . Usually, if I load a 600 000 lines file, at this point 200 000 lines would have already been loaded. Here they are all invalidated and rerendered. I tried to comment https://github.com/GNOME/gtk/blob/master/gtk/gtktextview.c#L2927 and everything seemed to work correctly. So Maybe this would be a solution. But I'm not sure. It may affect other things.
I'll have a look.
Here's a comparison with other editors: https://github.com/jhallen/joes-sandbox/blob/master/editor-perf/readme.md
Review of attachment 357514 [details] [review]: Thank you for this patch! There are a few issues though before a proper review can take place: Firstly, GtkSourceView is distributed as a separate project to gedit (you'll need to open this bug and attach the patch to it over there). Secondly, it doesn't maintain the same indentation style of the rest of the GtkSourceView source code (which is a tab character for every indentation level). And thirdly, commenting out in a patch isn't a good idea as it makes the code look untidy, i.e., your: //problematic //connect_view(...); <- why not just remove it then? The reviewer can see what you've removed in the diff.
Created attachment 358024 [details] [review] Connect GTKSourceMap to GeditViewFrame only when the GTKSourceMap is visible
Thank you for the review ! I created this bug for GTKSourceView: https://bugzilla.gnome.org/show_bug.cgi?id=786504. As a result of the suggestions from that bug I propose a new patch for gedit. As a side note, there is a problem even after applying the patch: If you open gedit and load a file, there will be a 2x increase in performance. If you open gedit, show the map, hide the map and then load a file, the performance will be the same as before. Either there is a bug in the disconnect_view function or I'm missing something. I will investigate it.
It's just as bad with 44KB html file. Brought my powerful laptop to its knees. Can't it be changed to something more efficient?
Created attachment 372658 [details] 44KB hmtl file that brings gedit to its knees
Mass-closing of all gedit bugzilla tickets. Special "code" to find again all those gedit bugzilla tickets that were open before the mass-closing: 2bfe1b0590a78457e1f1a6a90fb975f5878cb60064ccfe1d7db76ca0da52f0f3 By searching the above sha256sum in bugzilla, the gedit contributors can find again the tickets. We may be interested to do so when we work on a specific area of the code, to at least know the known problems and possible enhancements. We do this mass-closing because bugzilla.gnome.org is being replaced by gitlab.gnome.org.