GNOME Bugzilla – Bug 171215
Support lost partition recovery
Last modified: 2011-02-15 19:32:49 UTC
It would be great if gparted will support lost partition recovery using gpart: http://www.stud.uni-hannover.de/user/76201/gpart/
Sorry, probably the actively developed command line tools is no more gpart but testdisk http://www.cgsecurity.org/index.html?testdisk.html
Great ideas, but you should probably read my latest entry on the newspage ( http://gparted.sourceforge.net/news.php ) nofi :)
Yea, GUI for recovering (finding) lost partition would be very usefull. Several times gpart saved my a life ;)
Created attachment 164487 [details] [review] Patch for add support for gpart
Created attachment 164488 [details] Dialog to restore old partition table This is a ubuntu 9.10 System. When gpart finds a ext3/4 partition, gparted draws it as ext2 (See first partition)
Created attachment 164489 [details] The result of restoring a partition table with ext4 FS This is the result of restoring an old partition table. Ext4 is recognized without problems.
Hi, I made a patch that adds a gui to gpart in gparted. Gpart is an old program with some limitations, and probably writting a GUI for TestDisk could be preferable, but I chose Gpart over TestDisk because the last one is an interactive tool, ie the user must control it with the arrows and the enter key. So, writting a GUI for this program is very hard, but writting a GUI to Gpart is an easy task. The problem is that gpart doesn't have direct support for ext3 and ext4, which are probably the most used file systems in linux. If you try my patch, you will see that gpart recognizes the ext3/4 partitions like ext2, and gparted shows an ext2 in the visual representation of the disk in the dialog (See attached screenshots). However, gpart only writes in the MBR without making any changes in the partitions, then, when the changes are written, the partition is entirely ext3/4 and your OS will recognize it and mount it without problems. I hope will be useful. Regards, Joan.
Thank you Joan for taking the time to contribute to GParted. This new option to "Restore old partition table" I have successfully compiled and tested the patch on Ubuntu 8.04, 9.10, and 10.04. In my testing I have noticed some anomalies that might be limitations of the gpart command. These anomalies are as follows: A) Extended partitions appear to be ignored, instead logical partitions are identified as primary partitions. B) When the partition table search is performed on an uninitialized drive, if I choose "Cancel" to not restore a partition table then GParted crashes. An uninitialized drive can be created in RAM with a command such as the following: sudo modprobe scsi_debug dev_size_mb=100 sector_size=512 C) The "Restore old partition table" feature does not appear to be able to identify partition tables on disk devices that have sector sizes greater than 512 bytes. My guess is that this is likely a limitation of the gpart command. Many GParted users have been requesting improved feedback on the progress of long running commands. I have some ideas on how to improve this in the Dialog_Progress code, but have yet to implement my ideas. Do you have some ideas on how to improve feedback on progress while the gpart is running?
Hi Curtis, it's true gpart don't restores extended partitions, I think this is because gpart only recognizes filesystems, therefore, partitions with no filesystems like the extended are ignored. And, when gpart finds a filesystem, I don't know if it is able to know if is logical or primary. Anyway, the main purpose of gpart is to recognize lost partitions in order to make accessible to the user to mount and make backups or recovery data that in other way it would be lost. And it fulfills correctly. There are many people who knows gparted but there's less people who knows gpart, then, this new feature can save their lives :) Maybe we have to rename the menu->device option to "Search for old filesystems" what is probably more exact. don't you think? I tried in my debian the command "modprobe scsi_debug dev_size_mb=100 sector_size=512" but it returns an error saying I have not installed the module scsi_debug.ko, what package I need to install? I think I know why gparted crashes where you try to work with an uninitialized device. I'm moving now and probably can't work on this in the next days. I will try to send a new patch in the next week.
If gpart only recognizes file systems and not extended partitions, I wonder what it does when it encounters more than 4 file systems? > Maybe we have to rename the menu->device option to "Search for old > filesystems" what is probably more exact. don't you think? I think I need to understand gpart better to properly address this suggestion. If we consider gpart as a way for users to "search for old filesystems", then users might be surprised when their partition table is overwritten. It might be useful to provide users a way to backup and restore partition tables. That way they could experiment with the feature. Another alternative would be to search for the old file system, and then use libparted to create a new entry in the already existing partition table. This might be a better approach to "searching for old filesystems". That is provided that gparted would provide the necessary details to us. > I tried in my debian the command "modprobe scsi_debug dev_size_mb=100 > sector_size=512" but it returns an error saying I have not installed the > module scsi_debug.ko, what package I need to install? Following is a link to the documentation I referenced regarding scsi_debug: http://sg.danny.cz/sg/sdebug26.html On Ubuntu 10.04, I think scsi_debug is accessible by default. I do not recall installing a package to gain access to the scsi_debug module. I am not sure which package in Debian would provide this module. If needed I can perform further testing to isolate the problem area where the program crash occurs.
> If needed I can perform further testing to isolate the problem area where the > program crash occurs. Hi Curtis, I'll fix the error that makes GParted crash, I think I have it controlled. I am more worried about what will happen when gpart will find more of 4 partitions. I would wish make the test to check it. > I think I need to understand gpart better to properly address this suggestion. > It might be useful to provide users a way to backup and restore partition > tables. That way they could experiment with the feature This is a good idea. Ok, if gpart actually only writes the MBR, GParted only needs to save the first 512 bytes of the disk (MBR) or in any case the 64 bytes that contain the partition table, and re-write it to disk in the case anything goes wrong. This is easy to program. > Another alternative would be to search for the old file system, and then use > libparted to create a new entry in the already existing partition table. This > might be a better approach to "searching for old filesystems". That is > provided that gparted would provide the necessary details to us. I don't know how to program this, I would need investigation to program this feature, therefore I see more viable the first alternative. So I suggest that we first fix the current bugs in the patch, and then decide if we change the name of the menu option or we make a backup of the the MBR, etc.
Created attachment 164829 [details] [review] Patch for add support for gpart
Hi, I updated the patch fixing the crash. By the way I fixed a bug that exists in the dialog, now, when gpart don't find any partition don't shows an empty disk, it displays an error message.
I hope to test this patch soon. Another thing I noticed when testing the initial patch from comment #4 was that gpart did not appear to detect file systems on devices with sector sizes greater than 512 bytes. I was testing using the scsi_debug module with the sector_size set to 4096 bytes. I had created a single ext2 file system. I did not run gpart directly, but rather found that no file systems or partitions were found when using the "Restore old partition table" menu entry was used. Since 512 byte sector sizes have been the norm for the last three decades, I suspect that gpart does not contain code to handle larger sector sizes. This is just a guess though.
Taking a look at the source code fo gpart, You can find this defines in gpart.h #define MINSSIZE (512) /* min. sector size */ #define MAXSSIZE (16384) Then, it seems that gpart supports up to 16384 sector size. Actually, if you see the help of gpart: Usage: gpart [options] device Options: [-b <backup MBR>][-C c,h,s][-c][-d][-E][-e][-f][-g][-h][-i] [-K <last sector>][-k <# of sectors>][-L][-l <log file>] [-n <increment>][-q][-s <sector-size>][-t <module-name>] [-V][-v][-W <device>][-w <module-name,weight>] gpart v0.1h (c) 1999-2001 Michail Brzitwa <michail@brzitwa.de>. Guess PC-type hard disk partitions. There's a -s option to especify the sector size. Have you tried it? If it works, we simply have to modify the patch and make a customized -s option to make it work.
Unfortunately a serious bug has been found in the 0.6.0 release so I won't be able to look at this new enhancement request for a while. See Bug #623630. I would be more inclined to invest more effort investigating this feature if the patch could be enhanced to do one (or both) of the following: A) Restore entire partition tables including all partitions (primary, extended, and logical). The limit of only restoring 4 partitions as primary partitions could cause much grief for both our users, and for us in our support forums. B) Restore a lost partition without losing the existing partitions in the partition table. This would require using gpart to find lost partitions that did not overlap with existing partitions. Then the libparted library could be used to add the lost partition back into the partition table.
Hi Curtis, > A) Restore entire partition tables including all partitions > (primary, extended, and logical). I don't know right now what is the behavour of gpart when finds logical partitions. I don't have the means at this moment to try it, but in any case, we can't make anything to fix this (except modify gpart). > The limit of only restoring 4 partitions as primary partitions could cause > much grief for both our users, and for us in our support forums. I think when people uses gpart is because they have lost their partition table, this is, their MBR, in that situation any help is good, the other option is lost all partitions. All functions of Gparted are potentially destructive if you don't go with care or you don't know what are you doing. > B) Restore a lost partition without losing the existing partitions in the > partition table. > This would require using gpart to find lost partitions that did not overlap > with existing partitions. Then the libparted library could be used to add the > lost partition back into the partition table. Ok, this is a good idea and I can write it. Gparted can use gpart only for detect partition table but using libparted to write it. But this can mean a lot of work, I will work on this and send a patch in the next days.
Created attachment 165758 [details] [review] Gpart support for Gparted Hi, here's the patch
Created attachment 165769 [details] Trial Restore of two logical partitions This patch is looking very promising. I particularly like the ability to select which partitions to restore. :-) Since I have been testing a problem with move operations overwriting the EBR for a following logical partition, I thought I would try restoring two logical partitions. Originally, the disk was set up as described in the following link: https://bugzilla.gnome.org/show_bug.cgi?id=623630#c5 Basically the layout is as follows: user@lucid:~$ sudo parted /dev/sdc unit s print Model: Linux scsi_debug (scsi) Disk /dev/sdc: 204800s Sector size (logical/physical): 512B/512B Partition Table: msdos Number Start End Size Type File system Flags 1 2048s 204799s 202752s extended 5 4096s 20479s 16384s logical ext2 7 22528s 120831s 98304s logical fat32 6 122880s 204799s 81920s logical ext4 user@lucid:~$ The fat32 partition (/dev/sdc7) is 48.0 MiB in size because there is 1 MiB of unallocated space in front of and after this partition for Extended Boot Records. The ext4 partition (/dev/sdc6) is 40.0 MiB in size. As can be seen by the attached picture, the gpart enhancement/patch found both partitions. For some reason the fat32 partition is listed as 51 MiB, but this might be due to problems accounting for the unallocated space used for EBR's. I clicked to restore both partitions, however, neither partition was restored. I suspect that this is because these partitions are logical partitions and not logical partitions. I also tried this using primary partitions aligned to MiB: /dev/sdc1 - 8 MiB ext2 /dev/sdc3 - 51 MiB fat32 /dev/sdc2 - 40 MiB ext4 and both partitions were discovered properly as 51 MiB fat32 and 40 MiB ext4. Unfortunately upon restore these partitions appear to have been aligned differently (perhaps to cylinder alignment rather than MiB alignment). Hence GParted no longer recognizes the partition type. Later I will test using cylinder alignment only.
Hi, I not been able to reproduce the error,maybe it can be because I can't align to MiB, this option appears disabled in gparted, need I to install something? I think something: if you run 'gpart /dev/sdc' on the console, its output shows you exactly in what sector starts and ends the partitions, so you can check if is correct and if it can be due to the EBR. All both test have differents results in my computer, the first detects only the extended partition, and the second detects all properly and restore it without problems. Note I working with a device of 80MB, the scsi:_debug module with dev_size=100 don't works in my pc. Making this tests, I find some errors in this patch 1) Working whit extended partitions, the result is unpredictable, Gpart can detect an extended partitions and, when libparted restores it, gparted detects a fat, an ext2 or something. And I observe an interesting thing: When gpart finds an extended partition, and restores it exactly at the byte on it starts, all the logical partitions inside are restored also. It is because the extended partition has its EBR's unchanged. With the last modification of the patch, that prevents to use gpart to restore partitions and uses libparted instead, gparted wins the possibility to choose partitions to restore, but lost the ability to work with extended partitions, and I don't know why. 2) Sometimes gpart founds inconsistences in the partition table, then, I programmed the patch to abort the process, but I forgot to show any kind of error, therefore, gparted shows a dialog empty with only one "cancel" button. We have to decide if gpart shows an error or try to restore the partitions with inconsistences. I will work this days in the error 1), I hope it have a solution :)
I think I found a bug in bugzilla :), In the message above, I don't write any link in the word "patch" regards.
> Hi, I not been able to reproduce the error,maybe it can be because I can't > align to MiB, this option appears disabled in gparted, need I to install > something? Due to a bug #623630, I have temporarily disabled "Align to MiB". The relevant git commit is: http://git.gnome.org/browse/gparted/commit/?id=1e1ea1f09cd0bb15ed156d9dc018e779160588d1 I am working to fix this problem and hopefully will have a solution and re-enable "Align to MiB" soon.
I manage to fix the bugs that I was found in my previous post. Were caused by improper use of the constraints of libparted. But now i got an error with extended partitions. As I said before, when gpart finds an extended partition, it restores it with its logical partitions inside, but when Gparted restores it using libparted, a empty extended partition is created, I make sure the restored partition begins and ends exactly at the same sectors, however the new partition is empty. I think when libparted creates a new extended partition, probably writes a new EBR, and erases the old logical partitions. I wonder if exist any way to create an extended partition using libparted that doesn't write a new EBR. Do you know?
> I wonder if exist any way to create an extended partition using > libparted that doesn't write a new EBR. Do you know? I do not know if there is a way. Parted does have a rescue command, but I think this is for recovering file systems with recognizable signatures and not for extended partitions, but I have not investigated this thoroughly. Regarding "Align to MiB", I have added patches to the git repository to fix bug #623630 and re-enable "Align to MiB".
I found the way to create a extended partitions with parted without writting the EBR and restoring all its logical partitions. The only limitation is the extended partition must be restored completely with all of its partitions, there is no choice. To restore a extended partition with parted, you must create it as primary without fs_type, and then write its type manually in the MBR, in the exact byte. I'm working to control errors and make a solution in case of incosistences in gpart guessed partition table. I will send a patch in the next days.
Created attachment 166800 [details] [review] Gpart support for Gparted Hey, there is the new patch. Now, the extended partitions appears like a one more primary partition, and you can restore it with its logicals partitions inside. As I say in my previous post, for restore extended partitions, Gparted writes manually on the MBR, writes the byte 0x05 in the exact position of MBR. That is dangerous and should be thoroughly tested. And this implies other problem: This code works on msdos disklabels, OK, but I don't know if it can damage other disklabels. I do not have enough knowledge about disklabels to know what will happen. One question: All disklabels use the same structure in MBR? this structure? http://en.wikipedia.org/wiki/Master_boot_record If anything goes wrong, all disklabels that uses that structure will not be damaged.
> One question: All disklabels use the same structure in MBR? > this structure? http://en.wikipedia.org/wiki/Master_boot_record The structure of other partition tables is quite different. For an example of another popular partition table, you can read up on the GUID Partition Table (GPT) at the following link: http://en.wikipedia.org/wiki/GUID_Partition_Table GUID Partition Tables do not use "logical partitions" at all. Some other partition tables are totally incompatible with MS-DOS Partition Tables (e.g, MBR). Hence we cannot assume that the partition table is always MS-DOS. The libparted library tries to determine the partition table type. GParted stores this information in the Device class in the "disktype" member. You might wish to check this value to prevent a user from accidentally overwriting a valid non-MS-DOS partition table.
Ok, but the first 512 bytes of the disc are the same?, Gparted writes one byte of the MBR, only when detects a partition as extended. Then, if gpt don't have logicals partitions, Gparted never writes in the MBR. In any case, gpart only dectects 4 partitions, maybe we have set the restriction to work only with ms-dos partition tables, and disable this functionallity for other disklabels.
> Ok, but the first 512 bytes of the disc are the same?, Unfortunately this is not guaranteed to be the case. There is a partition type type that creates a hybrid partition table (both GPT and MS-DOS). It is called EFI and is used by Intel based Macintosh computers. Other partition tables are not guaranteed to leave the first 512 bytes of disc alone. I think your last suggestion to limit gpart to work on MS-DOS partition tables only is a good idea. :-)
Ok, then I will write the limitation and send a new patch.
Created attachment 166879 [details] [review] Gpart support for Gparted Here is the patch
Thanks for the new patch. I have done some testing and noticed some potential areas to improve this enhancement. 1) Permit restoration of an entire ms-dos partition table if the partition table is unknown I simulated this by creating an MS-DOS partition table and a few partitions within. Then I wiped out the partition table with the following command: sudo dd if=/dev/zero of=/dev/sdc bs=512 count=1 When I tried to "Restore old partition table..." I saw a dialog box indicating "No partition table found on device /dev/sdc". I think it would be useful to allow users to restore missing ms-dos partition tables. 2) If extended partition table already exists then do not prompt to restore the extended partition table. I simulated this by creating an MS-DOS partition table with some primary partitions, an extended partition, and some logical partitions. When I tried to "Restore old partition table..." I was prompted in a dialog box to restore the extended partition table. After selecting yes, a dialog box was shown indicating "Sorry. There was an error restoring some partition". This also happened if I was prompted to restore a logical partition. 3) If possible improve feedback to user regarding progress This might be done using a dialog box to indicate progress, such as what percentage of the disk had been searched so far. I'm not exactly sure how this would be implemented, but it would be helpful to users who wait a long time for a scan of a large drive.
Hi Curtis, > 1) Permit restoration of an entire ms-dos partition table if the partition table is unknown You mean make a backup of the partition table to restore it after? I make the same test, erasing the mbr with dd, and gpart finds the partitions (except the extended). > 2) If extended partition table already exists then do not prompt to restore the extended partition table. Ok, I think on this, now, the patch writes all partitions if is possible, if any partition exists (no matter if is primary, logical or extended), then shows an error, and don't touch the partition. I make that because I don't know how to check if partition exists. > 3) If possible improve feedback to user regarding progress I think it can't be. How we know what time remain to read the old partitions? Gpart don't shows any indicator.
(In reply to comment #33) > > 1) Permit restoration of an entire ms-dos partition table if the partition > table is unknown > > You mean make a backup of the partition table to restore it after? My intent was that the menu item for restoration of a partition table would be enabled only if the current partition table was either unknown or msdos. If a different partition table was in use, such as gpt, then this menu item would be grayed out and unavailable to the user. Either that, or sternly warn the user that their old partition table will be overwritten with an msdos partition table. I think that there should be a warning too if the user tries to restore a partition when there are outstanding operations in the queue. The user should be instructed to clear the operations before trying to restore a partition. > > 2) If extended partition table already exists then do not prompt to restore > the extended partition table. > > Ok, I think on this, now, the patch writes all partitions if is possible, if > any partition exists (no matter if is primary, logical or extended), then shows > an error, and don't touch the partition. I make that because I don't know how > to check if partition exists. I'm not sure how to easily do this either. The existing partition layout can be queried through libparted. Also GParted contains memory structures that map the current partition table (device.h and partition.h). > > 3) If possible improve feedback to user regarding progress > > I think it can't be. How we know what time remain to read the old partitions? > Gpart don't shows any indicator. This could be difficult if not impossible to do if gpart does not provide progress updates.
>If a different partition table was in use, such as gpt, then this menu item >would be grayed out and unavailable to the user. Either that, or sternly warn >the user that their old partition table will be overwritten with an msdos >partition table. Now, if the user has a gpt or other no-msdos disklabel, the patch shows an error dialog and do not continue, If the user want continue, should create manually the msdos disklabel. Thus forcing the users to know what they are doing. >I think that there should be a warning too if the user tries to restore a >partition when there are outstanding operations in the queue. The user should >be instructed to clear the operations before trying to restore a partition. The patch already performs this check. >I'm not sure how to easily do this either. The existing partition layout can >be queried through libparted. Also GParted contains memory structures that map >the current partition table (device.h and partition.h). Yes, but for make sure that the partition in libparted is indeed the same that gpart found, the start/end sectors should be the same, and this may not always be accurate, however, I will modify the patch to make this check and if some partition matches, remove it from the list of partitions to restore.
Hi Joan, It sound like you are making good progress on this enhancement. I hope to have more time for review later this month. I look forward to checking out your next updated patch.
Created attachment 169786 [details] [review] Gpart support for Gparted Hi Curtis, I wrote the patch but I forgot to upload it, here it is.
Hi Joan, Thank you for your patience. I just re-read this bug report and can see that much progress has been made, and that you have addressed many of the problems I mentioned. To get us started again I will try to summarize what we have discovered so far. A) gpart only works directly with msdos partition tables (E.g., for full partition table restoration) B) gpart works best with 4 partitions or less C) gpart works best with primary partitions D) gpart works by default with a 512 byte sectors size (There is a -s option to use up to a 16384 byte sector size) I think you stated the goal of this feature best in comment #9: ----- start of quote ----- Anyway, the main purpose of gpart is to recognize lost partitions in order to make accessible to the user to mount and make backups or recovery data that in other way it would be lost. And it fulfills correctly. There are many people who knows gparted but there's less people who knows gpart, then, this new feature can save their lives :) ----- end of quote ------ Now that gparted 0.6.3 has been released, I plan to spend more time on features for the next release of GParted (0.7.0) and I would like to include the ability to restore partitions as enabled by your patch. As such I plan to spend more time so we can get your patch in tip-top shape ready for the next GParted release. :-) Following are some suggestions for enhancements. Please feel free to question or comment on these as you wish. 1) Rename the Device menu entry from "Restore Old Partition Table..." to "Restore Lost Partitions..." The reason for the new title is that now we permit the user to restore individual partitions, and not the entire partition table. 2) Add the existing partition table type to the warning dialog that indicates this feature does not work with non-MSDOS partition tables. Currently the dialog box looks like this: Title: Wrong disklabel Bold Text: No msdos disklabel Other Text: This feature only works with msdos disk labels Since many users confuse disk labels with volume labels, I have tried to use the term "partition table" instead of "disklabel". I try my best to align GParted with the GNOME Human Interface Guidelines. As such the description of how to make an alert message can be found at the following link: http://library.gnome.org/devel/hig-book/stable/windows-alert.html.en Hence I suggest the dialog box should look like this: Title: <empty> Bold Text: No msdos partition table found. Other Text: A %1 partition table was found. This feature only works with msdos partition tables Where: %1 is the name of the type of partition table found. For example, gpt. I will continue to test the patch. I also plan to review the source code of the patch. Now that the patch permits a user to selectively restore individual partitions, we might be able to make it work on other partition table types, such as gpt. It's just a thought for now. Thanks again for all your work to date. Sincerely, Curtis
Hi Curtis, In the next days I will send a new patch that implements the new changes you mentioned. The capacity to restore partitions in other disklabels than ms-dos is more hard. I will study how to make it and I'll tell you something.
Created attachment 171581 [details] [review] Gpart support for Gparted Hi Curtis, here's the new patch. I've tried to adapt all the dialogs to the gnome human guidelines. In the process I found a bug and I fix it. I propose that we first do work well this patch, we adapt it as best as possible to the rest of the program and ensure it is sufficiently stable. And then we try to make it compatible with other disk labels. What do you think?
Hi Joan, I agree. :-) Let's get this patch working well before trying to extend it to other disk labels (partition tables). Thanks for the new patch with the above updates included. I have done some testing and discovered that gpart appears to have problems recognizing ext4 file systems. To find this problem I used the following steps. 1) Create a new RAM disk 2) Create a new MSDOS partition table on the RAM disk 3) Create a partition with the ext4 file system on it 4) Run gpart on the RAM disk and notice that no partitions are found. I will continue to test and will check out the other dialogs too and let you know what I discover.
Hi Curtis, yes, is really, I also noticed that sometimes gpart does not detect some partitions, is a bit of a lottery. Based on my experience, there are some filesystems such as Linux-Swap or NTFS, which are always detected, and others like ext4 it never know whether will be detected or not. I think that that 's one of the many limitations of gpart and we have to work with it.
Hi there. Late to the party. Why not think out of the box for a moment. If you just want to enable users to salvage lost partitions... There is no actual need for us to modify / restore partition tables at all. As you have discovered, this can lead to particularly hard to solve situations: 1. when no primary partition slot is left 2. when no extended partition exists/can be made 3. when the existing extended partition does not overlap the partition region fully 4. when the disk type is not msdos so... instead, we could add the 'gpart detected disk' as a /temporary view/ of the physical disk. And we could allow the detected fs-es to be mounted RO using a clever incantation of losetup, like losetup -f /dev/sdc --offset 12367836 --size-limit $((1024*1024*512)) mount -o ro /dev/loop0 /tmp/rescue-fs That way, we can a. stop worrying about all the edge cases b. allow the users to 'interactively' check whether the fs is consistent/readable before having to commit scary stuff to his/her partition tables c. allow the users to get data out and forget about the mess of partitions d. note that this even allows users to tentatively mount 'alleged/suspected' partitions, even when it would have required them to move/remove newer existing paritions out of the way to commit the partition. This could lead to a fs that is partially salvageable, without having to loose any data already on new partitions. Note that this 'temporary view' model of gpart output is necessarily readonly for all the above reasons. In data-recovery scenarios, this is often a good method anyway Should the users realyl want to reinstate the lost partition layout (say, in case he/she did actually accidentally overwrite _just_ the partition table and nothing else), that could be implement that as the current state of affairs is by Joan's work. Its just that all the 'but what if...' scenarios become a lot less important. Hope that helps Seth Heeren
Thank you Seth for sharing your ideas with us. You have presented an interesting alternative. For actually recovering partition tables, I think I speak for both Joan and myself when I say that testdisk is probably the best tool. Having said that, testdisk is difficult to control through gparted, so the gpart program was chosen. The reasoning here being that enabling users to easily restore lost partitions, even recognizing the problems with "edge" cases, was better than having no ability at all. As Joan pointed out to me though, many users are not aware of or are not comfortable with using command line tools. The patch that Joan has developed does indeed place partition recovery in the hands of those comfortable with GUI tools. You correctly point out that there are several "edge" cases. Some of these I think we can address (e.g., restore for different partition table types), but not all of these (e.g., gparted only finds and lists specific details for 4 partitions). If you haven't already done so, please do try out Joan's latest patch. It works quite well for restoring lost primary partitions, without overwriting or damaging existing partitions. Thinking more on your suggestion, perhaps allowing users to temporarily mount partitions would be a great additional recovery method. :-) Joan, what are your thoughts on this?
Sry Joan, jumping the gun a bit here, but I thought it will help to quickly feedback to what I understand you made of my ideas... I'm also interested to know what your thoughts and reservations might be --- On 10/06/2010 08:32 PM, gparted (bugzilla.gnome.org) wrote: > > I think I speak for both Joan and > > myself when I say that testdisk is probably the best tool. +1 it's the only tool I resort to in cases of dire need > > Having said that, > > testdisk is difficult to control through gparted IMHO: testdisk is difficult to control, period. However, it's scanning is the best around and I can always do the hard math myself, if the data is valuable enough to me :) In other words, I don't question gpart for the job. It is a good choice. 80% of functionality at 20% of the effort, so they say. > > there are several "edge" cases In fact, I don't worry about the edge cases (I don't care about GUI support for them). I have gotten more trouble out of reconstructing the difficult edge cases than I gained from it. And that is when I did everything in sfdisk/cfdisk/fdisk, parted and kpartx; and I must say I probably know my stuff in the area (at least for MBR/MSDOS). I don't trust a GUI tool to try to do that voodoo stuff for me. I must say, that If I would be in a squeeze again [1], I would certainly do the exact same thing: use testdisk (or gpart) to locate the filesystems (not partitions) and mount them using losetup, readonly. It is the most flexible solution, and allows operating in 'clean room'/forensics level guarantees of not risking _ANY_ data or metadata in the process. This is a sweet mode, especially since you guys are going to head the support calls :) I think if GParted is going to propose any modifications to user's partition tables, we had better steer *way* clear of the edge cases, and prevent users from shooting themselves in the foot, while they were limping anyway. The 'lay' user is not going to be helped with confusion he won't be able to set right himself. The experienced techie [2] knows when Gparted will be able to stitch it up easily, and also predict when it is not going to be easy. He won't regret it if GParted refuses to do tricky stuff and simply use the information GParted showed to do it himself in the CLI environment. [1] which won't happen soon because nowadays I run everything on Solaris ZFS via ISCSI, having bit-precise backup images is cheap etc. [2] say, the users of testdisk or gpart at this time - so we have an idea of who I'm talking about there
Hello Seth. I think your solution is great and I have not answered before because I've been thinking how to integrate it in gparted. I did not know losetup and is a good solution. I have been making some tests and is not hard to write. In my patch, I've not separated the concepts of "partition" and "filesystem". So Curtis, now, If a filesystem does not necessarily imply a partition, the cases that are truly necessary to restore the partition would be rare. So I think in three possibilities: 1.- Try to melt the two recover options (partition and filesystem), for example, showing the current dialog and adding a new button that opens nautilus with the read-only data. If the user wants, can push the "apply" button to restore the partition too. 2.- Make two menu entries with two dialogs. The first would restore partitions with its data and the other one would let you mount the temporary view to rescue data. 3.- Leave only the menu entry and dialog for make temporary views. In the three options, gparted uses gpart to determine where a filesystem begins. With the two first options, maybe some user becomes crazy and try to kill us.
If I am interpreting the message correctly, what I am hearing is that both of you would prefer to avoid tampering with the partition table when trying to restore data. With this new option that Seth has suggested, it is now possible to permit users to restore data without the need for editing the partition table. I am definitely in favour of this approach (Joan's item #3) since it should indeed reduce the number of support calls, and keep our users happier. :-) Since there are many different GNU/Linux desktops out there, it might be useful to search for more than one graphical file manager. For instance GNOME uses nautilus, KDE uses dolphin, XFCE uses thunar, etc. One possible caution I can think of is that I am unsure if the count of file system mounts is updated when the file system is mounted read-only. Some file systems, such as ext2/3/4, store a count in the metadata that is used to determine when the file system should be checked again. My thoughts are that no data should ever be written to the disk when a file system is mounted read-only, but I do not know if this applies to the count of file system mounts. Anyways this is just a thought, and should not prevent us enabling users to restore lost data.
Hi Curtis, If we find a way for the user to have both options without it being a mess, would be good if he had the two options because the option to restore partitions is yet written. Anyway, I'll add a new entry in the device menu with a new dialog, etc.. And then we can erase the first menu entry if we want. Regarding your thought toward the counter of mounts, I think that probably the person who is reading the data lost will not matter. The new option, something like "Read lost data" would not deal with the structure of the filesystem where data is found. That filesystem is lost, you get a copy of the data to a USB or something like and discard the rest. Don't you think?
> would be good if he had the two options because the option to restore partitions is yet written +1, completely agreed. I thought of the other way around because we seemed to be running in all kinds of complicated edge cases, with a distinct risk of putting a user in even more misery when he/she was already recovering data... > toward the counter of mounts, I think that probably the person who is reading the data lost will not matter The big worry, though, is with a 'lost partition' that might be detected in an area of the disk that is currently active in an other partition, or has been partly rewritten with parts of another filesystem that might contain valuable data as well... I know it is a a far-fetched scenario, and all the data loss would probably be less than 32 bits... but still, conceptually it would be nice to enforce as much of read-only ness that is possible
to even further clarify: I think it is nice to have the readonly, temporary views in addition to simple partition restores, because we can all but stop worrying about the difficult cases
OK, now I understand, but what should we do to prevent it?
As I suggested, we could look at losetup.c (attached, 4k) and the kernel module (/usr/src/linux-2.6/drivers/block/loop.c) for ideas Also, googling around learns that the message "mount: block device //XXX.XXX.XX.XX/myServer is write-protected, mounting read-only" crops up when using mount(1) Looking at the source, case EACCES: /* pre-linux 1.1.38, 1.1.41 and later */ case EROFS: /* linux 1.1.38 and later */ is returned from guess_fstype_and_mount when the block device node is readonly; so using losetup with chmod a-w would seem to be enough. Further reading of mount(1) source shows that perhaps losetup is not needed (mount includes a losetup.c and loop_check specifically sets SETLOOP_RDONLY in L1113: if (*flags & MS_RDONLY) loop_opts |= SETLOOP_RDONLY; which results in setloop() oponing the loop device node in RDONLY mode (lomount.c:687). So if we can figure out a way to not depend on losetup we can simply mount -o ro,loop,offset=123456,sizelimit=1230948823 /dev/sda /tmp/gparted_rescue_partitionid and have all the protection we need! Also, this would remove the need to manage loopdevices in any specific way (mount/losetup will manage it between themselves). Side note: Since Linux 2.6.25 is supported auto-destruction of loop devices and then any loop device allocated by mount will be freed by umount independently on /etc/mtab.
ps. no longer attaching losetup.c/loop.h because mount(1) looks more promising
(In reply to comment #49) > > would be good if he had the two options because the option to restore > > partitions is yet written > > +1, completely agreed. I thought of the other way around because we seemed to > be running in all kinds of complicated edge cases, with a distinct risk of > putting a user in even more misery when he/she was already recovering data... +1 for me too. Perhaps one option would be named "Restore lost partitions...", and the other one would be named "Recover lost partition data...".
> "Recover lost partition data...". > > Attempt recovery mount Attempt rescue data Test loop mount Read-only view --- PS. we would still need the partition overlap calculation so that we can .e.g. "Warning: the detected filesystem area overlaps with some known partitions." "It is recommended that you don't use the overlapping filesystems to avoid disturbing the lost partitions data. Do you want to do deactivate the following mount points: /dev/sda5 /mnt/mynew-mountpoint <Cancel> <Ignore> <Remount RO> <Unmount> just thinking aloud here
(In reply to comment #55) > PS. we would still need the partition overlap calculation so that we can > .e.g. > > "Warning: the detected filesystem area overlaps with some known partitions." > "It is recommended that you don't use the overlapping filesystems to > avoid disturbing the lost partitions data. Do you want to do deactivate > the following mount points: > > /dev/sda5 /mnt/mynew-mountpoint > > <Cancel> <Ignore> <Remount RO> <Unmount> This is a good point to make because we do not wish to harm existing partitions, and would not want overlapping disk space to be mounted at the same time. I believe that Joan has the overlap check coded already since my testing with the patch only offered to restore partitions that did not overlap with existing partitions.
Hi, the current patch checks if a guessed partition overlaps other partition, but only in the case gpart find inconsistences and in that case, adds a "(!)" at the end of the name of the filesystem, in the dialog. The user can try to restore the inconsistent partition and, if indeed is overlapping, parted returns an error trying to create a new partition and the active partition don't change, then gparted shows an error. In any case, the code to search inconsistences is already written and is not difficult to integrate in the new section. I am writing what we talked and I have a doubt. How can I launch the default file browser from c++? maybe there is a Glib function or something? That I would like to do is show a "View" button for each partition and when the user click over it, open the default file browser with in the path where the view is mounted, and when the user close the browser, the view is unmounted.
(In reply to comment #57) > I am writing what we talked and I have a doubt. How can I launch the default > file browser from c++? maybe there is a Glib function or something? Take a look at the Win_GParted::show_help_dialog method which uses the gtk_show_uri method. http://library.gnome.org/devel/gtk/stable/gtk-Filesystem-utilities.html#gtk-show-uri I think something similar can be used to open a directory, e.g., replace "ghelp:" with "file:" so that the uri for a directory looks something like the following: file:///path-to-mount-point/ I have not tested this myself.
_BUT_ you cannot 'wait' the child PID because it will probably fork off into the background. I know nautilus does. I just checked and dolphin does that too. Probably best to let the user unmount anyways? On the partition overlaps: as long as we mount the loop devices readonly there isn't any risk of "to harm existing partitions" as Curits was suggesting. It is just helpful to warn the user, because he/she might want to cease working on any partition that might overlap a lost partition, in order to _preserve the lost one_, not vice versa :)
Gparted builds a dialog with the list of guessed filesystems dynamically, then I need to connect all generated "View" buttons with the callback function, but I need to pass a parameter to this function, with the position (in the array) of the filesystem to mount. I am searching on google and I don't find anything, there are some way in sigc:: to call a function with parameters? I have a look at the show_help_dialog() function and it seems that there is no way to know when the user closes the browser. What I will do is mount all the ro-views in /tmp/gparted-roview-XXXXXX/ directories and unmount all of them when the user closes the dialog.
no particular experience with gtkmm here but 'closure' is the technical term (although a _bound_ (member) function ptr is also a common name (espec. in boost like templatey libs)) I remember seeing 'closure' in the glibmm apis, so I guess there are facilities in there
(In reply to comment #60) > Gparted builds a dialog with the list of guessed filesystems dynamically, then > I need to connect all generated "View" buttons with the callback function, but > I need to pass a parameter to this function, with the position (in the array) > of the filesystem to mount. I am searching on google and I don't find anything, > there are some way in sigc:: to call a function with parameters? > Unfortunately my knowledge isn't deep enough in this area to help. ;-( > I have a look at the show_help_dialog() function and it seems that there is no > way to know when the user closes the browser. What I will do is mount all the > ro-views in /tmp/gparted-roview-XXXXXX/ directories and unmount all of them > when the user closes the dialog. That sounds like a good suggestion. I would go one further and suggest that another dialog be opened indicating that all of these temporary mounts will be umounted when the user closes this second dialog box, and that the user should finish all of their file copying from these temporary mounts before closing this second dialog box.
Created attachment 173112 [details] [review] New section that makes temporary views of the guessed filesystems Hello, here is the patch with the new section, maybe this version has bugs or the English is not correct, besides, I'm not entirely sure what nautilus is closed properly when the user closes it, and I have not tried with other browsers. For that, I suggest someone test the patch in different environments. If gparted have beta-testers this is a good time to call them :). If we finally eliminate the option to restore partitions I do not mind, we must do what is best for the program and the users. I found the solution to pass parameters to the callback functions with sigc:: and is very easy, simply call the function sigc::bind in this way: btn->signal_clicked().connect(sigc::bind(sigc::mem_fun(this, &Dialog_Rescue_Data::on_view_clicked), numPart)); Greetings.
Thank you Joan for this updated patch. :-) I have posted a request for help with testing in the GParted Testers forum. Seeking Help Testing New GParted Data Recovery Enhancement http://gparted-forum.surf4.info/viewtopic.php?id=14382 I will also perform my own testing, and plan to review all of the text strings. This might take me a week or two because my time is also needed to shepherd the GParted 0.7.0 release planned for this Friday, October 29, 2010. I will report back here with my testing comments.
Hi Joan, So far it looks like it is just us testing this patch, since all is quiet on the forum front regarding testing. I have only recently starting testing this latest patch and it looks pretty good. I would like to re-word some of the strings and hope to get around to this soon. Would you prefer me to create a patch with the new strings based on your patch? If not I can list the original strings and updates for these in this post. (In reply to comment #63) > If we finally eliminate the option to restore partitions I do not mind, we must > do what is best for the program and the users. I believe that the option to "Attempt Data Recovery" meets our goal was to provide a way for users to recover their data from lost partitions. With this in mind and because testdisk does a better job of actually restoring partitions, I am in favour of eliminating the option to restore partitions. Would you be able to update your patch accordingly?
Hi Curtis, I think it's better that you paste here the strings when you can and I start to modify the patch on my own, to expedite the work. I will upload something in the next days
Hi Curtis, when you will upload the strings? I have a patch ready and when I paste the strings I will upload it. Thanks.
Hi Joan, sorry for the delay. I hope to finish the string translation later today or tomorrow. I am working on translating only the portion for "Attempt Rescue Data.." and not for "Restore Lost Partitions..."
Yes, I deleted the section "Restore Lost Partitions".
Hi Joan, following are some suggestions for string changes. Please feel free to question any of my suggestions as perhaps between us we can come up with a more descriptive text string. In File Utils.cc ---------------- Change name of method "convertToInt" to "convert_to_int" so that it matches the other method names. In File Win_GParted.cc ---------------------- Under GParted Device menu, entry "Attempt Rescue Data..." should be "Attempt Data Rescue..." "Gpart command not found" should be "Command gpart was not found" "This feature uses Gpart. Please, install it and try again." should be "This feature uses gpart. Please install gpart and try again." "Searching for old filesystems" should be "Search for file systems" "You need to scan the disk to find old partitions, this may take a bit, do you want to continue?" should be "A disk scan will be performed to find recognizable file systems. " concatenated with "After the scan You can mount any of these discovered file systems enabling you to copy the data to other media. " concatenated with "The full disk scan might take a very long time. " concatenated with "Do you want to continue?" "Searching for old data on %1" should be "Searcing for file systems on %1" "No filesystems found" should be "No file systems found" "Sorry, gparted wasn't found filesystems on this disk." should be "The disk scan by gpart did not find any recognizable file systems on this disk". In File Dialog_Rescue_Data.cc ----------------------------- "Attempt rescue data on %1" should be "Attempt to rescue data on %1" Title "Search lost data" should be "Search disk for file systems" The word "inconsistences" is mispelled. The correct spelling is "inconsistencies" (note the extra i in "ies"). "WARNING!: The filesystems marked with (!) are inconsistents and try to view them may cause errors." should be "WARNING!: The file systems marked with (!) are inconsistent. You might encounter errors trying to view these file systems." "The 'View' buttons create read-only views of the filesystems.\nAll mounted views will be umounted when you closes this dialog." should be "The 'View' buttons create read-only views of each file system." concatenated with "\n" (not translatable) and then concatenated with "All mounted views will be unmounted when you close this dialog." ^^^ Note full spelling for unmounted "Filesystems" should be "File systems" "There was an error creating the read-only view. Maybe the filesystem can't be mounted (like swap) or there are inconsistences or errors in the filesystem" should be "An error was encountered while creating the read-only view. " concatenated with "Either the file system can not be mounted (like swap), or there are inconsistencies or errors in the file system." "Warning: the detected filesystem area overlaps with some known partitions." should be "Warning: The detected file system area overlaps with at least one existing partition." "It is recommended that you don't use the overlapping filesystems to avoid disturbing the lost partitions data. Do you want to try to deactivate the following mount points?:" should be "It is recommended that you do not use any overlapping file systems to avoid disturbing existing data. " concatenated with "Do you want to deactivate the following mount points?"
Created attachment 174967 [details] [review] Section "Restore old partitions" removed Hi Curtis, attached the new patch. The strings are fine, I see only a couple of possible changes: 1.- In the warning dialog that appears when you click on "Attempt data rescue" is a description of how the next dialog works. When later shown the following dialog reappears a short description that perhaps be redundant. 2.- In the warning that a file system is overlapping an active partition, I add the word "try" because we may be the case that the root partition being overlapped and can not be unmounted. So perhaps the program that afirme unmount a partition and then do not do this thing. In any case it would be appropriate just like tell to the user that does not perform too many actions while the process lasts.
> In the warning that a file system is overlapping an active partition, I add the word "try" because we may be the case that the root partition being overlapped and can not be unmounted. Why not try to remount ro? This is the good-old fashion to avoid disrupting more than necessary. I think Alt-SysRq-U (or something) is supposed to forcibly remount all fs-es as readonly so I guess it is possible to make this non-fallible, but with an extra caveat to the user when the 'force' is needed (e.g. "Cannot remount the ..... as readonly. Do you want to force the mount to readonly mode anyway?"
Hi Joan, Thanks for the new patch. :-) (In reply to comment #71) > The strings are fine, I see only a couple of possible changes: > > 1.- In the warning dialog that appears when you click on "Attempt data rescue" > is a description of how the next dialog works. When later shown the following > dialog reappears a short description that perhaps be redundant. My intention was to ensure that the user knew in advance what they would be able to do with "Attempt Data Recovery" before initiating this option and having to wait a very long time for large drives. I do think that the initial message is on the long side though, and will try to reword it into something shorter. I am open to suggestions too. Since it might take a very long time to scan large hard drives, I think that the extra redundancy in the descriptions should be acceptable to the user. After waiting an hour or more I suspect that the user will have forgotten some of the text from the initial dialog. ;-) > 2.- In the warning that a file system is overlapping an active partition, I add > the word "try" because we may be the case that the root partition being > overlapped and can not be unmounted. So perhaps the program that afirme unmount > a partition and then do not do this thing. In any case it would be appropriate > just like tell to the user that does not perform too many actions while the > process lasts. I agree with you here. The word try is important since in my testing I encountered situations in which part of this process failed. Perhaps the new wording should be: The 'View' buttons will try to create read-only view of each file system. 3. I have a new suggestion due to a problem I encountered in one of my test scenarios. If the call to open a file manager fails, but the mount did succeed, perhaps a dialog box could be shown indicating that: GParted was unable to open a file manager for this file system. The file system has been mounted on "/tmp/xxxxx". Hi Seth, Thank you for the suggestion. I did try this patch with mounting a temporary "view" partition that was already mounted rw, and this did work. I am hesitant to suggest this patch try remounting one or all existing file systems as read-only since this might cause bigger problems depending on what else the user is currently running. For example if a process is running that is writing to a log file.
(In reply to comment #73) > Perhaps the new wording should be: > > The 'View' buttons will try to create read-only view of each file system. Doh! This should have said: The 'View' buttons will try to create a read-only view of each file system. > 3. I have a new suggestion due to a problem I encountered in one of my test > scenarios. If the call to open a file manager fails, but the mount did > succeed, perhaps a dialog box could be shown indicating that: > > GParted was unable to open a file manager for this file system. > The file system has been mounted on "/tmp/xxxxx". GParted was unalbe to open a file manager to view this file system. The file system has been mounted on "/tmp/xxxxxx".
My fingers must be itchy because they are not typing exactly what I am thinking. ;-) s/unalbe/unable/
> I am hesitant to suggest this patch try remounting one or all existing file systems as read-only since this might cause bigger problems depending on what else the user is currently running. For example if a process is running that is writing to a log file. I don't suggest you silently remount r/o; I just think it is worth offering it as a better alternative to simply saying "no can do". Big warnings should be given, but it might spark the power user with the very idea that he _could_ mount ro. This is just being helpful, IMO. Vice versa, we might check when overlapping partitions are mounted: if they are already mounted R/O, there shouldn't be a problem in going ahead (but a warning would seem in order). Perhaps you can devise a dialog box that displays a list of messages (with severities) instead of many separate dialogs. Like Curtis mentioned, by the time the third dialog appears, the messages from the first will be all but forgotten. People (like me) like to re-read especially when they are doing high-pressure tasks like data recovery.
(In reply to comment #76) > I don't suggest you silently remount r/o; I just think it is worth offering it > as a better alternative to simply saying "no can do". Big warnings should be > given, but it might spark the power user with the very idea that he _could_ > mount ro. This is just being helpful, IMO. Thank you Seth for the clarification. I do see this as being helpful.
Hi Curtis and Seth, both suggestions, try to re-mount as RO and to unify all dialogs in one are in my opinion correct, but I think we ought to leave it for later, because now I no longer have time to make major changes in the patch. When I started to collaborate with gparted 5 months ago I had more time. So I think that for now, we should try to fix errors that are in the patch right now, for example, if you look, not all times you open a file browser, it closes when you close the dialog, the browser window dissapears, but the process remains open and do not know what is due. I think we should fix it somehow. >Since it might take a very long time to scan large hard drives, I think that >the extra redundancy in the descriptions should be acceptable to the user. Ok, you're right. >I have a new suggestion due to a problem I encountered in one of my test >scenarios. If the call to open a file manager fails, but the mount did >succeed, perhaps a dialog box could be shown indicating that Would not it be better to try to find why fails and fix it? >Doh! This should have said: >The 'View' buttons will try to create a read-only view of each file system. I was referring to the string: "It is recommended that you don't use the overlapping filesystems to avoid disturbing the lost partitions data. Do you want to try to deactivate the following mount points?:" that you change to: "It is recommended that you do not use any overlapping file systems to avoid disturbing existing data. " concatenated with "Do you want to deactivate the following mount points?" ...removing the word "try". I'm not sure that we are talking about the same dialog.
(In reply to comment #78) > Hi Curtis and Seth, both suggestions, try to re-mount as RO and to unify all > dialogs in one are in my opinion correct, but I think we ought to leave it for > later, because now I no longer have time to make major changes in the patch. > When I started to collaborate with gparted 5 months ago I had more time. So I > think that for now, we should try to fix errors that are in the patch right > now, for example, if you look, not all times you open a file browser, it > closes when you close the dialog, the browser window dissapears, but the > process remains open and do not know what is due. I think we should fix it > somehow. Good suggestion Joan. Let's get something useful out first and then work on refining it to make it better. I think this new patch which performs a read-only mount significantly reduces the chance of data loss as compared to our first approach which would overwrite the partition table. > >I have a new suggestion due to a problem I encountered in one of my test > >scenarios. If the call to open a file manager fails, but the mount did > >succeed, perhaps a dialog box could be shown indicating that > > Would not it be better to try to find why fails and fix it? I will plan to investigate this behaviour further, and see what can be done. > >Doh! This should have said: > >The 'View' buttons will try to create a read-only view of each file system. > > I was referring to the string: > > "It is recommended that you don't use the overlapping filesystems to > avoid disturbing the lost partitions data. Do you want to try to > deactivate the following mount points?:" > > that you change to: > > "It is recommended that you do not use any overlapping file systems to > avoid disturbing existing data. " concatenated with > "Do you want to deactivate the following mount points?" > > ...removing the word "try". I'm not sure that we are talking about the same > dialog. Sorry, my mistake. I was thinking of a different dialog. Please feel free to add the word "try" back into the appropriate sentence. When I have a chance I will work on shortening the dialog that says: A disk scan will be performed to find recognizable file systems. After the scan You can mount any of these discovered file systems enabling you to copy the data to other media. The full disk scan might take a very long time. Do you want to continue?
Hi Joan, Following is my attempt to shorten the dialog message, but still retain enough information so that the user can make an informed decision: A full disk scan is needed to find file systems. The scan might take a very long time. After the scan you can mount any discovered file systems and copy the data to other media. Do you want to continue?
Created attachment 175490 [details] [review] Some strings changed Hi, I changed two strings, the wird "try" in "Do you want to try to deactivate the following mount points?" and the new string "A full disk scan is needed to find file systems. The scan might take a very long time. After the scan you can mount any discovered file systems and copy the data to other media. Do you want to continue?" instead of the previous and largest.
Wow, this enhancement is really shaping up. I am impressed how it works when gpart is able to find file systems. :-) At the moment the only updates I saw were that there was a missing period at the end of the following two sentences: "After the scan you can mount any discovered file systems and copy the data to other media" "The disk scan by gpart did not find any recognizable file systems on this disk" I will continue testing as I believe that is only the three of us involved with this enhancement. It would be nice if we could have it ready for release in December or early January.
Created attachment 175644 [details] Data recovery attempt using gpartedbin copied to GParted Live 0.7.0-7 Hi Joan, I've started some LiveCD testing by copying a compiled gpartedbin binary to a running virtual machine instance of the GParted Live 0.7.0-7 image. While testing "Attempt data recovery" I once again ran into the situation where nothing appears to happen when I click on the view button (screen shot attached). I suspect that the call to open a file manager silently failed, because when I examined the currently mounted file systems in another window I saw there was a /tmp/gparted-roview-JY8lKk file system mounted. As such I think we could try to improve the user experience by checking the return code when spawning the file system browser. If the spawn fails, we could display a dialog box indicating this failure, but also stating where the file system is currently mounted. That way a user could still use the command line to copy files to other media. Does that sound like a reasonable approach to this issue? The GParted code should probably do something similar when trying to open the help browser too.
(In reply to comment #83) > The GParted code should probably do something similar when trying to open the > help browser too. Silly me. ;) It looks like GParted already displays a dialog message if there is a failure to open a help browser: The code snippet from the Win_GParted.cc file is: #ifdef HAVE_GTK_SHOW_URI gtk_show_uri( gscreen, uri .c_str(), gtk_get_current_event_time(), &error ) ; #else Glib::ustring command = "gnome-open " + uri ; gdk_spawn_command_line_on_screen( gscreen, command .c_str(), &error ) ; #endif if ( error != NULL ) { Gtk::MessageDialog dialog( *this , _( "Unable to open GParted Manual help file." ) , false , Gtk::MESSAGE_ERROR , Gtk::BUTTONS_OK , true ) ; dialog .set_secondary_text( error ->message ) ; dialog .run() ; }
Created attachment 175801 [details] [review] Testing if cannot open a file browser; some strings changed Hi Curtis, I made a new patch with the missing periods on those string and checking for error and message if it fails the opening of the file browser. Today I had a little time and tried to figure out why when you close gparted does not show the command line until you press Ctrl + C. I thought it does not close because nautilus remains open, but looking at the gnome-system-monitor I've seen yes, what remains open is a "mount" command with the state "Uninterruptible". Is a bit strange because this "mount" is other than the mount of the ro-view. I would like you help me with this because I think that we can not publish while there is this error, and now I do not have much time to see what is happening.
Thanks again Joan for the updated patch. :) So far I have not encountered the situation you mention. Do you have a reproducible case? If so, what are the steps so that I can recreate the problem? In the meantime I will attempt to recreate the problem of GParted not closing properly (returning to command prompt) due to an open mount.
Hi Curtis, in my case the error occurs always, when I mount a ro-view and then I close gparted, an instance of "mount" remains open. I use debian squeeze.
Created attachment 176078 [details] Screen shot of GParted dialog Unable to open default file browser Thank you Joan for your perseverance in improving this code. I do appreciate your efforts to debug this enhancement for a production release. So far I have been unable to replicate the problem with GParted not closing due to an open mount. I have tested in the following 5 environments: Kubuntu 8.04 (Hardy Heron) Ubuntu 10.04 (Lucid Lynx) Ubuntu 10.10 (Maverick Meerkat) GParted Live 0.7.0-7 (Debian Squeeze) Fedora 13 In the GParted Live instance, a file browser is not installed. The mount of the file system succeeds, and opening the file browser fails as expected. I wonder what the key difference is between my test systems, and your Debian Squeeze system? On a different note, there is a problem with the dialog "Unable to open default browser". It is a best practice not to include html markup directly in the translatable string. I tried to find where I originally learned this, but the closest I could find on web searching was this article: Preparing Translatable Strings http://www.gnu.org/software/hello/manual/gettext/Preparing-Strings.html As can be seen in the attached screen shot, we should probably remove the "<b>" and "</b>" html markup anyway because it does not display correctly in the dialog box. To improve the dialog box I suggest the following string changes: From: The file system has been mounted on <b>%1.</b> To: The file system is mounted on %1. From: Unable to open GParted the default file browser. To: Unable to open the default file browser. I also suggest that the sentence containing the error should be displayed on a separate line.
Oops, almost forgot that when substituting parameters in a translatable string it is helpful to the translators to place a sample line directly above the string. For example: /*TO TRANSLATORS: looks like No partition table found on device /dev/sda */ String::ucompose( _( "No partition table found on device %1" ), device_name ),
Created attachment 176197 [details] [review] A string fixed Hi Curtis, I modified the patch to correct the text string. Now I have corrected the errors and I put the text with tags outside the function gettext. I kept the tags because I think it's important to put the bold since it is possible that people closes the dialog without reading. Please, see if it looks good and tell me something. I do not know why the error with the mount command occurs, How it happens in my computer, I'll try to make time to watch it.
Created attachment 176212 [details] GParted dialog box - Unable to open file browser Attached is screen shot of what the dialog box looks like when GParted is unable to open a file browser. I agree that the bolded text does make the mount point stand out. It might help to not split the mount point sover two lines, and also to remove the period at the end of the mount point as it may cause some confusion. We might use something like the following: <--- sample screen begins ---> Unable to open the default file browser GParted was unable to open a file manager for this file system. The file system is mounted on: /tmp/gparted-roview-reTopw Error: No application is registered as handling this file <--- sample screen ends --->
Created attachment 176234 [details] [review] Changes on the error dialog when no browser found Hi, here is a new patch with the changes.
Thanks for the new patch Joan. Would you be able to rebase the patch against the current master branch in the git repository? That will make it easier for me to apply the patch. :-) In the meantime I will begin work on updating the GParted Manual to describe this new feature. I would like to create a new GParted release with this patch for release this December or January. Hopefully we can find out how to fix the problem you experienced with GParted not closing due to an open mount. If not, I still believe that the current patch is worthy of a production release. What are your thoughts with regard to releasing this patch in Dec/Jan?
Hi Curtis, I confused a few days with git, I restarted my copy of the repository and now I'm getting errors when trying to apply my own patch, I get this error: error: patch failed: src / GParted_Core.cc: 51 error: src / GParted_Core.cc: patch does not apply You can not make the rebase or tell me how is it done?
Hi Joan, Do you still have your old git repository with your patches installed? If so do you have these on a separate branch, or on the master? If you do not have your old git repository then no worries. If needed I can manually resolve the differences between the master and the patch.
Hi Curtis > Do you still have your old git repository with your patches installed? No, I don't > If so do you have these on a separate branch, or on the master? I had it in the master > If you do not have your old git repository then no worries. If needed I can manually resolve the differences between the master and the patch. Yes, please. Thank you.
Created attachment 176675 [details] [review] Attempt data recovery patch Attached is the updated "Attempt data recovery" patch. This one has been rebased against the git repository master branch as of Dec 14, 2010 commit: 203107f600f0384fb062a0143450747a34fc8743
Hello Curtis, thank you for the patch. I'm just starting with git and I would like to know how you did it. Moreover, I have been investigating the error with mount and have seen that only occurs when running gparted as normal user with the following command: $ gksu-u root /src/gpartedbin But if you're running as root: # src/gpartedbin The error does not occur.
Hi Joan, I was able to duplicate the error with mount problem if I start GParted with: $ gksu -u root src/gpartedbin Since the purpose of gksu is to be used from a graphical interface, and not from a terminal window, it is possible that this glitch exists only in gksu. As such I do not think we would need to hold up releasing this enhancement for this issue. With regards to git it appears as if a problem with the last patch was not resolved prior to being committed to git. The problem was an unresolved conflict in the 14th patch which looked as follows: ---------- BEGIN SNIPPET ---------- diff --git a/src/GParted_Core.cc b/src/GParted_Core.cc index 7aadf28..bc80076 100644 --- a/src/GParted_Core.cc +++ b/src/GParted_Core.cc @@ -52,11 +52,7 @@ #include <sys/stat.h> #include <unistd.h> #include <dirent.h> -<<<<<<< HEAD -#include <fstream> -======= #include <mntent.h> ->>>>>>> 356ef5e75f0a718c71984988fbca86099d5b10e8 std::vector<Glib::ustring> libparted_messages ; //see ped_exception_handler() ---------- END SNIPPET ---------- When git has a problem resolving an issue, it will notify the user of the problem and the steps to take. Git also places "<<<<<<<", "=======", and ">>>>>>>" around the section git was unable to resolve. Then it is up to the user to resolve the issue and delete these extra markers. After fixing the problem, a user will then need to add the fixed file to git, and issue the command to continue as git would have instructed when the problem was discovered. Hopefully this helps you with how to resolve future git problems. :-) On a different note, I found a spelling mistake in Win_GParted.cc: Searcing for file systems on %1 should be Searching for file systems on %1 Best wishes for the holidays, Curtis
Created attachment 176794 [details] [review] One string fixed Hi Curtis, there is a new patch. My problem with git occurred when running "git am -3 <patch>" It shows me an error and says that when I resolved it I must type "git -- continue" to continue. In this particular case, git doesn't wrote brands in the files or creates either .rej files, there was no way found the conflict... with what order do you apply the patch? By the way, in the last attemps to guess partitions with gpart, Only ntfs partitions are found, gpart seems not find any other filesystems. Did you noticed that?
RE: Git problems - no brands in files. This can happen if git is unable to match the commit stamp. If that should happen, then a user has to manually apply each of the changes listed in the file for the patch that failed to apply. Ideally it is useful to rebase each patch to the master branch before posting the patch. This helps to reduce the chance of more work being required to apply the patch. I also apply the patches using "git am -3 <patch>" (In reply to comment #100) > By the way, in the last attemps to guess partitions with gpart, Only ntfs > partitions are found, gpart seems not find any other filesystems. Did you > noticed that? I knew that there are problems recognizing file systems sometimes. We noted this earlier in comment #42. In my testing, FAT and NTFS file systems are recognized consistently. It seems to be hit and miss as to whether ext2/3/4 file systems are recognized.
Hi Curtis, I have arranged back the git. If you or Seth do not have more suggestions/bugs in the patch and you want to launch it, it's ok for me. Because we have not had volunteers to test this new feature, we must have to wait for the calls for help in the future. Regards
Happy New Year Joan and Seth, Over the break I installed kubuntu 10.04 on my computer. This has permitted me to test the patch on KDE instead of GNOME. Most everything works as expected with the exception of the "gtk_show_uri" and "gnome_open" calls. It would be preferable if gparted also worked well on KDE too. What do you think of also trying "xdg-open" to open a file manager in case "gtk_show_uri" or "gnome-open" fails? For example: xdg-open file:/tmp/gparted-roview-1EgoLm/
Hi Joan, Would you be able to incorporate another change to a string in the Dialog_Rescue_Data.cc file? From: "GParted was unable to open a file manager for this file system." To: "Unable to open a file manager for this file system."
Hey, Happy Year, > What do you think of also trying "xdg-open" to open a file manager in case > "gtk_show_uri" or "gnome-open" fails? Why not to call xdg-open instead of gnome-open?
That is a good question. My thoughts were to try to make this as robust as possible so that it works on as many systems as possible. I was thinking something like the following code: #ifdef HAVE_GTK_SHOW_URI gtk_show_uri( gscreen, uri .c_str(), gtk_get_current_event_time(), &error ) ; #else Glib::ustring command = "gnome-open " + uri ; gdk_spawn_command_line_on_screen( gscreen, command .c_str(), &error ) ; #endif if ( error != null ) { Glib::ustring command = "xdg-open " + uri ; gdk_spawn_command_line_on_screen( gscreen, command .c_str(), &error ) ; } This way xdg-open would be tried as a backup if either of gtk_show_uri or gnome-open was not available. If you have another idea for improvement I am open to suggestions.
Hmm... I just did some testing with this and it did not work as I expected. I think gtk_show_uri should work on both GNOME and KDE. Now I am thinking that my KDE environment somehow does not have a default association for "file://path-to-file".
Hi Joan, I have not yet been able to figure out why my KDE install does not open a file manager with the gtk_show_uri call. Hence let us not worry about adding "xdg-open" since it does not resolve this problem. While testing the code I came across another string in the Dialog_Rescue_Data.cc file that we could improve: Change from: An error was encountered while creating the read-only view. To: An error occurred while creating the read-only view.
Created attachment 177820 [details] [review] Clean patch Hi Curtis, I changed the strings that you told me. Here's the new patch By the way, in the holidays I've learned to use git and I made a clean patch that contains only the differences with current origin/master
Thanks Joan for making this a single patch. This will make it easier for me to do one more review. If you are in agreement, I would like to commit this enhancement to the master branch soon to give translators extra time before the next GParted release.
OK, go ahead
This time when I reviewed the patch, I concentrated on code that might not be used because we changed from restoring partitions to rescuing data from partitions. Following are some suggestions for improvement: 1) The method restore_extended_partition() is no longer used (GParted_Core.h and GParted_Core.cc). 2) The methods thread_restore_partitions() and activate_table_restore() are not used (Win_GParted.h). 3) The structure std::vector<Partition> restore_list is not used (Win_GParted.h). 4) The variable bool restore_result is no longer used (Win_GParted.h). 5) The file Dialog_Table_Restore.cc no longer exists (po/POTFILES.in). 6) The file Dialog_Rescue_Data.cc should be added to po/POTFILES.in. 7) In Dialog_Rescue_Data.cc, add examples to translation lines that contain string substitutions. These are the strings I found: this ->set_title( String::ucompose( _("Attempt to rescue data on %1"), this->device_path ) ); 8) The following string in Dialog_Rescue_Data.cc should probably be available for translation because it contains some text, even if this is only "MiB". This string would also need an example for translators. Gtk::Label *fsLbl= manage(new Gtk::Label( String::ucompose("#%1: %2 (%3 MiB)", i+1, fs_name, (this->partitions[i].get_byte_length()/1024/1024)))); 9) There is an extra space in front of the following string. If not needed then this space should be removed. Glib::ustring error_txt=_(" An error occurred while creating the read-only view."); 10) The following translator comment should include the ":" and a line feed before the "/tmp/gparted-roview-Nlhb3R." /*TO TRANSLATORS: looks like The file system is mounted on /tmp/gparted-roview-Nlhb3R. */ Please feel free to ask if you have any questions about the above suggestions.
Created attachment 178270 [details] [review] Added support to lost data recovery using gpart Hi, I agree with your suggestions, here is a new patch with the last changes.
Thanks again for the updated patch Joan. I will review once more and do some more testing.
Created attachment 178646 [details] GParted data recovery attempt showing button to view extended partition Hi Joan, In the attached picture the user is shown the option to view an extended partition. Since it is not possible to view an extended partition because it is not a file system partition, would you be able to prevent this option from displaying? Either that or perhaps there is a better way to handle this situation. As a side note, I have found that ntfs file systems tend to be reliably recognized by gpart.
Ok, I've been taking a look at the code and I found the error. I hope to send a patch in the coming days.
While creating the documentation for this new feature, I came across another string for enhancement. Would you be able to change the following string in Win_GParted.cc? From: Search for file systems To: Search for file systems on %1 Where: %1 is replaced by the path to the disk device. Since this would involve a substitution it is a good idea to add a comment for translators with an example with the %1 value substituted.
While working on the documentation for attempting data recovery I came across another similar string for enhancement. Would you be able to change the following string in Win_GParted.cc? From: No file systems found To: No file systems found on %1 Where: %1 is replaced by the path to the disk device.
Hi Joan. Thanks again for your patience with me while developing this enhancement. I am finding that writing this documentation is really helping me with identifying strings to improve for better readability and consistency. To better align with the string update in comment #118, would you be able to change the following string in Dialog_Rescue_Data.cc? From: Attempt to rescue data on %1 To: File systems found on %1 Where: %1 is replaced by the path to the disk device.
Hi Curtis, can you tell to this user if he can reproduce the error and send me the output of the command "gpart /dev/sde -s <sector size>" ?
Hi Joan, I apologize but I am not sure what you intended in your request in comment #120. Is there a specific person or user that is experiencing a problem that you would like me to contact to request information? Or did you have something else in mind? If so please try to explain your request in different words.
I was referring to the comment #115, the "user" is shown the option to view an extended partition. I need the output of running gpart /dev/sde -s <sector_size> in this scenario
Hi Joan, I certainly can do. Here is the output from running the requested command: gedakc@quad:~$ sudo gpart /dev/sde -s 512 Begin scan... Possible partition(Windows NT/W2K FS), size(2mb), offset(1mb) Possible partition(Windows NT/W2K FS), size(3mb), offset(5mb) Possible extended partition at offset(8mb) Possible partition(Windows NT/W2K FS), size(3mb), offset(9mb) Possible partition(Windows NT/W2K FS), size(2mb), offset(17mb) * Warning: short read near sector(48992), 65536 bytes instead of 66048. Skipping... End scan. Checking partitions... Partition(OS/2 HPFS, NTFS, QNX or Advanced UNIX): primary Partition(OS/2 HPFS, NTFS, QNX or Advanced UNIX): primary Partition(OS/2 HPFS, NTFS, QNX or Advanced UNIX): logical Partition(OS/2 HPFS, NTFS, QNX or Advanced UNIX): invalid Ok. Guessed primary partition table: Primary partition(1) type: 007(0x07)(OS/2 HPFS, NTFS, QNX or Advanced UNIX) size: 2mb #s(4096) s(2048-6143) chs: (2/0/1)-(5/31/32)d (2/0/1)-(5/31/32)r Primary partition(2) type: 007(0x07)(OS/2 HPFS, NTFS, QNX or Advanced UNIX) size: 3mb #s(6144) s(10240-16383) chs: (10/0/1)-(15/31/32)d (10/0/1)-(15/31/32)r Primary partition(3) type: 015(0x0F)(Extended DOS, LBA) size: 11mb #s(22528) s(16384-38911) chs: (16/0/1)-(37/31/32)d (16/0/1)-(37/31/32)r Primary partition(4) type: 000(0x00)(unused) size: 0mb #s(0) s(0-0) chs: (0/0/0)-(0/0/0)d (0/0/0)-(0/0/0)r gedakc@quad:~$ If you would like, I can create a script that creates the SCSI_DEBUG device and each of these partitions too.
OK, the script can help. Thanks
Created attachment 179108 [details] [review] Added support to lost data recovery using gpart Hi Curtis, here's the patch with the bug fixed and the strings changed.
Created attachment 179231 [details] init-partition script to setup a disk device with some formatted partitions Thank you Joan for the updated patch. I retested with this patch and it no longer displays the option to view data from an extended partition. :-) In case you still need it, I developed a script called "init-partitions" to set up the example partition layout I used for my tests. To set up the SCSI_DEBUG device in RAM for testing, I use the following command: sudo modprobe scsi_debug dev_size_mb=24 From my testing, I think the patch will permit "data recovery" for the first four primary partitions only. Is this correct? If so, we might want to inform our users of this limitation before they initiate a full disk scan. What are your thoughts on this?
Created attachment 179296 [details] Updated init-partitions script to setup a disk device with some formatted partitions Attached is an updated init-partitions script. The previous one did not leave a full MiB between logical partitions which is how GParted creates MiB aligned logical partitions.
Hi Curtis, the patch has the same limitations that gpart command, which has a maximum limit of 4 partitions detected and is not necessarily to be primary, it may detect logic partitions, by 4 as max.
Created attachment 179312 [details] GParted data found with inconsistensies makes a wide window Thanks for the clarification Joan. I will add something to the documentation similar to what you have stated as gpart's limitations. When I have a first draft of the documentation I will post a copy here for you to review. On another note, I have another string enhancement suggestion. Would you be able to split the following string into two separate sentences with a line feed between these sentences? From: WARNING!: The file systems marked with (!) are inconsistent. You might encounter errors trying to view these file systems. To: WARNING!: The file systems marked with (!) are inconsistent. + "\n" You might encounter errors trying to view these file systems. That way the string should not cause the windows to stretch as wide as in the attached screen shot.
Created attachment 179339 [details] [review] Added support to lost data recovery using gpart Patch with the string updated.
Hi Joan, I think we are very close to having this patch ready for release. I have almost finished my first draft of the documentation. While documenting I came across another dialog box that I think we can improve. This dialog box is in Dialog_Rescue_Data.cc and uses two different terms: file browser and file manager. I believe that we should be consistent and use file manager because that is what nautilus calls itself. Current the dialog box looks like this: Unable to open the default file browser Unable to open a file manager for this file system. The file system is mounted on: /tmp/gparted-ro-view-XXXXXX Error: %error-message% I suggest that we can remove some duplication and make the dialog box look like this: Unable to open the default file manager Error: %error-message% Note: The file system is mounted on: /tmp/gparted-ro-view-XXXXXX I am open to other suggestions for improvement too. :-)
Created attachment 179526 [details] [review] GParted help manual updates for Attempt Data Rescue The attached patch includes updates for the new "Attempt Data Rescue" feature. After applying the patch, you can view the gparted.xml help manual with the following command: yelp ghelp:/full-path-to-gparted.xml Joan and Seth, please let me know if you have suggestions for improving these manual updates or any section of the manual.
Created attachment 179536 [details] [review] Added support to lost data recovery using gpart Hi, this patch contains the last changes in the dialog. How can I read the xml attached?
The GParted help manual is written in gnome-doc format. The application to view gnome-doc formatted documents is yelp. To view the help manual, the patch in comment #132 should be applied to the GParted source code from the git repository. E.g., git am -3 help-manual-updates.patch To view the help manual, use the following command: yelp ghelp:/full-path-to-the-file-gparted.xml
I use "git am" and it returns an error: "Patch format detection failed."
Created attachment 179539 [details] GParted help manual including updates for attempt data rescue I wonder if that is because I did not commit these help manual updates yet? I generated the patch with "git diff > help-manual-updates.patch". I have attached the complete gparted.xml file including the updates for attempt data rescue. You should be able to download gparted.xml and place it in the .../gparted/help/C directory. Then you can use the following command to view the help manual: yelp ghelp:/full-path-to-gparted.xml
The manual seems ok.
Hi Joan, thanks for the review of the manual. I am ready to commit your patch to the master branch of the git repository. Are there any last changes you would like to make? The only thing I could think of is perhaps updating the copyright to include both 2010 and 2011, but this is minor. We should also plan on creating a GParted release (0.8.0) for sometime in February. We will need to give the translators sufficient time to do their work, so I would think Feb 12th at the earliest. Ideally we should pick a date when both of us will be available for at least a few days afterwards in case any nasty bugs are discovered after release. Does Feb 12th as a release data work for you? If not is there another date later on that would work better?
Hi Curtis, I usually have much less time than a few months ago, so for me no matter what day it is released, I will be busy anyway :). The February 12th is Saturday, maybe would be appropriate to launch it on Monday or Tuesday, if any errors probably I can fix it the same weekend. If it launches on Saturday, and Sunday or Monday someone discovers an error, Maybe I can't work until the following weekend.
That is a good suggestion Joan. Tuesday, Feb 15th, 2011 sounds like a good day to me and provides the translators with some extra time. I plan to commit your patch and the documentation updates later today.
The patch by Joan to add the new feature to attempt data rescue for lost partitions has been committed to the git repository. Updated documentation has also been committed for inclusion in the next release of GParted (0.8.0). The relevant git commits can be viewed at the following links: Added support to lost data recovery using gpart http://git.gnome.org/browse/gparted/commit/?id=ef37bdb7de1bbdc145aa40ae6e66a88e0e3e5ae9 Add attempt data rescue section to help manual (#171215) http://git.gnome.org/browse/gparted/commit/?id=614a781d2faa22c9b0f16701d8db5debdf9502ce A big thanks goes to Joan for developing this new feature for GParted. :-)
Hi, I saw the commit. The day 15 is ok. If someone finds a bug in the patch, how can I know it?
There are several ways for our users to contact the GParted Project. These are via bug reports, discussion forum, email, and Internet Relay Chat. We recommend to GParted users to use the forum and bug reports. Since I follow each of these, I can forward reports of bugs to you. If you wish, you can indicate in bugzilla to follow the gparted bug reports. You can also sign up as a forum member in the GParted forums and then check for new posts from time to time.
Thanks again Joan for creating the "attempt data rescue" patch. GParted 0.8.0 was released on Feb 15, 2011. As such I am closing this bug report.