GNOME Bugzilla – Bug 671784
enhancement: convert partitions from one file system to another
Last modified: 2020-12-12 06:31:22 UTC
With approval from gparted maintainer, I am working on integrating my program 'fstransform' (http://sf.net/projects/fstransform) with gparted. Fstransform can convert the contents of a partition from one file system to another, in-place and without need for backup (it is obviously a RISKY operation, even if fstransform tries to be as safe as possible). It is currently tested only on Linux, and supports converting between the following file systems: minix ext2 ext3 ext4 jfs reiserfs xfs. Historical background: The only other known program that tries to accomplish something similar is 'convertfs', but it seems unmaintained since 2005. In 2006, there was even a request to integrate convertfs into gparted - see "Bug 353065 - convert partitions from between filesystems and other advanced features" (https://bugzilla.gnome.org/show_bug.cgi?id=353065) but it was closed exactly because convertfs was unmaintained. Current status: I am writing a patch for gparted to add support for fstransform. A tiny patch for fstransform is also ready - it is useful to simplify the integration. I expect some discussion and requests for improvements (my initial patch is a bit invasive).
Created attachment 209417 [details] [review] add support for fstransform - first version
Please find above the first version of my patch to add support for fstransform. It still has some shortcomings and rough edges that I would like to hear an opinion about: 1) it is a bit invasive. While somewhat necessary, I am not sure if it's acceptable for the maintainer 2) gparted allows formatting (and in my patch, transforming) only UNMOUNTED devices. Instead fstransform expects the device to be MOUNTED. I plan to solve this "conflict" by patching fstransform to work both on MOUNTED and UNMOUNTED devices. At the moment it means that performing a transformation from gparted will fail... 3) I added two more columns "Transform (from)" and "Transform (to)" to DialogFeatures. The labels are probably too long, and the dialog was overcrowded already: two more columns are probably not so welcome 4) fstransform can take a VERY LONG time to complete. It shows progress percentage and estimated time left on standard output, and I know how to parse them, but I do not know how to read them from gparted and pass information to the progress bar. Can someone point me to an example of how to interact with gparted progress bar using data from an external command still in progress?
*** Bug 353065 has been marked as a duplicate of this bug. ***
Thank you Massimiliano for creating this bug report and posting the initial patch to add support for fstransform. I noted that a Debian package for fstransform has been created: http://packages.debian.org/search?suite=default§ion=all&arch=any&searchon=names&keywords=fstransform This is great news and will make it much easier to include fstransform on the GParted Live image since GParted Live is based on Debian Sid. :-) I plan to review your patch this week. In the meantime I can provide some answers. 2) See xfs::copy() in xfs.cc for examples of how to create temporary directories and mount/unmount file systems. 3) Can the file systems supported by fstransform be used as both a source and a destination? If so, is there a need for both columns "Transform (from)" and "Transform (to)"? Would "Transform" be sufficient? 4) For the moment do not worry about progress reporting. Phillip Susi is working on an enhancement to improve progress reporting in GParted. The problem with progress reporting is not unique to fstransform. See Bug #467925 - gparted: add progress bar during operation
Hello Curtis, I am trying to give visibility to fstransform, so asking a friend to package it in Debian was an obvious choice. Pity I do do not know any maintainer for other distributions... Thanks for the pointer on how to create temporary directories, I will do that on fstransform side, since it make much more sense to transform an unmounted file system (and it's safer, no risk of processes using it). I consider an "implementation detail" the fact that in order to actually transform a file system, it must be mounted temporarily during some of the internal steps. About "source" and "destination" file systems: they have different requirements, so in theory the two lists can be different. But most modern POSIX file systems satisfy both requirements, so they can be in both columns (as soon as somebody manages to test fstransform on them). Non-Unix file systems, as FAT, FAT32 and NTFS are a different story. So, at least for the moment I am fine if you want to collapse the two columns to simply "Transform" - each box will be checked if (and only if) a file system can be BOTH "source" and "destination". I will update my patch correspondingly. About reporting progress: I know how to do it with GTK+ plus low-level POSIX calls - as long as the output produced by a program is reasonably parseable - but I will happily leave it to who is working on the topic already.
Created attachment 209660 [details] [review] add support for fstransform - 2nd version merged DialogFeatures columns Transform (from) and Transform (to) into a single column "Transform". Less precise, but the dialog was overcrowded already
I am also posting here a patched version of fstransform. The patch will be included in fstransform 0.9.3, and it simplifies the integration with gparted. In particular, the patched fstransform does NOT require the device to transform to be already mounted, and it add the options --list-source-file-systems and --list-target-file-systems. In this way, running fstransform from gparted will "just work".
Created attachment 209669 [details] fstransform 0.9.2.work2 - contains patches to better integrate with gparted patched version of fstransform to simplify integration with gparted. The patches will be included in fstransform 0.9.3
Thank you for the updated patches Massimiliano. I compiled and installed the fstransform version in comment #8 and then applied the patches in comment #6 and compiled GParted. To test I created an empty 512 MiB ext2 file system. Then I queued and operation to transform this to an ext4 file system. When I applied the operation, fstransform exited stating that it was unable to mount a loopback device for ext2. I would have saved and copied the file, but weird things began to occur. My system began to act strangely and got progressively worse until I could not close windows or even bring up a menu to shutdown properly. I ended up hard booting to recover. My testing was with Kubuntu 11.04 which uses Linux kernel 2.6.38. Is there some minimum version of the Linux Kernel, or perhaps some key program missing for which we need to check? For an example of a kernel version check see nilfs2::get_filesystem_support() in xfs.cc. From a very quick review of the code, I have the following suggestions for improvement: 1) Remove other names from the copyrights for new files you created. For new files created by you, you are the author with the copyright. Hence you can remove my name from the new files you created. I know that these files often start as a copy of another file but that is only for convenience. ;-) 2) Remove trailing whitespace from the patch. When I applied the patch, git indicated to me that 17 lines had trailing whitespace. Even though the GParted code is riddled with trailing spaces and tabs, I would like to work towards slowly cleaning this up. Hence with new patches I do not want to make the problem worse. To maintain the ability to use git blame with the code, we do not wish to remove all trailing whitespace from the GParted code. Instead we plan to chip away at the problem by removing the trailing whitespace when we modify lines of code. If you use emacs, you can view the trailing whitespace by using the tips at the following link: http://trey-jackson.blogspot.com/2008/03/emacs-tip-12-show-trailing-whitespace.html 3) Add new files to the appropriate Makefile.am file. For example Fstransform.h should be added to include/Makefile.am Similarly Fstransform.cc should be added to src/Makefile.am Files included in Makefile.am will be automatically included in a tarball when we run "make dist". Also I wonder if we should name this class FSTransform to make the acronym stand out better (File System Transform). 4) Remove the reference to Windows C: in warning Since fstransform is not able to covert FAT16, FAT32, or NTFS file systems, we should remove the warning text that refers to Windows C: because this situation does not occur. 5) Try to be consistent with indentation. The GParted code is far from perfect regarding consistency with indentation. Ideally we prefer tabs to indent one level. We use spaces to make multi-line commas and brackets look pretty. With each review I might discover more areas for improvement. Your patience is appreciated, and do feel free to question any of my suggestions. You might also have some improvements of your own. :-)
On 03/14/2012 06:45 PM, gparted (bugzilla.gnome.org) wrote: > https://bugzilla.gnome.org/show_bug.cgi?id=671784 > gparted | application | GIT HEAD > > --- Comment #9 from Curtis Gedak <gedakc@gmail.com> 2012-03-14 17:45:42 UTC --- > Thank you for the updated patches Massimiliano. > > I compiled and installed the fstransform version in comment #8 and then applied > the patches in comment #6 and compiled GParted. > > To test I created an empty 512 MiB ext2 file system. Then I queued and > operation to transform this to an ext4 file system. Hello Curtis, thanks for taking the time to test the integration between gparted and fstransform. > When I applied the operation, fstransform exited stating that it was unable to > mount a loopback device for ext2. I would have saved and copied the file, but > weird things began to occur. > > My system began to act strangely and got progressively worse until I could not > close windows or even bring up a menu to shutdown properly. I ended up hard > booting to recover. I apologize for the problem you experienced. This is VERY unusual, but without a more detailed error report, it's difficult for me to help much. Fstransform writes log files in /var/tmp/fstransform/fstransform.log.* and /var/tmp/fstransform/fsremap.job.*/fsremap.log. Being in /var/tmp, they should survive reboots. Can you check if they are still there, and send them? I placed a lot of detailed checks and error reporting in fstransform, and the logs should shed light on the problem you experienced. One word of warning: fstransform intentionally uses a lot of RAM: it autodetects amount of free RAM and allocates more than 50% of it. Starting LARGE programs while it runs can be a bad idea. > My testing was with Kubuntu 11.04 which uses Linux kernel 2.6.38. I have seen problems with Ubuntu when it's installed INSIDE a Windows disk (using a huge loop file), but it should not be your case. Maybe the amount of free RAM was misdetected? If you can give me a bit more details about your system (at least the output of the commands "free", "cat /proc/meminfo", "uname -a", "g++ -v" and "ldd `which fsremap`") in addition to the log files, I will try to figure out what went wrong. > Is there some minimum version of the Linux Kernel, or perhaps some key program > missing for which we need to check? > > For an example of a kernel version check see nilfs2::get_filesystem_support() > in xfs.cc. fstransform checks for required programs at startup, and has no minimum kernel version, provided that loop files are supported: if they aren't, one of the required programs, 'losetup', will either be missing or will fail, and fstransform will notice. Same story for some of the low-level ioctls() used: if they are not supported, fstransform (actually fsremap) will fall back on simpler ones, and in the worst case it will quit with a detailed report of what failed. Unluckily, if you executed fstransform from gparted, you will see fstransform output only when it finishes: seeing external programs output while they run is one of the gparted proposed enhancements you mentioned. For initial evaluation of fstransform, I can suggest to run it from command line, and try the integration with gparted only once you are confident that it works. > From a very quick review of the code, I have the following suggestions for > improvement: > > > 1) Remove other names from the copyrights for new files you created. > [...] ok > 2) Remove trailing whitespace from the patch. > [...] > http://trey-jackson.blogspot.com/2008/03/emacs-tip-12-show-trailing-whitespace.html ok > 3) Add new files to the appropriate Makefile.am file. ok > Also I wonder if we should name this class FSTransform to > make the acronym stand out better (File System Transform). it sounds a bit heavy capitalization to me... > 4) Remove the reference to Windows C: in warning ok > 5) Try to be consistent with indentation. No problem. Everybody has his own ideas about indentation, and it makes sense to keep uniformity within the code for the same project. I will release shortly an updated patch to address your suggestions. In the meantime, I would greatly appreciate if you find fstransform logs and forward it to me (or attach to this bug report, if you prefer). Sincerely, Massimiliano
P.S. sorry if this looks like you're testing fstransform rather than the gparted patch. I devoted a lot of effort in fstransform, but should it turn out that it's not yet stable enough, I will be the first to withdraw the proposal to integrate it until it's more mature.
Created attachment 209777 [details] fstransform log file - Strange behaviour afterwards > I apologize for the problem you experienced. No worries. These things can happen during development. :-) Thanks for informing me where the fstransform log is stored. The log is attached to this post. My system is an Asus P8H67-M Pro/CSM Motherboard with 8 GB RAM and an intel core 2 i7-2600K. For disk devices I have a 500 MB SATA drive, a 160 MB IDE drive, and two 160 GB SATA drives set up as mirrored RAID in the BIOS. Output from the commands you requested is as follows: $ free total used free shared buffers cached Mem: 8025048 5809664 2215384 0 141384 3642708 -/+ buffers/cache: 2025572 5999476 Swap: 16777212 0 16777212 $ cat /proc/meminfo MemTotal: 8025048 kB MemFree: 2203480 kB Buffers: 141444 kB Cached: 3651056 kB SwapCached: 0 kB Active: 2235940 kB Inactive: 3191068 kB Active(anon): 1688216 kB Inactive(anon): 1146156 kB Active(file): 547724 kB Inactive(file): 2044912 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 16777212 kB SwapFree: 16777212 kB Dirty: 2148 kB Writeback: 0 kB AnonPages: 1634524 kB Mapped: 1321596 kB Shmem: 1199872 kB Slab: 123324 kB SReclaimable: 80876 kB SUnreclaim: 42448 kB KernelStack: 4360 kB PageTables: 52276 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 20789736 kB Committed_AS: 4796328 kB VmallocTotal: 34359738367 kB VmallocUsed: 383196 kB VmallocChunk: 34359185800 kB HardwareCorrupted: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 81920 kB DirectMap2M: 8148992 kB $ uname -a Linux octo 2.6.38-02063802-generic #201103281246 SMP Mon Mar 28 12:50:24 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux $ g++ -v Using built-in specs. COLLECT_GCC=g++ COLLECT_LTO_WRAPPER=/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.5.2/lto-wrapper Target: x86_64-linux-gnu Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.5.2-8ubuntu4' --with-bugurl=file:///usr/share/doc/gcc-4.5/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.5 --enable-shared --enable-multiarch --with-multiarch-defaults=x86_64-linux-gnu --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib/x86_64-linux-gnu --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.5 --libdir=/usr/lib/x86_64-linux-gnu --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-plugin --enable-gold --enable-ld=default --with-plugin-ld=ld.gold --enable-objc-gc --disable-werror --with-arch-32=i686 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu Thread model: posix gcc version 4.5.2 (Ubuntu/Linaro 4.5.2-8ubuntu4) $ ldd `which fsremap` linux-vdso.so.1 => (0x00007fffaa5fd000) libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fcd0b0de000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fcd0ae59000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fcd0ac42000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fcd0a8ad000) /lib64/ld-linux-x86-64.so.2 (0x00007fcd0b40e000) If you need any more information then please let me know.
Oops, missed responding to this comment. >> 3) Add new files to the appropriate Makefile.am file. > > ok > >> Also I wonder if we should name this class FSTransform to >> make the acronym stand out better (File System Transform). > > it sounds a bit heavy capitalization to me... I suppose this could be considered heavy capitalization. By uppercasing the extra letters though this should better match the current naming scheme. For instance: DMRaid.h which is short for Device Mapper RAID. FS_Info which is short for File System Information. LVM2_PV_Info which is short for Logical Volume Management 2 Physical Volume Information. At least that is my reasoning behind the suggestion. I know it's a bit picky. ;-)
Thanks a lot for the quick reply and the log file. This is the problem reported by fstransform in your logs: 2012-03-14 10:47:00 fstransform: unmounting and running '/sbin/fsck' (disk check) on loop file '/tmp/fstransform.mount.7191/.fstransform.loop.7191' 2012-03-14 10:47:00 umount: Segmentation fault I am sure you don't need me to know that a segmentation fault inside "umount" is very difficult to cause with a software bug. At the point where fstransform stopped for you, it didn't do yet any low-level disk access bypassing the file systems, so it cannot cause (in a worst case scenario) any file system corruption that could justify the above error. Honestly, if this happened on my PC, I would suspect that my Linux installation is corrupted, or that I have some faulty RAM. I don't know if this can be the case for your PC or you know you can exclude it... especially because I guess that by know you would have realized already a problem of this magnitude.
I agree with your conclusion regarding it being difficult for a software bug to invoke a segmentation fault in another program. Since my system has been rock steady until running GParted with fstransform, I do not immediately suspect my system. Of course RAM can go bad, and other hardware can fail so I will keep an eye on it. With your next patch I will test in a virtual machine with a more recent GNU/Linux distribution.
Created attachment 209787 [details] [review] add support for fstransform - 3rd version removed other names from the copyrights of new files, removed trailing spaces from patch, added new files to include/Makefile.am (src/Makefile.am was done already), removed reference to Windows C: in warning, used tabs for indentation
Hello Curtis, here is an updated patch addressing your considerations. After the problem you reported, I became suspicious and tested again launching fstransform from gparted (patched with the 3rd version of the patch). So far, it succeeded with no strange effects - but my PC is a Debian amd64, I have Ubuntu 10.04 amd64 on a laptop but it's not with me at the moment
Massimiliano, would you be able to rebase and recreate the patch using the latest git version of GParted? For some reason I receive an error about a corrupt patch when I apply the patch from comment #16. $ git apply bug671784-fstransform.patch bug671784-fstransform.patch:192: trailing whitespace. fatal: corrupt patch at line 304 $ As the message states too, it looks like there is some trailing whitespace on line 192 of the patch.
Created attachment 209889 [details] [review] add support for fstransform - 4thrd version
Created attachment 209891 [details] [review] add support for fstransform - 5th version It's the first time I use git, I am starting to realize it is MUCH more picky than patch(1). The 4th version still gives some space-related warnings, so here is another one. Sorry for my ignorance on git...
No apologies needed. We are all learning. Hopefully something new each day. :-) This new patch applied with no problems. With my first test I ran into three problems: 1) Only file systems of a valid size should be available for transformation. I created a 10 MiB EXT2 partition and applied this operation. Then I queued a transform to JFS operation. When I applied this operation it failed because the minimum size supported by JFS is 16 MiB. You might look at the code for "Partition --> Format to" to see how this is handled (a warning is displayed). 2) After failure, the ext2 file system remained mounted. In the above test, the ext2 file system was still mounted after the failed transform action. In my opinion the mount status of the file system should be returned to the state it was in prior to applying the operation. GParted does this when it mounts file systems. See xfs.cc for an example. Is there a reason why the mount/umount is not performed in GParted? 3) When I tried to use GParted "Partition --> Unmount" option the unmount failed. The error message from GParted on unmount is: Could not unmount /dev/sdb31 umount: /tmp/fstransform.mount.9663: device is busy (In some cases useful infor about process that use the device is found by lsof(8) or fuser(1)) Using the above steps in #1 you should be able to recreate this problem 4) File system incorrectly recognized after fstransform. In my next test I created a 100 MiB EXT2 file system and applied the operation. Then I queued a transform to XFS operation. I was able to successfully apply the transformation operation. However, upon refreshing devices by GParted, I received a warning indicator beside the partition. The problem originates because parted still thinks the partition is formatted as ext2. You can check by running: sudo parted /path-to-disk-device print to see what parted thinks file system is on the partition. As a further test I tried rebooting my test Virtual Machine (Ubuntu 11.10) and then checking again. At this time the transformed file system was recognized as JFS. Hence there is something strange with how parted sees the partition format after it is transformed. A reboot appears to clear the problem. At this point I stopped testing. Please note that I did not review the code this time.
Hello Curtis, thanks for the time and effort you are dedicating to evaluating fstransform. You definitely went beyond your "job" by testing it, and your comment is both flattering and frustrating... Flattering, because I found some of the comments received by convertfs (which fstransform is trying to replace) after 5 years it was released: in comparison the issues you report are annoying for a user but (mostly) straightforward to fix. Frustrating, because you reminded me that fstransform is still BETA, and you found a lot of the reasons WHY it is BETA. > 1) Only file systems of a valid size should be available for transformation. I was not aware of such minimum file system sizes. I will check the "Format to" command and implement an analogous solution > 2) After failure, the ext2 file system remained mounted. > 3) When I tried to use GParted "Partition --> Unmount" option the unmount failed. Cleanup after a failure is missing in fstransform. > 4) File system incorrectly recognized after fstransform. This is a bigger issue. It happened once to me, but I still have to understand how to reproduce reliably in order to investigate and fix it. In summary: fstransform is still BETA and it has some known issues. I would not recommend shipping it as STABLE, or together with a gparted version supposed to be STABLE. Sorry if this was not clear enough from the beginning... Anyway, if there is something as a TESTING/UNSTABLE version of gparted, integrating fstransform there (until it's more stable) could make more sense.
Massimiliano, for the time being I agree with you that it is probably best to hold off on shipping GParted and fstransform together as a STABLE release. Unfortunately we do not ship a TESTING version of GParted. The testing release of GParted Live is meant more to deal with the problems involved when some of the underlying file system tools change. Hence the GParted Live testing branch is not the proper place either because GParted is supposed to be a STABLE product in the testing branch. Having said that I would like to work towards the improvement of both fstransform and its use in GParted. It is difficult for one person to think of and find all of the problems and use cases for any piece of software. I am not discouraged by the problems encountered to date. Hopefully you are not discouraged either. Please do continue your work to improve fstransform, and to enhance GParted to work with fstransform. :-)
Hello Curtis, I am quite stubborn, don't worry. No way I am giving up :) I will prepare another gparted patch addressing minimum and maximum size of file systems as you suggested. The difficult part I am trying to organize is to find beta testers for fstransform... it's very difficult and time consuming for a single person (and especially the author) to find all possible issues in all possible usage scenarios, or even just a fair percentage of them.
Sometimes a project needs to reach a certain amount of usage before extra help becomes available. Also since all of this work is done by volunteers, it can only progress as fast as we have free time to devote to the project. And you have at least one beta tester - me. ;-) I will continue to test your GParted patches, and hence fstransform, when I have free time too.
Created attachment 211667 [details] [review] add support for fstransform - 6th version Here I am again :) in the attachment, a new gparted patch that also adds the check for "minimum and maximum filesystem size" when transforming a partition. It is intended to be used with fstransform 0.9.3 - published recently, downloadable from http://sourceforge.net/projects/fstransform/files/fstransform-0.9.3/fstransform-0.9.3-src.tar.bz2/download - it adds proper cleanup in case of failure and, VERY important, the ability to resume interrupted remapping jobs (remapping is the last step of the transformation). It is also possible to use the patch with fstransform version 0.9.2.works attached above, but of course you will miss the enhancements in 0.9.3. I can only add that fstransform STILL remains BETA at the moment...
Any news on this topic? You clearly needed your time to pick up with the activity when you returned from holiday, but as I did not hear any news, I am wondering whether there is something you're waiting for me to do first? Beyond finding more beta-testers, of course...
Hi Massimiliano, Thank you for checking. My life has been very busy as of late, and I have many priorities I have to deal with before I can find some more free time to look into this and some other enhancements. I have not forgotten though, and even if it is a few more months before I can free up the time, this is certainly on my list of tasks to follow up on. Curtis
Hi Massimiliano, Thank you for your patience with this enhancement. If you have the time, I would like to review your patch set. Before starting review, I have a couple of questions: 1) Is there a new or development version of fstransform that you would like me to test with? 2) Would you be able to rebase your patch set to the most recent git version of GParted? Curtis
Hello Curtis, no problem, real life takes precedence for everyone :) 1) the latest fstransform version is still 0.9.3. I have continued the testing, and some people helped too. 2) of course I can, I will update the patch this weekend and re-submit it. Regards, Massimiliano
Created attachment 219447 [details] [review] add support for fstransform - 7th version Here is the updated patch. I checked for gparted API modifications and adapted the patch for changes in Partition.h, then re-built gparted and tested it. On a not-so-unrelated topic, which signal does gparted send to external programs when the user clicks the "Cancel" button _while_ operations are being applied? fstransform catches practically all possible signals (HUP INT QUIT ILL TRAP ABRT BUS FPE KILL SEGV PIPE TERM URG XCPU XFSZ VTALRM PROF IO PWR SYS) to perform appropriate cleanup and exit if interrupted, yet gparted "Cancel" button kills it without killing child processes - not good, and I don't understand how it can happen Massimiliano
Thanks massimiliano for the updated patch. (In reply to comment #31) > On a not-so-unrelated topic, which signal does gparted send to external > programs when the user clicks the "Cancel" button _while_ operations are being > applied? > > fstransform catches practically all possible signals (HUP INT QUIT ILL TRAP > ABRT BUS FPE KILL SEGV PIPE TERM URG XCPU XFSZ VTALRM PROF IO PWR SYS) to > perform appropriate cleanup and exit if interrupted, yet gparted "Cancel" > button kills it without killing child processes - not good, and I don't > understand how it can happen. Another developer, Phillip Susi, brought this issue to my attention a while ago. It seems that there are problems with Glib::Thread and GParted. As such Phillip is working to remove threads, and instead spawn commands asynchronously. Phillip's work to date can be viewed in the psusi/refactor branch of the GParted git repository.
Created attachment 219710 [details] gparted_details.htm log file for failure to transform 200 MiB ext2 to xfs. Hi Massimiliano, I've done some testing of GParted with fstransform and things are improving. :-) For my testing I am using Ubuntu 12.04 in a virtual machine. Using the tests in comment #21, the outcome was as expected in most of the test cases. Step 1 from comment #21 completed successfully. Hence steps 2 and 3 were not required. One minor change suggestion to the dialog for step 1: A) Split the word "filesystem" into the two words "file system" in the following dialog message: Cannot transform this file system to jfs. A jfs filesystem requires a partition of at least 16.00 MiB. ^^^^^^^^^^ With step 4, when it completes successfully, the new file system type is now recognized in GParted. :-) Unfortunately, sometimes this step fails an unmount step claiming that a device is busy. This might be a timing issue, or a problem with auto-mounting of file systems. The reason I suspect auto-mounting is because a file manager pops up and displays the contents of the /tmp directory when step 4 is running. The gparted_details.htm log file is attached for when I tried to transform a 200 MiB ext2 partition to xfs. Curtis.
(In reply to comment #33) > Created an attachment (id=219710) [details] > gparted_details.htm log file for failure to transform 200 MiB ext2 to xfs. > > Hi Massimiliano, > > I've done some testing of GParted with fstransform and things are > improving. :-) > For my testing I am using Ubuntu 12.04 in a virtual machine. > > Using the tests in comment #21, the outcome was as expected in most of > the test cases. > > Step 1 from comment #21 completed successfully. Hence steps 2 and 3 > were not required. > > One minor change suggestion to the dialog for step 1: > > A) Split the word "filesystem" into the two words "file system" in the > following dialog message: > > Cannot transform this file system to jfs. > A jfs filesystem requires a partition of at least 16.00 MiB. > ^^^^^^^^^^ Yes, it's a typo, sorry. I will fix it. > With step 4, when it completes successfully, the new file system type > is now recognized in GParted. :-) Good > Unfortunately, sometimes this step fails an unmount step claiming that > a device is busy. This might be a timing issue, or a problem with > auto-mounting of file systems. > > The reason I suspect auto-mounting is because a file manager pops up > and displays the contents of the /tmp directory when step 4 is > running. > > The gparted_details.htm log file is attached for when I tried to > transform a 200 MiB ext2 partition to xfs. > > Curtis. From the log file you attached it seems exactly an auto-opening problem, i.e. some daemon that detects the new device being mounted, and opens a file manager window that keeps the file system busy, preventing it from being unmounted. If this is the problem, the solution is simple but long-winded and error prone: we must know all kind of daemons that perform such auto-opening, and interact with them to convince them NOT to open a file manager window in this case. At least on Ubuntu, I have seen exactly the behaviour you describe: when plugging a USB pendrive, all partitions get automounted and a file manager window pops up for each of them.
(In reply to comment #34) > From the log file you attached it seems exactly an auto-opening problem, > i.e. some daemon that detects the new device being mounted, and opens a file > manager window that keeps the file system busy, preventing it from being > unmounted. If the operating system is automounting the file system, then perhaps something is now broken in Ubuntu 12.04. The reason I say this is because GParted uses a calling script to invoke hal-lock, devkit-disk, or udisk as appropriate to take out locks on the disk devices. See the source file gparted.in which becomes the script "gparted" on install. This was a problem a number of years ago that was solved by using hal-lock, etc.
Hi Massimiliano, The automount problem does appear to be related to changes in Ubuntu. It would appear that this behaviour changed in Ubuntu 11.10. In the versions prior to this (11.04), hal-lock is used to prevent automounting, and successfully prevents automounting and running more than one instanc of GParted. With Ubuntu 11.10 and higher, hal-lock appears to have been dropped in favour of udisks. When only udisks is available, GParted uses the following invocation command: udisks --inhibit -- /usr/local/sbin/gpartedbin $* The udisks documentation says the following: <--- Begin snippet ---> --inhibit [-- program arg ...] Inhibits clients from invoking methods on the daemon that require authorization (all methods will return the org.freedesktop.UDisks.Error.Inhibited error) if the caller is not the super user. This is typically used by OS installers and other programs that expects full control of the system, specifically to avoid automounting devices. Only the super user can do this. If no program is given, the daemon is inhibited until Ctrl+C is pressed. Otherwise the program is spawned and the daemon is only inhibited until the program terminates. <--- End snippet ---> From this I infer that the inhibit restriction only applies to non-root users. Because GParted is run as root, the --inhibit option does not prevent subprocesses of GParted from accessing the disk devices. Since this wasn't working, I also tried the udisks option --inhibit-all-polling but this too appears to fail to prevent automounting of file systems.
Hello Curtis, in my ignorance I was supposing all this automounting stuff to be pretty complicated and non-standard (read: distribution specific). From what you write, it is even MORE non-standard than what I expected. Anyway, for fstransform the problem does NOT seem to be devices automounting, but rather auto-OPENING the devices mounted by fstransform during its work: you reported file manager windows popping up, and the logs show that fstransform managed to mount the devices, but failed to unmount them because they were busy... why busy? my guess is what I wrote previously: busy because of file manager windows opened on them. I have even less experience on devices auto-opening than I have on automounting them, so if you have any experience on this topic it will be more than welcome. Regards, Massimiliano
Hi Massimiliano, The distinction between auto-OPENing versus auto-MOUNTing could be important in solving this problem. I suspect that we are simply missing one piece of the puzzle. Once we figure out the missing piece, we will be able to resolve this problem. I will spend some time thinking on this. Regards, Curtis
Hi Massimiliano, I haven't forgot about this problem with auto-OPENing. Recently I thought to myself "Resizing BTRFS file systems involves mounting and unmounting, so maybe this is also broken on Ubuntu 12.04". So with that in mind I fired up a VM with Ubuntu 12.04 and tested resizing BTRFS. What I discovered is that resizing BTRFS still works, even though it involves mounting and unmounting the file system. To me this means that there must be some other call or action other than mounting/unmounting in fstransform that is causing Nautilus to auto-OPEN the transformed file system. I don't know yet what that might be, but this realization might help us get to the root of the problem. Regards, Curtis
I think I run into a similar problem whenever I run the parted test suite. In addition to the udisks command, I also use unshare -m to fork the filesystem namespace so that any mounts created by the test suite will not be visible to the rest of the system. Maybe that would work here too?
No progress?
bugzilla.gnome.org is being replaced by gitlab.gnome.org. We are closing all old bug reports and feature requests in GNOME Bugzilla which have not seen updates for a long time. If you still use gparted and if you still see this bug / want this feature in a recent and currently supported version, then please feel free to report it at https://gitlab.gnome.org/GNOME/gparted/-/issues/ by following the guidelines at https://wiki.gnome.org/Community/GettingInTouch/BugReportingGuidelines Thank you for creating this report and we are sorry it could not be implemented so far (volunteer workforce and time is limited).
Forwarded to https://gitlab.gnome.org/GNOME/gparted/-/issues/127