GNOME Bugzilla – Bug 499202
gparted does not see the difference if partition size differs from filesystem size
Last modified: 2012-07-13 16:49:30 UTC
It seems gparted takes the partitionsize from the MBR and the free space from the filesystem and throws that up on the table. (This is not really a problem unless you are resizing partitions like I do(resize2fs). I do not honestly know if this happens with "normal" partitions, since I've only experienced it on LVM partitions (/dev/<vg>/<lv>), but I resized my filesystem to shrink the partition(moving everything over to another lvm) and opened gparded, found my partition and it was the size of the partition with the free space of the filesystem. Steps to reproduce(maby... dunno how it will behave with "normal" partitions) 1.Resize filesystem (resize2fs or similar tools) 2.Do NOT resize the partiton 3.Check gparted status Setting this one as minor due to the relative few people dumping into this bug.
Created attachment 111092 [details] created a ext3 partition covering whole sdb Here I have created the ext3 partition that covers the whole of sdb. But it shows 400ish MB is in use? Nothing is in use from what I can see.
Created attachment 111094 [details] Gparted after the FS resize Gparted after the FS resize. It is now telling me that half my disk is full...
I can confirm this bug, and that it also affects "regular" partitions. What i did: I have a 40Gig external usb drive without any partition on it. root@kkt-laptop:~# fdisk -l /dev/sdb Disk /dev/sdb: 40.0 GB, 40007761920 bytes 64 heads, 32 sectors/track, 38154 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Disk identifier: 0xcccdcccd Device Boot Start End Blocks Id System root@kkt-laptop:~# I start Gparted and make a ext3 partition covering the whole device. root@kkt-laptop:~# fdisk -l /dev/sdb Disk /dev/sdb: 40.0 GB, 40007761920 bytes 255 heads, 63 sectors/track, 4864 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0xcccdcccd Device Boot Start End Blocks Id System /dev/sdb1 1 4864 39070048+ 83 Linux root@kkt-laptop:~# Gparted now display this partition as 37.26GB in size but with a Used space of 477.54MB root@kkt-laptop:~# mkdir /tmp/sdb1 root@kkt-laptop:~# mount /dev/sdb1 /tmp/sdb1 root@kkt-laptop:~# cd /tmp/sdb1/ root@kkt-laptop:/tmp/sdb1# ls -lah total 24K drwxr-xr-x 3 root root 4.0K 2008-05-18 13:34 . drwxrwxrwt 15 root root 4.0K 2008-05-18 13:35 .. drwx------ 2 root root 16K 2008-05-18 13:34 lost+found root@kkt-laptop:/tmp/sdb1# ls -lah lost+found/ total 20K drwx------ 2 root root 16K 2008-05-18 13:34 . drwxr-xr-x 3 root root 4.0K 2008-05-18 13:34 .. root@kkt-laptop:/tmp/sdb1# But there is nothing there... Attaching screenshot of gparted. Now comes the part where I get a bit confused. I resize the filesystem and NOT the partition (like you said Kolbjørn). Checking that it actually covers the whole partition. root@kkt-laptop:/tmp# resize2fs /dev/sdb1 resize2fs 1.40.8 (13-Mar-2008) The filesystem is already 9767512 blocks long. Nothing to do! root@kkt-laptop:/tmp# Now, let's resize it root@kkt-laptop:/tmp# resize2fs /dev/sdb1 5000000 resize2fs 1.40.8 (13-Mar-2008) Resizing the filesystem on /dev/sdb1 to 5000000 (4k) blocks. The filesystem on /dev/sdb1 is now 5000000 blocks long. Now gparted tells me wierd things about my partition... Attaching screenshot. So, when I resize the FS, but don't resize the partition, from what I can see gparted thinks that free space withing the partition is used space within the filesystem? I'll check to see what fdisk and parted tells me.
root@kkt-laptop:/tmp# fdisk -l /dev/sdb Disk /dev/sdb: 40.0 GB, 40007761920 bytes 255 heads, 63 sectors/track, 4864 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0xcccdcccd Device Boot Start End Blocks Id System /dev/sdb1 1 4864 39070048+ 83 Linux root@kkt-laptop:/tmp# resize2fs -f /dev/sdb1 5000000 resize2fs 1.40.8 (13-Mar-2008) Resizing the filesystem on /dev/sdb1 to 5000000 (4k) blocks. The filesystem on /dev/sdb1 is now 5000000 blocks long. root@kkt-laptop:/tmp# fdisk -l /dev/sdb Disk /dev/sdb: 40.0 GB, 40007761920 bytes 255 heads, 63 sectors/track, 4864 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0xcccdcccd Device Boot Start End Blocks Id System /dev/sdb1 1 4864 39070048+ 83 Linux root@kkt-laptop:/tmp# root@kkt-laptop:/tmp# parted /dev/sdb print Disk /dev/sdb: 40.0GB Sector size (logical/physical): 512B/512B Partition Table: msdos Number Start End Size Type File system Flags 1 32.3kB 40.0GB 40.0GB primary ext3 Information: Don't forget to update /etc/fstab, if necessary. Somethings not right.
From a bit further thinking, I'll let fdisk off the hook this time around. "fdisk - Partition table manipulator for Linux". Since it is only a partition TABLE manipulator, but parted on the other hand I'll need to look a bit more into what kind of magic it can do with the right love.
*** Bug 349616 has been marked as a duplicate of this bug. ***
*** Bug 612573 has been marked as a duplicate of this bug. ***
*** Bug 639204 has been marked as a duplicate of this bug. ***
How about raising the priority of this bug? This bug also lacks description of what should be done, unlike bug 639203 and bug 349616#c1
s/bug 639203/bug 639204/
(In reply to comment #9) > How about raising the priority of this bug? Raising the priority of this bug will not get this issue resolved any quicker. What would help is if someone could examine this problem and propose a patch to address the issue. > This bug also lacks description of what should be done, unlike bug 639203 and > bug 349616#c1 WORK AROUND: If the file system is smaller than the partition, and GParted supports growing the type of file system (e.g., ext2), then you might try the following: 1) In GParted, select the partition that has the file system which is smaller than the partition. 2) Select the menu entry "Partition --> Check" 3) Apply this operation 4) Check to see if the file system has grown to fill the partition.
(In reply to comment #11) > (In reply to comment #9) > > How about raising the priority of this bug? > > Raising the priority of this bug will not get this issue resolved any quicker. > What would help is if someone could examine this problem and propose a patch to > address the issue. I've examined the problem in bug 639203 and found out that GParted should run the test to detect mismatch between sizes of filesystem and partition during disk scanning stage. If there is a mismatch, it should display a warning icon near partition entry. The same icon is placed near LVM volumes. Upon click on this icon there should be instructions how to fix this problem (your "Partition --> Check" advice). It looks like a good patch proposal. If you're able to point to relevant code inside GParted, somebody may come up with patch implementation. I am unlikely to help you with implementation, unless the patch can be coded in PHP/Python. The parts of the code that are required to solve this bug: 1. size mismatch check - this one is most likely called from Partition --> Check menu 2. disk scanning stage injection point 3. LVM icon placement code as an example If size mismatch check code is already in place then I can evaluate the complexity of this bug (for someone who knows C++) as 'easy'. If you mark this issue as 'normal' (even though I'd mark this as 'major', because GParted gives wrong information about volume size) and 'easy' then any C++ coding GSoC student may include it in his/her plan. > > This bug also lacks description of what should be done, unlike bug 639203 and > > bug 349616#c1 > > WORK AROUND: > > If the file system is smaller than the partition, and GParted supports growing > the type of file system (e.g., ext2), then you might try the following: > > 1) In GParted, select the partition that has the file system which is smaller > than the partition. > > 2) Select the menu entry "Partition --> Check" > > 3) Apply this operation > > 4) Check to see if the file system has grown to fill the partition. That's not a workaround for this particular bug. The subject of this bug is that "gparted does not see the difference", and hence can't notify users. There is no workaround for this issue.
Damn. s/bug 639203/bug 639204/
Created attachment 205490 [details] (GParted showing unhandled space) Hi Curtis, I'm working on a fix so that GParted can display the unallocated space in a partition when a file system doesn't fill it. I have a few questions about the UI, so when you have a few moments can you have a look at this attached screen shot. Top in the screen shot is current GParted behaviour and below are the enhancements so far. The Information dialog display a 3 colour representation of the file system and partition as used (pale yellow), unused (white) and unallocated (grey). Questions: 1) Should this same method be applied to the graphic of the disk in the main windows? Probably yes. 2) Should the same method of unallocated space appearing as grey be applied to the Resize/Move dialog? Probably not. Resizing will resize the file system to fill the partition so there should be no unallocated space afterwards. Does a move perform a resize? Only for overlapping source and destinations or always? 3) Is there any where else in GParted that should used the 3 colour representation of a partition? Thoughts please. Thanks Mike
Thank you Mike for looking into a possible solution for this problem. Your attached picture of how GParted might handle displaying unallocated space within a partition looks good to me. (In reply to comment #14) > Questions: > 1) Should this same method be applied to the graphic of the disk in the > main windows? Probably yes. I would agree, yes. Any unallocated space less than 1 MiB is probably not worth displaying in the user interface. In fact GParted purposefully doesn't display the first 1 MiB of unallocated space at the front of the disk because this is where the MBR and primary GPT are stored. > 2) Should the same method of unallocated space appearing as grey be > applied to the Resize/Move dialog? Probably not. Resizing will > resize the file system to fill the partition so there should be no > unallocated space afterwards. Does a move perform a resize? Only > for overlapping source and destinations or always? For the file systems that support both resize and move, any unallocated space within the partition will be fixed after the operation is applied. Unfortunately not all file systems support resizing. Hence it is possible for GParted to perform a move without a corresponding resize to fill the partition. Since GParted cannot currently fix this situation, there might be some value to showing the unallocated space in the Resize/Move dialog. I don't see this as critical though so if you want to proceed and not show the unallocated space in the Resize/Move dialog then I would be okay with that. > 3) Is there any where else in GParted that should used the 3 colour > representation of a partition? The Partition --> Information dialog and the Copy/Paste dialog should probably also show the unallocated space within a partition.
Hi Mike Without knowing of your work, I have also been investing some time in this issue :-( I guess I should have communicated about it earlier... How far have you progressed / what is your status ? I am wondering if there is a chance that any of the work I did might still be of use, or if I should just forget about it for the moment, wait for you to complete your patch, and see from there ? Kind Regards, Rogier.
Hi Rogier, Good chunk of it has been done. It queries unallocated space for all mounted and unmounted file systems and displays unallocated space in the Info dialog and main window. My code needs tidying up a bit before submission and updating to account for LVM PV and UUID changes we recently added to GParted. One thing I haven't yet decided is how to or whether to display unallocated space in the Paste (Copy and paste into empty space) and Resize/Move dialogs. As soon as a partition is resized in the UI, GParted will perform a file system resize to fill the new partition size so displaying unallocated space becomes wrong. However, if the partition is only moved I think GParted will simply block copy the partition (except for XFS) so unallocated space will remain the same. All the code is in the unallocated-devel branch in my GParted repo on my home machine: https://rockover.homeip.net/cgit/gparted/ Clone with: $ git config --global http.sslverify false $ git clone https://rockover.homeip.net/cgit/gparted/ I'm on holiday next week, but after that I will tidy up the code and submit it for review. Thanks, Mike
Created attachment 209815 [details] [review] Improve reiser4 uuid reading method patch Hi Rogier, While working on this bug I noticed that for reiser4 an fsck is performed to read the UUID. Is there a particular reason for this, or not? Curtis, If not please apply this patch which changes to using debugfs to read the UUID instead. Thanks, Mike
Created attachment 209817 [details] [review] Improve reiser4 uuid reading method patch (v2) D'oh! The patch was wrong. It tried to read the UUID from debugfs's error rather than standard output. In my defence GParted still worked by using the blkid command to read the UUID in its fallback method. Fixed now.
(In reply to comment #18) > While working on this bug I noticed that for reiser4 an fsck is > performed to read the UUID. Is there a particular reason for this, or > not? I would have done that because I found no better alternative. Apparently, I overlooked debugfs.reiser4 for some reason... Obviously, using debugfs.reiser4 is a way better solution. Rogier.
Good catch Mike and thanks for the patch. Also thank you Rogier for responding too. The patch in comment #19 has been committed to the GNOME git repository. The relevant git commit can be viewed at the following link: Improve reiser4 file system uuid reading method http://git.gnome.org/browse/gparted/commit/?id=2919057abd1e5870c5f974c0a3344ff790503723 In an ideal world a new bug report would be created for this reiser4 uuid read patch. Having said that I think it is most important to improve the code so I do appreciate the patch. :-)
Hi Curtis, While testing this change I have found what I think is a separate bug. Copy a partition and paste into an existing partition may, depending on partition alignment, result in the virtual "Copy of ..." partition being smaller that the actual partition on disk it represents when snap to alignment is applied. After the copy operation is performed the partition is displayed correctly again as represented on disk. (Having the size of the partition stored in the Partition object shrink after the object was created was breaking my code to display used/ unused/unallocated space in the partition, but I have now worked around it). Is this a known bug or do I need to raise a new one? Thanks, Mike
Good catch Mike. This problem definitely sounds like a new bug. Pasting into an existing partition should keep the original partition boundaries. Please do create a new bug report so that we do not lose track of the problem.
Created attachment 211076 [details] [review] GParted unallocated space patchset Hi Curtis, Finally ready to submit my patchset for review, after GParted 0.12.1 is released. Few notes: 1) Does not display unallocated space in the Resize/Move and Paste (into unallocated space) dialogs. As mentioned in comment #17, it is not simple. (As soon as a partition is resized in the UI, GParted will perform a file system resize to fill the new partition size so displaying unallocated space becomes wrong. However, if the partition is only moved I think GParted will simply block copy the partition so unallocated space will remain the same). Could represent this to the user by displaying unallocated space in the resize/move dialog when the size matches it's existing size but as soon as it differs display it as unused space instead. Could be confusing to the user and complicated to implement. 2) Is there a need for any changes to the GParted Manual? Is the display and warning completely understandable without any documentation? 3) When pasting into unallocated space or an existing partition the amount of unallocated space is duplicated, but the warning isn't. When the operation is applied and GParted refreshes its display the unallocated space warning is produced as required. 4) As noted in the first commit message, unallocated space and warning aren't displayed until the threshold of 5% is reached to avoid the worst case false positive I found. Newly created XFS file system in a 100MiB partition reports as 4.7% unallocated via statvfs() when mounted. Thanks Mike
Hi Mike. Thank you for this patch. Now that GParted 0.12.1 has been released I tried to apply this patch. Unfortunately patch 007 fails to cleanly apply due to changes in src/Win_GParted.cc. When you have a chance would you be able to rebase this patch against the latest git repository? If not then no worries, I can look into doing this later. When I get a chance to review the patch in detail I plan to respond to your questions. It might be about a month before I get a chance to properly review this patch.
Created attachment 211879 [details] [review] GParted unallocated space patchset (v2) Hi Curtis, Here's an updated patchset which applies cleanly again. Mike
Thank you Mike for this great patch set. :-) When I applied the patches and ran the resulting executable I was impressed with what I saw. Your choice for displaying the unallocated space fits in well with the visual disk area display. While reviewing the patch set I particularly liked how you separated the changes into individual patches, and described the purpose of each patch. This makes it much easier to follow and review than one huge patch. Thank you for going to the extra effort to do this. Following are some answers to your questions, some minor suggestions, and two more questions. Answers ------- A1) Not showing the unallocated space in the Resize/Move and Paste dialog is okay with me. If this becomes an issue in the future then we can look at possible solutions at that time. A2) My opinion is that we do not need to add a description of the unallocated space to the manual. Currently the manual does not contain a description of total file system space, or used space within the file system. I think your warning message regarding unallocated space in the partition describes the situation. A3) I agree with your design choice to not duplicate the warning message for a paste operation in the queue. A4) I believe your choice of a 5% threshold before adding a warning of unallocated space is a reasonable design choice. Suggestions ----------- S1) Change "depreciated" to "deprecated". While reading the patch description and comments, I came across the word "depreciated" several times. My thoughts are that the word you were searching for was "deprecated". Following are some definitions of these words: "Depreciated" - means to lessen the price or value of. "Deprecated" - means in Computer Science to mark (a component of a software standard) as obsolete to warn against its use in the future so that it may be phased out. S2) Minor wording change on translated string. Change: "If supported, Checking the partition will also grow the file system to fill the partition." To: "To try to grow the file system to fill the partition, select the partition and choose the menu item:" +"\n" +"Partition --> Check." In your original text, I like your idea of trying to suggest to the user how to address the problem. My preference is for short clear instructions as these make the translators jobs easier. Regarding the "If supported" part, we could check to see if the grow action is indeed available for the file system in question. This might not be worth the extra complexity though. Perhaps there is an even better wording that we haven't thought of yet? More Questions -------------- MQ1) When a grow action is queued, the visual disk area shows the future partition as the new size. However, the future partition shows the old total FS size, old used FS size, and the remainder as unallocated FS. Should this be adjusted to show what the future total FS size with no unallocated area? MQ2) What is your comfort level with this patch? Are there areas you think might need more testing? Thanks again Mike for this work to enhance GParted. :-)
Created attachment 213943 [details] [review] GParted unallocated warning update Hi Curtis, Thank you for your positive review comments. It seemed like the right way to structure the patches, and to make it as hard as possible for you to say no too :-) Answers A1) - A4). Cool. Suggestions ----------- S1) depreciated -> deprecated. Yes, that's what I meant. Will update. S2) Warning message wording update and check if possible. Attached patch "GParted unallocated warning update" applies on top of "GParted unallocated space patchset (v2)" you just reviewed. It updates the "how to resolve bit" of the warning in line with your suggestion and only includes it when the file system supports the check and grow methods. Seems quite simple to perform the check and not too repetitive compared to the actual execution of the check and grow methods in check_repair_filesystem() and resize_filesystem() respectively. More Questions -------------- MQ2) I had though that the patchset was ready to merge. I had tested everything worked, including for every supportable file system type. MQ1) suggests I missed something though. MQ1) Queued grow operation show increased FS space as unallocated rather than as free. Yes it (almost certainly) should just show the extra space as free rather than unallocated before the grow operation is applied. This may (or may not) have changed in forward porting the patch set. Will re-check the code and update. I just wanted to reply because it will be a few more days before I have the time to address MQ1) fully. At which time I will produce a patch to show the change addressing MQ1) and a new full patchset (v3) with all updated mentioned here included. Thanks, Mike
Thank you Mike for your consideration of the suggestions and your offer to create a new full patchset. I look forward to testing the new patchset when it is available. :-)
Created attachment 214467 [details] [review] GParted unallocated fix resize preview This patch and the previous one from comment #28 are not for applying, they are to show the changes made to address: S2) Warning message wording update and only display message if grow supported, M1) Queued grow operation shows increased FS space as unallocated rather than as free. M1 is quite involved. There are 3 ways and 2 code paths which can shrink, maintain size and grow partitions: * Copy Paste into existing partition (Paste into smaller existing partition is naturally not allowed) Win_GParted::activate_paste() * Copy Paste into empty space creating new partition Dialog_Base_Partition::Get_New_Partition() * Resize/Move partition Dialog_Base_Partition::Get_New_Partition() Also note the comment in the patch about pasting into existing larger partitions when the FS doesn't support growing. It's the only case when the preview does not reflect what will happen after the operations are applied.
Created attachment 214468 [details] [review] GParted unallocated space patchset (v3) Hi Curtis, Here's v3 of the full patchset ready for applying. All discussed changes since v2 are integrated into it. Thanks, Mike
Hi Mike, Your work on this patch is looking better all the time. :-) I was surprised to magically see unallocated space show up in some of my test partitions. Of course, I had forgotten that this is how I had left the partitions the last time I tested these patches. Consequently I can see how important this enhancement would be for a normal user. Without this patch set, most users would be oblivious to the missing space (unallocated) within a partition. While testing and reviewing this patch set, I came across three items of note: N1) Difference in JFS used/unused space results It would appear that you fixed a bug in the calculation of free space for JFS file systems. In GParted 0.12.1 under "Partition --> Information", a newly created 4096 MiB JFS file system reports: Size: 4.00 GiB Used: 16.95 MiB ( 0% ) Unused: 3.98 GiB ( 100 %) With the patch set from comment #31 applied, the used space differs: Size: 4.00 GiB Used: 644.00 KiB ( 0% ) Unused: 3.98 GiB ( 100 %) When I mount the partition, df shows that 644 KiB is free so I'm betting that your calculation of free space is the correct one. Hence I do not think we need to do any work on this discrepancy. N2) Difference in LVM2 PV used/unused space results In GParted 0.12.1 under "Partition --> Information", an LVM2 PV partition of 19,945,472 sectors (512 byte sectors) reports: Size: 9.51 GiB Used: 9.51 GiB ( 100% ) Unused: 0.00 B ( 0 %) With the patch set from comment #31 applied, the used space differs: Size: 9.51 GiB Used: 9.50 GiB ( 100% ) Unused: 0.00 B ( 0 %) "Size" should equal "Used" plus "Unused". 9.51 GiB does not equal 9.50 GiB plus 0.00 B. Perhaps there is a rounding error, or perhaps unallocated space accounts for the rest? Would you be able to look into this further? N3) Question on patch comment In patch numbered 7/8, the patch comment notes the following: NOTE: The file systems: exfat, fat16, fat32, hfs, hfs+, nilfs2 and ufs can't be grown. For nilfs2, I believe the file system can be grown if kernel support is available. Perhaps you meant reiser4 instead of nilfs2? For fat16 and fat32, this would be true if parted-3.0 was used. With parted versions <> 3.0, both fat16 and fat32 file systems can be grown, within their respective maximum sizes of course. Did you encounter a different situation in which these file systems could not be grown? We are very close to ironing out the last few items of note. Then we can commit this patch set to the master repository for inclusion in the next GParted release. Keep up the great work! :-)
Curtis, can you run this command for me so that I can see exactly what GParted knows about your LVM2 setup. lvm pvs --nosuffix --units b -o pv_name,pv_size,pv_free,vg_name,vg_attr,lv_name,lv_attr In case you have more than one PV, please highlight the one giving the result in N2).
Hi Mike, I discovered the discrepancy in my fedora-15 virtual machine. Strangely enough I am no longer able to boot this virtual machine. So to get the following data I booted the virtual machine using gparted-live-0.12.1-5 so that I could access the drives for information. Results as follows: user@debian:~$ sudo lvm pvs --nosuffix --units b -o pv_name,pv_size,pv_free,vg_name,vg_attr,lv_name,lv_attr PV PSize PFree VG Attr LV Attr /dev/sda2 10200547328 0 vg_fedora15 wz--n- lv_swap -wi-a--- /dev/sda2 10200547328 0 vg_fedora15 wz--n- lv_root -wi-a--- Hope that helps. If you need more information then just let me know. I can rebuild the VM from scratch and try the test again if needed. Curtis
Sorry, another bit of information please. I am after the size of the partition too. sfdisk -s /dev/sda2
Certainly. :-) Results from the command as run from GParted Live 0.12.1-5 are as follows: user@debian:~$ sudo sfdisk -s /dev/sda2 9972736 Since I will need my Fedora 15 VM for other testing, I will rebuild this VM and retest.
There's no need to retest. It will produce the same results again. It's caused by even file systems which fill their partitions having some unallocated space and how it is accounted for and presented to the user. I will explain more fully in a bit.
Hi Mike, I retested and our posts crossed. :-) Here is the results anyway.... I have rebuilt my Fedora 15 VM and the (N2) discrepancy listed in comment #32 still shows up. The output from within the VM is as follows: [root@fedora15 fedora]# lvm pvs --nosuffix --units b -o pv_name,pv_size,pv_free,vg_name,vg_attr,lv_name,lv_attr PV PSize PFree VG Attr LV Attr /dev/sda2 10200547328 0 vg_fedora15 wz--n- lv_swap -wi-ao /dev/sda2 10200547328 0 vg_fedora15 wz--n- lv_root -wi-ao [root@fedora15 fedora]# sfdisk -s /dev/sda2 9972736 Hope that helps.
Hi Curtis, To your three items. N1) Difference in JFS used/unused space results I've been through the calculations and the difference is because the newly created JFS had 16.32 MiB of unallocated space. Previously GParted included it within used space. My patchset separates it out. old_used = new_used + new_unallocaed = 644 K + 16.32 M = 16.95 M N2) Difference in LVM2 PV used/unused space results Figures from your initial Fedora 15 VM for sda2, LVM2 PV ... FS size = 10200547328 (bytes) = 19922944 (512 byte sectors) FS free = 0 (bytes) = 0 (512 byte sectors) Partition size = 9972736 (1K blocks) = 19945472 (512 byte sectors) Calculation: size = ptn_size = 19945472 = 9.51 GiB used = fs_size - fs_free = 19922944 - 0 = 19922944 = 9.50 GiB unused = fs_free = 0 = 0.00 B unallocated = ptn_size - fs_size = 19945472 - 19922944 = 22528 = 11.00 MiB This is displayed in the Information dialog as: Size: 9.51 GiB [ptn_size] Used: 9.50 GiB ( 100% ) [used] Unused: 0.00 B ( 0% ) [unused] As unallocated < 5% it is not displayed. With unallocated not displayed the %age calculation is adjusted to include unallocated space within the used %age but the actual figures for used and unused are still displayed. Also the actual used and unused figures are displayed in the main window too. So when there is any unallocated space the used + unused won't add up to the size in the main window either. Possible options: 1) Do nothing. 2) Always show unallocated space in the Information dialog so the sum of space adds up. Still use 5% to trigger the warning and inclusion of unallocated space in the partition usage graphics everywhere. (Keeps the code simple and gives users enough information to to understand what is going on). 3) Display used figure as used + unallocated everywhere when unallocated < 5%. (More complicated code to display used as either used or used + unallocated everywhere depending on %age of unallocated space). 4) Something else. I prefer option 1 or 2, not 3. N3) Patch 7/8 comment about FSs which can't be grown I must admit to being a bit sloppy. I just looked at View --> File System Support in GParted with parted-3.0 to see which FSs could be grown (fat16 and fat32), didn't have a kernel which supported nilfs2 and just missed reiser4. Will update commit message with: NOTE: GParted will attempt to grow the the file system in this situation regardless of whether the file system supports it or not. These file systems: exfat, hfs, hfs+, reiser4 and ufs can't be grown and will fail with "growing is not available for this file system". The preview will also be wrong in this case for the same reason. Everything will be depicted correctely after the refresh. Cases in N1) and N2) show that even file systems which "fill" their partitions still contain small amounts of unallocated space and that it is a little tricky to deal with and present to the user as simply and clearly as possible. The important question, from N2), is how to represent cases when used + unused < ptn_size and unallocated is < 5% outstanding. N4) Would it be useful to have a worked example, using the old and new methods, of calculating the partition usage figures? Either in a file such as the README file, or in the first commit message? Mike
Hi Mike, Thank you for the detailed analysis. I can better see the challenges faced with this enhancement. I appreciate the list of possible solutions. Before I respond to your list of options and (N4), I would like to dig deeper into this challenge. To decide which solution to proceed with, we need to consider what is the purpose of the "used" and "unused" values. Purpose of Used and Unused Values: 1) To display the space usage in the partition to the user 2) To limit shrinking a partition smaller than the "used" value. This second purpose is the one I am most concerned about. The value is used in Dialog_Partition_Resize_Move::Resize_Move_Normal(). More specifically in this line: http://git.gnome.org/browse/gparted/tree/src/Dialog_Partition_Resize_Move.cc#n151 frame_resizer_base ->set_used( Utils::round( selected_partition.sectors_used / ( total_length / 500.00 ) ) ) ; If the "unallocated" space cannot actually be placed into the file system, then we should account for it somehow as being "used" when we display the resize/move dialog. I can think of two possible ways to handle this situation: A) The simple solution for resizing would appear to be "used" = "used" + "unallocated", but this would present an incorrect picture in the case where the "unallocated" space could actually be used, e.g. more than 5%. B) A possible improvement on solution (A) might be to check if there is unallocated space. If unallocated space exists, then add the unallocated space up to some minimal percentage (perhaps 5% since that is our warning threshold) to the used value for the purposes of the resizing dialog. That way if more than 5% unallocated space is available, the extra unallocated space over 5% will still be available to the user when resizing. My preference would be for solution (B) as this would probably help with the following bug report: Bug #673166 - NTFS partition trashed when resized to minimum size Now back to your list of options and (N4). I think that your option (2) to always show the unallocated space might be the best course of action -- keep the user informed. To address (N4), I think that a detailed example might be more than most GParted users need. Instead, I think it might be helpful to place a small note in the Information dialog just under the "unallocated" value. The note would indicate a reason why some unallocated space might be expected. Perhaps something like: "Some unallocated space is normal" or "Due to internal calculations, some unallocated space is normal" or "For some file systems, up to 5% unallocated space is normal" or some other wording that we agree upon. Does this sound to you like a reasonable solution to this challenge?
Hi Curtis, (In reply to comment #40) > If the "unallocated" space cannot actually be placed into the file system, then > we should account for it somehow as being "used" when we display the > resize/move dialog. > > > I can think of two possible ways to handle this situation: > > A) The simple solution for resizing would appear to be "used" = "used" + > "unallocated", but this would present an incorrect picture in the case where > the "unallocated" space could actually be used, e.g. more than 5%. > > B) A possible improvement on solution (A) might be to check if there is > unallocated space. If unallocated space exists, then add the unallocated space > up to some minimal percentage (perhaps 5% since that is our warning threshold) > to the used value for the purposes of the resizing dialog. That way if more > than 5% unallocated space is available, the extra unallocated space over 5% > will still be available to the user when resizing. > > > My preference would be for solution (B) as this would probably help with the > following bug report: > > Bug #673166 - NTFS partition trashed when resized to minimum size Yes I'll go with solution (B) with a few enhancements if that's OK. E1) I suspect that unallocated space in an FS which fills a partition will be bounded by a fixed amount for larger file systems rather than a percentage I used to handle worst case smaller file systems. (I will test for every FS type GParted supports to confirm). When I was only thinking about it being use to trigger a warning it was less important but now I am aware we want to be cautious on the minimum FS shrink size too, I want to see if I can make it better. For example I will make the calculation for max_norm_unallocated (maximum "normal" FS overhead unallocated space) something like minimum of 5% partition size and 20 MiB. Use this new calculation for triggering the warning about unallocated space. E2) Implement something like Partition::estimated_min_fs_size() for use by the FS resize dialogs to make it clear what is happening. It will use the following calculation: return = used + min(unallocated, max_norm_unallocated) E3) Display unallocated space figure in the Information dialog whenever it is greater than zero, rather than when it represents a significant amount. > To address (N4), I think that a detailed example might be more than most > GParted users need. Instead, I think it might be helpful to place a small note > in the Information dialog just under the "unallocated" value. The note would > indicate a reason why some unallocated space might be expected. Perhaps > something like: > "Some unallocated space is normal" > or > "Due to internal calculations, some unallocated space is normal" > or > "For some file systems, up to 5% unallocated space is normal" > or > some other wording that we agree upon. Sorry, I must not have been clear. I was trying to ask if the worked example would be useful for Developers. I suggest including it either in a GIT commit comment or a source file such as the README file. > Does this sound to you like a reasonable solution to this challenge? All good. Probably going to take me a week or so to have the time to complete all the changes. I'll produce a new full patchset when I'm done. Thanks, Mike
(In reply to comment #41) > Sorry, I must not have been clear. I was trying to ask if the worked > example would be useful for Developers. I suggest including it either > in a GIT commit comment or a source file such as the README file. My apologies too. I misinterpreted your intention. I agree that adding a clearly documented worked example calculation would be helpful. Since the audience is developers, the example could be placed in the git comment as well as the HACKING file. Since the README file is intended for everyone, I thought it might be better to use the HACKING file for the calculation example. If you feel strongly that it should be in the README then I could be convinced that way too. > All good. Probably going to take me a week or so to have the time to > complete all the changes. I'll produce a new full patchset when I'm > done. 'Sounds good. No time pressures from this side. :-)
Created attachment 216056 [details] [review] GParted unallocated space patchset (v4) Hi Curtis, Here's v4 of the full patchset. All discussed changes are integrated. Summary of changes since v3: N2/B/E1) Bound estimated of maximum unallocated space -> New patch [PATCH 11/11] Enhance calculation of significant unallocated space N2/B/E2) Include intrinsic unallocated space up to warning limit in minimum partition resize -> New patch [PATCH 10/11] Include intrinsic unallocated space for resizing N2/B/E3) Always display unallocated space Always display unallocated space graphically in main window and Info dialog. Display unallocated space figure if greater than zero in Info dialog. Done so that the user can always see unallocated space and so the figures in the main window and Info dialog are the same rather than hiding small amount of unallocated space as used in the Info dialog. -> Integrated into previous patches [PATCH 02/11] Display unallocated space in the information dialog [PATCH 06/11] Display unallocated space in the main window N3) Patch message 7/8 about FSs which can't be grown -> Updated message as indicated in comment #39 in [PATCH 07/11] Set unallocated space when performing simple operations N4) Worked example space calculation -> Added to HACKING file as part of [PATCH 11/11] Enhance calculation of significant unallocated space Resolve rounding of btrfs file system size leading to GParted thinking FS size is greater than partition size! -> New patch [PATCH 09/11] Handle btrfs tools rounding of figures Plus correction of a few spelling mistakes. Thanks, Mike
Hi Mike, Thank you for this enhanced patch set. Following are two more notes. I have been testing, and I can certainly see the challenge with determining free and unallocated space. For example, even with a 512 MiB ext4 partition, different values are reported from "df" when the partition is mounted, and from dumpe2fs -h" when the partition is unmounted. Specifically, when mounted, the ext4 partition shows some unallocated space (2%) whereas when it is unmounted, there is no unallocated space shown. This difference caused me confusion because I couldn't figure out why one 512 MiB ext4 partition showed no unallocated space, whereas another did. Later on I realized that one partition was not mounted whereas the other was mounted. Since I was confused, I suspect that some GParted users will also get confused. N5) Perhaps we should make the display of used and unallocated space follow the same rules as partition resize_move and copy (i.e., use estimated_min_size() for the used value)? When viewing the partition information for linux-swap and unknown partitions, the area in the graphical display is grey -- the same colour as unallocated. I suspect that this is because we cannot calculate used/unused/unallocated for these types of partitions. N6) Would you be able to make this space show as white, as it currently displays prior to this patchset? You thoughts on these notes are appreciated. Regards, Curtis
Hi Curtis, N5) Use estimated_min_size() for used figure to hide intrinsic unallocated space? Getting the figures wasn't too hard. Having every file system, except for jfs, report different figures for free space and FS size using its own tools and its kernel driver is proving to be a pain in the butt. I do want to make sure that the figures displayed to the user in the main window and Information dialog always agree with each other. The graphics should agree too. I can see that having different figures whether the FS is mounted or not is confusing. Looking through my spreadsheet of used/unused/unallocated figures shows that the tool and the kernel are in close agreement over the free space figure and that the difference in used is a close match for intrinsic unallocated space. Your idea to display used as estimated_min_size() is the right call. I'll see what I can do code wise. N6) Show unknown usage as white in the Information dialog? Yes, definitely will correct. Partitions with unknown usage are shown as white in the main window and it should match in the Information dialog. Mike
Created attachment 216545 [details] [review] GParted unallocated space patchset (v5) Here's v5 of the full patchset. All discussed changes are integrated, plus a few more. Summary of changes since v4: N5) -> New patch [12/12] Don't show intrinsic unallocated space N6) -> Incorporated into patch [02/12] Display unallocated space in the information dialog Also added const object qualifiers to some of the new methods I have added throughout the patchset and removed unnecessary "this->". Mike
Awesome work on this patchset Mike! My reviewing and testing has gone extremely well. I have tried resizing most all types of file systems and have compared the prior version results to this patchset and all looks good. If you are in agreement, then I will commit this patchset to the master branch. Can you think of any reason not to commit this patchset? Curtis
Thank you Curtis for your ongoing reviews. The code is much better for it. Yes, please go ahead and commit the patchset. I can't see any reason not to commit, especially after addition of the last patch which only shows unallocated space when it becomes significant so GParted will report identical figures to previously until then. One very small thing I missed. Can you fix-up a small typo in the commit message to the last patch "Don't show intrinsic unallocated space". First paragraph is missing a "system" and probably should be "the" instead of "a" giving: ... ext2/3/4, fat16/32, hfs, nilfs2, reiserfs and xfs. Showing either a little or no unallocated space, depending on whether the file system is mounted or not, could be confusing to the user. Thanks, Mike
Thanks to much work by Mike Fleetwood, the problem with GParted not seeing the difference when partition size differs from file system size should now be resolved. The twelve enhancements to address the problem have been committed to the git repository for inclusion in the next release of GParted. The relevant git commits can be viewed at the following links: 1) Record unallocated space within a partition (#499202) http://git.gnome.org/browse/gparted/commit/?id=8093ba2ebd53b1e4ce869d5bf7cb7798238f28d6 2) Display unallocated space in the information dialog (#499202) http://git.gnome.org/browse/gparted/commit/?id=8ffd68b0121f047f49a469cdf411fcf0d4903882 3) Query unallocated space for unmounted file systems (#499202) http://git.gnome.org/browse/gparted/commit/?id=719e73e3351b01bc7b0cf968b777dd4aeac027d8 4) Query unallocated space for LVM2 PVs (#499202) http://git.gnome.org/browse/gparted/commit/?id=e6290dbbcf0a5cf4557b014ed7cbaa2cea027710 5) Query unallocated space via libparted (#499202) http://git.gnome.org/browse/gparted/commit/?id=30385cbd3750e6a221f498e9ab3e74b9957b9671 6) Display unallocated space in the main window (#499202) http://git.gnome.org/browse/gparted/commit/?id=2b9c6425b535973533927a1abbbee53a2becf174 7) Set unallocated space when performing simple operations (#499202) http://git.gnome.org/browse/gparted/commit/?id=04f5250644ecd47536910b5c3bd4bc6aa7b08d6e 8) Set unallocated space for paste or resize/move operations (#499202) http://git.gnome.org/browse/gparted/commit/?id=7ddbc9bd9281bab38ecb5e451f3a9e474a9df0d9 9) Handle btrfs tools rounding of figures (#499202) http://git.gnome.org/browse/gparted/commit/?id=7fc16a1b6905f7f2d6876d317c4757ff09cdd57c 10) Include intrinsic unallocated space for resizing purposes (#499202) http://git.gnome.org/browse/gparted/commit/?id=3737224028cf0de93e5f278c1d1be05e4f137afa 11) Enhance calculation of significant unallocated space (#499202) http://git.gnome.org/browse/gparted/commit/?id=b5c80f18a9ce27fda2b11e9b590d7f856209ac32 12) Don't show intrinsic unallocated space (#499202) http://git.gnome.org/browse/gparted/commit/?id=7ebedc4bb3b81e85fb4c628a2a05308ada147d68 A big thanks goes to Mike Fleetwood for tackling and solving this problem.
Created attachment 216972 [details] [review] GParted unallocated space fix1 patchset Hi Curtis, I have found a minor display issue. Went a bit OTT and ended up creating a small cleanup patch set ;-) It started as fixing ... For specific partition usage values the right hand border of the partition graphic in the Information dialog would be displayed as grey rather than the color assigned to the partition. Steps to reproduce fault: Create 1024 MiB partition # lvm pvcreate /dev/sda12 # lvm vgcreate GParted-VG1 /dev/sda12 View partition information Mike
Don't apply this. I have realised that there are cases in get_usage_triple() where the results aren't the "best". One of many found by my search program: (35, 35, 34) <- get_usage_triple(451585024, 451585536, 451586048, 104) The sector usage figures are strictly s1 < s2 < s3 but there is only 512 bytes difference between each of them. However the output pixel numbers aren't ordered i1 <= i2 <= i3 because of i1 and i2 rounding upwards. I doubt a user is ever going to have used, unused and unallocated sectors very close like this but I will fix get_usage_triple() so that it orders the magnitude of the output pixels as expected. (Might be next week before I send patchset v2 as I am meant to be packing for a weekend away rock climbing). Mike
Hi Mike, Thanks for the heads up on the this new patch (comment #50). I was planning on testing the patch today. I will hold off until you have a chance to address the situation you discovered. Have fun rock climbing! Curtis
Created attachment 217205 [details] [review] GParted unallocated space fix1 patchset (v2) Hi Curtis, Here's fix1 patchset v2. Fixes get_usage_triple() issue mentioned in comment #51. Mike
Good work again Mike. Using your test case I was able to see the problem first-hand, and also confirm that the patches fixed the display problem. The patch set in comment #53 has been committed to the git repository for inclusion in the next release of GParted. The relevant git commits can be viewed at the following links: Fix minor unallocated space display issue in the Info dialog (#499202) http://git.gnome.org/browse/gparted/commit/?id=67f334a8ac90efab662bf91b3ada49c0010c6dd3 Simplify main window partition usage graphic code http://git.gnome.org/browse/gparted/commit/?id=ac3ce5ec2bf5821734de411dc4f8b0c948902a1f Simplify calc_usage_triple() interface and rename http://git.gnome.org/browse/gparted/commit/?id=6c96ab34b39eb23a906d6161a3d3b873025f4ec3 Remove and rename local variables in Display_Info() http://git.gnome.org/browse/gparted/commit/?id=d9e314f82f25f137a93ffc48fb74dfe7fd921356
Created attachment 217730 [details] [review] GParted unallocated space fix2 Hi Curtis, Sorry about this but after you published fix1 patchset v2 I immediately saw a small change I had not intended to include. I have been biting my metaphorical lip all week because it doesn't make any difference (unless a cosmic ray changes sectors_unallocated to a negative value) but I can't deny it any longer, so here's a small correction. Mike
No worries about this Mike. :-) The patch in comment #55 has been committed to the git repository for inclusion in the next release of GParted. The relevant git commit can be viewed at the following link: Restore original logic regarding negative values in get_usage_triple() http://git.gnome.org/browse/gparted/commit/?id=367945bbb2f413d290e55b4a223a117d7003d864 I would like to make a new release of GParted soon, but want to get a partition resize overlap problem fixed, and investigate another potential problem first.
Created attachment 218262 [details] [review] Make btrfs_size_*() static member functions Hi Curtis, I have found another small correction for this patchset, after finally just fully understanding the difference between member function / member function with const qualifier / static member function. Still learning this new fangled C++ language ;-) Thanks Mike
Thanks for the update Mike. I certainly don't claim to be an expert in C++ either. There is always more to learn. :-) In general I try to ensure that GParted is stable and reliable, and if we succeed with that then I think we are doing well. The patch in comment #57 has been committed to the git repository. The relevant git commit can be viewed at the following link: Make btrfs_size_*() static member functions http://git.gnome.org/browse/gparted/commit/?id=a6ff181fafafcb34612f44191880829ab57a7bf5 This patch will be included in GParted 0.13.0 which is planned for release on Friday, July 13, 2012.
The enhancements to address this bug report have been included in GParted 0.13.0 released on July 13, 2012.