Hi Patrick,

Thank you for raising this.


I've not yet built e2fsprogs-1.43 and tried this, but I've researched
the code.  This commit changes the defaults for mke2fs from only adding
the 64bit feature when the block device is > 16T and needs it, to always
adding the 64bit feature.

http://git.kernel.org/cgit/fs/ext2/e2fsprogs.git/commit/?id=cd27af3ecb83e8fd1e3eaa14994284a1818c7c15
mke2fs: enable the metadata_csum and 64bit features by default

Unfortunately there is nothing in the release notes about this being
included in v1.43.
http://e2fsprogs.sourceforge.net/e2fsprogs-release.html


Have you tested trying to boot from a EXT4 with 64bit feature?  Did it
fail?

Assuming it does fail my initial reaction is that GParted Live CD and
your Parted Magic should just revert that change to the /etc/mke2fs.conf
file restoring the previous default, which is what I assume the
Slackware patch you mentioned would be doing.

This does mean that if the GParted distro app is used on a distro which
doesn't revert /etc/mke2fs.conf GParted will still allow mke2fs 1.43
to create EXT4 file systems with the 64bit feature.  I would prefer that
we just make GParted app create EXT4 how they use to be done, only
enabling the 64bit feature on partitions > 16T, rather than adding a
tick box for the user to get wrong.


Mike

Apparently I can't read.  The e2fsprogs release notes for 1.43 say this:

  Mke2fs will now create file systems with the metadata_csum and 64bit
  features enabled by default.

Hi Patrick and Mike,

Currently the GParted UI does not support file system specific options.  Moreover I anticipate that changing the GUI to do this is a non-trivial effort.

(In reply to Mike Fleetwood from comment #1)
> ...snip....  I would prefer that
> we just make GParted app create EXT4 how they use to be done, only
> enabling the 64bit feature on partitions > 16T, rather than adding a
> tick box for the user to get wrong.

This suggestion to address the issue by only enabling 64bit ext4 for partitions greater than 16TB sounds like a good compromise to me.

Curtis

Update bug summary to better describe the issue not one possible fix.
Old summary:
    Add 64Bit checkbox when creating EXT4 file systems
New summary:
    Multiple boot loaders don't work on 64bit EXT4 file systems


Further details and references (fragments from the commit message):

E2fsprogs version 1.43 always creates 64bit ext4 file systems by default
[1][2] regardless of the partition size.  Previously it only enabled the
64bit feature when required on ext4 volumes 16 TiB and larger.  Also
note that RHEL / CentOS 7 always create 64bit ext4 file systems by
default from e2fsprogs 1.42.9 [3].

(At least some versions of) Grub 2 and syslinux boot loaders don't work
with 64bit ext4 file systems [4][5][6].  For maximum boot loader
compatibility make GParted implement what mke2fs previously did, only
setting the 64bit feature on volumes 16 TiB and larger and clearing it
otherwise.

[1] Release notes, E2fsprogs 1.43 (May 17, 2016)
    http://e2fsprogs.sourceforge.net/e2fsprogs-release.html#1.43

    "Mke2fs will now create file systems with the metadata_csum and
    64bit features enabled by default".

[2]
http://git.kernel.org/cgit/fs/ext2/e2fsprogs.git/commit/?id=cd27af3ecb83e8fd1e3eaa14994284a1818c7c15
    mke2fs: enable the metadata_csum and 64bit features by default

[3] Comment 20 and 21 in Red Hat bug 1099237
    https://bugzilla.redhat.com/show_bug.cgi?id=1099237#c20

    "..., is rhel7 default behavior w.r.t. 64 bit really different from
    upstream ?

    Yes it is. This is what we have in RHEL7:
    Patch6: e2fsprogs-1.42.9-enable-64bit-feature-by-default.patch
    it fixed this bz: https://bugzilla.redhat.com/show_bug.cgi?id=982871
    and this is upstream proposal:
    http://www.spinics.net/lists/linux-ext4/msg42294.html"

[4] Grub 2 not working on Slackware with 64bit EXT4
http://www.linuxquestions.org/questions/slackware-14/grub-mkconfig-error-in-slackware-current-64-bit-4175580544/

[5] Syslinux not working on Slackware with 64bit EXT4
http://www.linuxquestions.org/questions/slackware-14/slackware64-current-5-20-2016-syslinux-booting-and-ext4-formatting-4175580324/

[6] Syslinux not working on RHEL 7 with 64bit EXT4
    Bug 1099237 - rhel7 ext4 defaults to 64 bit, which extlinux can't
                  reliably read
    https://bugzilla.redhat.com/show_bug.cgi?id=1099237

Created attachment 328728 [details] [review]
Only enable ext4 64bit feature when required (v1)

Hi Curtis,

Here's patchset v1 for this.

P1: Limit maximum ext2/3/4 volume size to just less than 16 TiB (#766910)
P2: Don't limit ext4 volume size when 64bit feature is available (#766910)
P3: Only enable ext4 64bit feature when required (#766910)

The first two patches are setting maximum volume sizes for ext2/3/4.
The third patch resurects the pre-1.43 e2fsprogs behaviour implemented
by auto_64-bit_support; sets 64bit feature for volumes >= 16 TiB and
clears 64bit feature for smaller volumes.  It does this by passing the
relevant flags on the command line to mkfs.ext4.

I keep wondering if this is something that should be decided at the
distro level as it is effectively overriding what the e2fsprogs package
or the distro have set in the /etc/mke2fs.conf file.  So GParted Live CD
could override it, rather than the GParted app.  The issue is that an
ext4 file system created smaller than 16 TiB, without 64bit feature,
can't be resized larger that 16 TiB.  Given that people are discovering
that boot loaders don't handle 64bit ext4, I expect most distros will
revert the change in mke2fs.conf in the short term.  Long term I expect
Grub 2 and Syslinux to be fixed to work with 64bit ext4.  Then GParted
app is left enforcing what might not be the distros chosen features for
created ext4 file systems.  Still probably need to do this in the app
though.  Better to create file systems which can be booted from, rather
than not be resized over 16 TiB, without first converting to 64bit.

Thanks,
Mike

Hi Mike,

Thank you for the patch set.  I will review and test it soon.

I agree that it is better to create file systems which can be booted
from, but cannot be resized over 16 TiB without first converting to
64bit.  I also agree that for the near term it is best to do this
within the GParted application, rather than relying on all distros to
revert the default 64bit ext4 behaviour of es2fsprogs 1.43.

My thoughts are that following approval of this patch set that we
should consider a 0.26.1 GParted release.

There is another reported problem [1][2] related to our current
GParted Live 0.26.0-2 release due to the use of e2fsprogs 1.43-WIP
that needs to be addressed by an updated GParted Live release.

[1]: Bug 766693 - Can't resize the "root" or "/" partition from Ubuntu
     https://bugzilla.gnome.org/show_bug.cgi?id=766693

[2]: Forum Post: Can't shrink partition - ERROR 238277632K 00:00:05
     http://gparted-forum.surf4.info/viewtopic.php?id=17558

Curtis

Hi Mike,

How much free space did you have to create a sparse 16T file?

Currently the largest free space I have is ~250GB and it appears to be insufficient.

$ df /vmimage
Filesystem     1K-blocks      Used Available Use% Mounted on
/dev/sdc12     619142920 335991460 251677796  58% /vmimage
$ sudo truncate -s 16T /vmimage/sparse.img
truncate: failed to truncate '/vmimage/sparse.img' at 17592186044416 bytes: File too large

The code in patch set v1 looks like it should work, but at the moment I am unable to test the boundary 16T case.

How comfortable are you with the testing you did?

Curtis

Hi Curtis,

The error is not trying to say you don't have enough free space, it is
trying to say the file system can't handle a file offset (size) of 16T.
I guess that the file system in sdc12 is ext3 or ext4.

Use an XFS file system to host the sparse file.

Ext4 is quite a sparsely written file system when initially formatted
so will actually write less than 2 GiB to format at 16 TiB size.

(It's easy for me to say after I had the same error).

Mike

Hi Mike,

Thanks for the xfs tip.  It didn't occur to me that the problem might be the 32-bit limited ext4 file system I was using.  {face-palm}

While testing I discovered I was unable to create maximum size ext2 and ext3 file systems using the technically correct "fs.MAX = 16 * TEBIBYTE - 4 * KIBIBYTE;" because GParted's smallest unit is 1 MiB, so this is 16,777,216 MiB in the GUI (the - 4 KiB is lost).

I was able to create 16,777,215 MiB partitions with ext2/3 file systems.

To prevent this issue, I think we need to change from the technically correct:

   fs.MAX = 16 * TEBIBYTE - 4 * KIBIBYTE;

to the GParted GUI correct:

   fs.MAX = 16 * TEBIBYTE - 1 * MEBIBYTE;

What do you think?

Curtis

(In reply to Curtis Gedak from comment #9)
> While testing I discovered I was unable to create maximum size ext2 and ext3
> file systems using the technically correct "fs.MAX = 16 * TEBIBYTE - 4 *
> KIBIBYTE;" because GParted's smallest unit is 1 MiB, so this is 16,777,216
> MiB in the GUI (the - 4 KiB is lost).

What if you format a whole disk file system.  No partition to be created
using create dialog with "New size" entry working in MiBs.

I think formatting a whole disk file system is a secondary feature of GParted.  First and foremost it is a partitioning tool.  I think the loss of ~1 MiB of space is more than acceptable on a 16 TiB disk.

Furter, a person can still use GParted to format an ext4 file system that is larger than 16 TiB - 1 MiB using the 64 bit feature.

> What if you format a whole disk file system.  No partition to be created
> using create dialog with "New size" entry working in MiBs.

Thinking more on this, I don't think GParted can alter what occurs when a person tries to format a whole disk file system with a file system type (ext2/ext3) that does not support the large size of the disk (16 TiB in this case).  Such an operation should fail both on the command line and within GParted.

With regards to a partition, if GParted uses a maximum of 16 TiB - 1 MiB, then the partition creation and format with ext2/3 will succeed.

If we wish to maintain the technically correct max size, and implement a floor function to set an integer MiB value, one might consider the following:

  # Round down to an even MiB multiple of maximum 32bit file system size
  fs.MAX = Utils::floor_size( 16 * TEBIBYTE - 4 * KIBIBYTE, MEBIBYTE );

In theory the above line should yield a value of 16 TiB - 1 MiB or
16,777,215 MiB, which should succeed when used creation of an ext2/3 partition.

Curtis

The dialogs (Copy, New, Resize/Move) should take any minimum and maximum
sizes and work correctly, not requiring them to be pre-rounded to whole
MiBs.

I looked at all the FS .MIN and .MAX sizes.  Only nilfs2, apart from
this case, has a setting which is not already an exact multiple of
1 MiB.  Nilfs2 minimum size is 128M+4K.  That is rounded up ot 129M when
creating a new partition.  It's done by those calculations in:
    Dialog_Partition_Copy::optionmenu_changed()
    Dialog_Partition_New::set_data()
    Dialog_Partition_Resize_Move::Resize_Move_Normal()
    Dialog_Partition_Resize_Move::Resize_Move_Extended()
leading up to:
    spinbutton_size.set_range( ceil(fs.MIN/1M), ceil(fs.MAX/1M) )

Because .MIN and .MAX are ceil()inged all over the place that is
rounding up for the nilfs2 min size.  For this ext2/3/4 it should use
floor() to round down.  However the calculations around with
MIN_SPACE_BEFORE_MB, TOTAL_MB, etc. look like they are tied together
re-computing ceil() of min and max sizes all over the place.
Complicated.

I am very tempted to do use your suggestion and mark is as a FIXME for
later.

Mike

Yes, there is much alignment and min/max logic mixed in with the GUI resize bar calcs.  Ideally much of the logic might be separated from the GUI, but this is a likely a significant task in itself.

When you mentioned this line:

  spinbutton_size.set_range( ceil(fs.MIN/1M), ceil(fs.MAX/1M) )

it made me think that the fs.MAX should use a floor() function because the ceil() function could possibly make the fs.MAX larger than the maximum that can be handled.  As you mention the ceil() function is used in many places, and likely incorrectly associated with fs.MAX.

I think sorting out this complexity and confusion might be better handled in a separate enhancement.

Curtis

Upon thinking more about the logic, I remembered the case where a user is moving a non-MiB aligned partition containing a file system for which we do not have resize ability.  In this case it is important that the containing partition is not smaller than the existing file system.  This case calls for a ceil() function versus a floor() function.

The existing code has undergone real-world testing that has occurred over a number of years.  Ideally many fringe cases would be identified and we would have test cases for these.

I think any changes to this GUI code should ideally undergo testing for as many fringe cases as we can think of regarding combinations of alignment, create/resize/move, min/max, and availability of grow/shrink tools.

Created attachment 329099 [details] [review]
Only enable ext4 64bit feature when required (v2)

Hi Curtis,

Here's patchset v2.  Changes from v1 are use of floor_size() to limit
the maximum size of ext2/3 and ext4 without 64bit feature to 16T-1M,
along with an associated FIXME.

Thanks,
Mike

Hi Mike,

Thank you for working through this issue to come up with a workable compromise.

My testing of patch set v2 from comment #16 has gone well.  As such I have committed the changes to the git repository for inclusion in the next release of GParted (0.26.1).

The relevant git commits can be viewed at the following links:

Limit maximum ext2/3/4 volume size to just less than 16 TiB (#766910)
https://git.gnome.org/browse/gparted/commit/?id=af17808d08c762b1c3b6093211a91290df2c993e

Don't limit ext4 volume size when 64bit feature is available (#766910)
https://git.gnome.org/browse/gparted/commit/?id=a13526bf4102bd18019cab83abf0c3b83b9fb232

Only enable ext4 64bit feature when required (#766910)
https://git.gnome.org/browse/gparted/commit/?id=cc7e412bc6ec2e27d049438d2a5d4cf2548cbaae

Curtis

This enhancement was included in the GParted 0.26.1 release on June 13, 2016.