After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 701075 - Setting the backup boot sector when resizing NTFS
Setting the backup boot sector when resizing NTFS
Status: RESOLVED FIXED
Product: gparted
Classification: Other
Component: application
unspecified
Other Linux
: Normal minor
: ---
Assigned To: gparted maintainers alias
gparted maintainers alias
Depends on:
Blocks:
 
 
Reported: 2013-05-27 08:56 UTC by Jean-Pierre André
Modified: 2013-09-18 16:51 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Patches to ntfsresize for setting the backup boot sector (2.02 KB, application/x-compressed-tar)
2013-05-27 15:18 UTC, Jean-Pierre André
  Details
Shrink file systems to exact size (v1) (4.75 KB, patch)
2013-05-27 19:44 UTC, Mike Fleetwood
none Details | Review

Description Jean-Pierre André 2013-05-27 08:56:26 UTC
The last sector of an ntfs partition is usually reserved for backing up the normal boot sector. When shrinking an existing ntfs partition, ntfsresize does not set the backup boot sector because it does not know where the last sector of the partition will be (because of approximations in defining the size and alignment constraints on the partition).

gparted has the knowledge of the future partition size when it starts ntfsresize, so ntfsresize could set the backup boot sector if it were given the exact partition size. However, apparently, the given size is the partition size minus one byte, which denies ntfsresize to write the full last sector.

What is the rationale for not giving the full partition size ?

(Original request on  http://tuxera.com/forum/viewtopic.php?f=2&t=30157&sid=5c69c294ac8e8a9c8f7936e3bc3ba157)
Comment 1 Mike Fleetwood 2013-05-27 14:26:28 UTC
Hi Jean-Pierre,

I've had a look back at the code and it appears to be an historical
accident that GParted specifies the new partition size minus one byte
when shrinking ntfs (ext2/3 and reiserfs too).  I'll fix GParted.

   GParted use to resize file systems 1 cylinder to small then grow it
   to fill the partition [1].  When this was removed it continued to set
   the file system 1 byte too small [2].
[1] correct partitiontype of destination is now set while copying resizing now
https://git.gnome.org/browse/gparted/commit/?id=52dfb1c3647f2b545e865d3487fd479346b3aa74
[2] removed cylindersize buffering during resize from the filesystems
https://git.gnome.org/browse/gparted/commit/?id=d663c3c277f60da2ac1785c039f25f7920a315e9


After reading the original forum post I have a few questions:

1) When shrinking an ntfs should GParted be specifying the new partition
   size or the new partition size minus 1 sector to allow for the backup
   boot sector?
   If the sector size is > 512 (such as 4K) should it be ptn size minus
   512 bytes or 1 sector?

2) Currently GParted follows this process to shrink an ntfs:
   a) Check FS:
         ntfsresize -P -i -f -v /dev/PTN
   b) Shrink FS:
         ntfsresize -P --force --force /dev/PTN -s SIZE --no-action && \
         ntfsresize -P --force --force /dev/PTN -s SIZE
   c) Shrink partition using libparted

   Should it be doing anything differently?
   Can it call ntfsfix to create the backup boot sector?
   If GParted does create the backup boot sector should we still recommend
   users boot into windows twice?
      http://www.gparted.org/faq.php#faq-7

Thanks,
Mike
Comment 2 Jean-Pierre André 2013-05-27 15:14:28 UTC
> I've had a look back at the code and it appears to be an historical
> accident that GParted specifies the new partition size minus one byte
> when shrinking ntfs (ext2/3 and reiserfs too).  I'll fix GParted.

Fine.

> When shrinking an ntfs should GParted be specifying the new partition
> size or the new partition size minus 1 sector to allow for the backup
> boot sector?

It should specify the partition size. The manual for ntfsresize is unclear about the meaning of the size argument, and I attach a patch to improve that.
Also, currently ntfsresize reserves a full cluster for the backup boot sector. This is too much, (patch available) but the important aspect is that changes in gparted and ntfsresize do not depend on each other, though a correct operation requires both to be fixed.

> If the sector size is > 512 (such as 4K) should it be ptn size minus
> 512 bytes or 1 sector?

It is always a full sector. When the sector size is greater than 512 byte, more boot code (for a bootable partition) is copied to the backup boot sector.

> Should it be doing anything differently?

AKAIK the only useful change is to specify the (target) partition size. Best as a number of bytes which is a multiple of the sector size. When expanding a partition, the partition has to be expanded first, and ntfsresize does not need to be given the new size.

> Can it call ntfsfix to create the backup boot sector?

That is another possibility : ntfsfix can fix the backup boot sector after the partition has been resized (available in recent versions).

> If GParted does create the backup boot sector should we still recommend
> users boot into windows twice?

I would keep this recommendation, though both gparted and ntfsresize are now mature. This, at least, sets a divide between problems caused by resizing and those which may result from subsequent mounts.
Comment 3 Jean-Pierre André 2013-05-27 15:18:02 UTC
Created attachment 245392 [details]
Patches to ntfsresize for setting the backup boot sector

Proposed patches may be useful for testing gparted
Comment 4 Mike Fleetwood 2013-05-27 19:44:40 UTC
Created attachment 245408 [details] [review]
Shrink file systems to exact size (v1)

Hi Curtis,

Here's the patch set to fix this fault:
    Shrink file systems to exact size (#701075)
    Place options before device name for the ntfsresize command

I tested ext2, ntfs and resierfs like this.  Setup:
S1) Create 100M file system
S2) Create second file system larger and shrink to 100M
    T1) Look at the sized passed during the shrink
Further tests:
T2) Use FS specific commands GParted uses to show their sizes.
    # tune2fs -l /dev/PTN | \
      egrep 'Block count|Block size|Filesystem volume name'
    # ntfsresize -P -i -f -f /dev/PTN | \
      egrep 'Cluster size|Current volume size|Current device size'
    # debugreiserfs /dev/PTN 2> /dev/null | \
      egrep 'Blocksize|Count of blocks on the device|LABEL'
    (S2) is 1 block smaller than (S1).
T3) Perform check operation in GParted on both.
    (S1) reports nothing to do for the resize.
    (S2) reports increasing the size of the file system.

Thanks,
Mike
Comment 5 Curtis Gedak 2013-05-27 20:18:40 UTC
Thank you Jean-Pierre for raising this problem and working with Mike to resolve the issue.

I was a contributor to this problem since I thought GParted was performing an unnecessary second resize (maximize) of the file system and removed this functionality in GParted 0.11.0.  See:

  Bug #663980 - Avoid redundant file system maximize actions on copy,
                move, and resize

The patch in comment #4 should be an improvement because it avoids the redundant resize (maximize) and should also set the NTFS backup boot sector properly now that the partition size is provided.


Mike, I will review and test this patch shortly.
Comment 6 Mike Fleetwood 2013-05-27 21:04:55 UTC
(In reply to comment #5)
> I was a contributor to this problem since I thought GParted was performing an
> unnecessary second resize (maximize) of the file system and removed this
> functionality in GParted 0.11.0.  See:
> 
>   Bug #663980 - Avoid redundant file system maximize actions on copy,
>                 move, and resize

 Curtis,

I think you were merely doing you bit to remove some old crud from the
code base :-)

It hapened to reveal the fact that ext[234], ntfs and resizerfs were
resized 1K or 1 byte too small.  This being a left over from code
removed before your tenure.  Quoting my first commit message:

    This looks like an accidental leftover from when GParted use to:
    1) Shrink the file system 1 cylinder too small
    2) Shrink the partition
    3) Grow the file system to fill the partition
    Relevant commit:
   
        d663c3c277f60da2ac1785c039f25f7920a315e9
        removed cylindersize buffering during resize from the filesystems


> ... and should also set the NTFS backup boot sector
> properly now that the partition size is provided.

As I understand it, this bit requires ntfsresize to be updated with the
patches Jean-Pierre posted in comment #2 so that when it is given a new
size specified in bytes only (not K,M,G,Ki,Mi,...) ntfsresize will
backup the boot sector.  Ntfsresize already backs up the boot sector
when it is maximised.  My patch mearly passes the correct new partition
size to ntfsresize.

Thanks,
Mike
Comment 7 Jean-Pierre André 2013-05-28 04:44:55 UTC
> As I understand it, this bit requires ntfsresize to be updated with the
> patches Jean-Pierre posted

Not a requirement, the patches to gparted and ntfsresize need not be done synchronously.

> Ntfsresize already backs up the boot sector when it is maximised.

I do not think so, though when partition is expanded downwards the backup boot sector has to be used to locate the metadata, and both boot sectors are then updated.

> My patch mearly passes the correct new partition size to ntfsresize.

Thank you for doing so.
Comment 8 Curtis Gedak 2013-05-28 21:28:39 UTC
Hi Mike and Jean-Pierre,

Using Mike's steps in comment #4 and MiB alignment, I can confirm
Mike's results -- the values are the same for creation and resize.

However, when using cylinder alignment, there are differences between
the creation and resized values.

Following are my cylinder testing notes, and the results including hypothesis
on the value differences.


CYLINDER TESTING NOTES
======================
For my testbed, I used Kubuntu 12.04 with up-to-date patches.

Since cylinder alignment uses the first track (sector 63) for the
first partition, I created a 10 GiB MiB aligned first partition.  This
removes the problem with the first track discrepancy.  Then I created
cylinder aligned partitions for the second and third primary
partitions.

I used the steps in comment #4, but when resizing I chose a value of
94 MiB so that this would cylinder align to the same size as the
created partition (end result 94.13 MiB).

Following are my test results and hypothesis for ext3, ntfs, and
reiserfs file systems.


TEST RESULTS AND HYPOTHESIS
===========================

PARTITION TEST BOUNDARIES - CYLINDER ALIGNMENT
-----------------------------------------------

Following is the disk partition layout.  The same partition boundaries
were used for all tests.

$ sudo fdisk -l -u /dev/sdd

Disk /dev/sdd: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders, total 312581808 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0005c432

   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1            2048    20973567    10485760    7  HPFS/NTFS/exFAT
/dev/sdd2        20980890    21173669       96390    7  HPFS/NTFS/exFAT
/dev/sdd3        21173670    21366449       96390    7  HPFS/NTFS/exFAT


EXT3 TEST - CYLINDER ALIGNMENT
------------------------------

$ sudo tune2fs -l /dev/sdd2 | \
  egrep 'Block count|Block size|Filesystem volume name'
Filesystem volume name:   ext3-test
Block count:              96388
Block size:               1024
$ sudo tune2fs -l /dev/sdd3 | \
  egrep 'Block count|Block size|Filesystem volume name'
Filesystem volume name:   ext3-test2
Block count:              96390
Block size:               1024
$

Actual resize command from GParted details log:
  resize2fs /dev/sdd3 96390K


EXT3 Differences
----------------

The resized block value (96390) was 2 blocks larger than the
originally created partition (96388).  With shrinking, I would have
expected a value the same or smaller than the originally created
partition.  Interestingly, using the old code which subtracted 1 kiB,
the resized block value (92689) was still 1 block larger than the
originally created partition.

Q1)  Perhaps we should subtract 2 blocks (2 x 1024 byte blocks) when
     resizing ext2/3/4 using cylinder alignment?


NTFS TEST - CYLINDER ALIGNMENT
------------------------------

$ sudo ntfsresize -P -i -f -f /dev/sdd2 | \
  egrep 'Cluster size|Current volume size|Current device size'
Cluster size       : 4096 bytes
Current volume size: 98701824 bytes (99 MB)
Current device size: 98703360 bytes (99 MB)
$ sudo ntfsresize -P -i -f -f /dev/sdd3 | \
  egrep 'Cluster size|Current volume size|Current device size'
Cluster size       : 4096 bytes
Current volume size: 98697728 bytes (99 MB)
Current device size: 98703360 bytes (99 MB)
$

Actual resize command from GParted details log:
  ntfsresize -P --force --force -s 98703360 /dev/sdd3

Ntfsresize version:
  ntfsresize v2012.1.15QR.1 (libntfs-3g) on kubuntu 12.04 LTS


NTFS DIFFERENCES
----------------

The resized volume size (98697728) is 4096 bytes smaller than the
originally created volume size (98701824).  This might be due to
additional metadata as a result of resizing.  The same results are
seen with the old code that subtracted 1 byte.

In this situation I do not think we need to do anything different when
resizing ntfs file systems using cylinder alignment.


REISERFS TEST - CYLINDER ALIGNMENT
----------------------------------

$ sudo debugreiserfs /dev/sdd2 2> /dev/null | \
  egrep 'Blocksize|Count of blocks on the device|LABEL'
Count of blocks on the device: 24096
Blocksize: 4096
LABEL: reiserfs-test
$ sudo debugreiserfs /dev/sdd3 2> /dev/null | \
  egrep 'Blocksize|Count of blocks on the device|LABEL'
Count of blocks on the device: 24097
Blocksize: 4096
LABEL: reiserfs-test2
$

Actual resize command:
  sh -c 'echo y | resize_reiserfs -s 98703360 /dev/sdd3'


REISERFS DIFFERENCES
--------------------

The resized block count (24097) is one block larger than the
originally created block count (24096).  I am baffled by this one
because even the previous code which subtracted 1 byte had the same
24097 block count after resizing.

Q2)  Perhaps we should subtract 1 block (1 x 4096 byte blocks) when
     resizing reiserfs using cylinder alignment?


FURTHER TESTING
===============

Whatever we decide to do with regards to resizing and cylinder
alignment, we should also test with logical partitions.  Logical
partitions will align the partition start to be at a cylinder boundary
plus 1 track (63 sectors).
Comment 9 Jean-Pierre André 2013-05-29 07:47:03 UTC
Hi Curtis,

I have some difficulty interpreting your tests.

First, did you apply the patches attached to comment #3 (they were meant for ntfs-2013, but there is apparently no relevant difference) ?

Then, in the ntfsresize results you show the original volume and device sizes, so the values shown result from a former action (probably initial formatting). The actual resize command ("ntfsresize -P --force --force -s 98703360 /dev/sdd3") is the one I would expect.

Did you mean you started ntfsresize twice, and what is shown is the result of the second one, like :
ntfsresize -P -f -f /dev/sdd3
ntfsresize -P -i -f -f /dev/sdd3 | egrep etc. ?

Anyway, I do not understand why the result can be different on sdd2 and sdd3 : ntfsresize only knows about the writeable space within a partition (this is 192780 sectors in both cases, and still far from the end of the device).

Can you please clarify what the shown figures mean ?

> The resized volume size (98697728) is 4096 bytes smaller than the
> originally created volume size (98701824).

The value of 98697728 does not appear to be a consequence of resizing (unless some command is not shown). A maximum of 24097 clusters can fit in a partition of 98703360 bytes ((98703360-512)/4096), yielding a usable size of 98701312 bytes (and a "volume size" of 98701312+512 = 98701824)

> This might be due to additional metadata as a result of resizing.

Resizing may cause additional fragmentation, hence additional metadata, but this is not visible in the actual volume size.

> The same results are seen with the old code that subtracted 1 byte.

If the partition size is 98703360 and the cluster size is 4096, there remains 3 unusable sectors. When subtracting one byte, there are still 2 unusable sectors, so the difference is not visible.
Comment 10 Mike Fleetwood 2013-05-29 10:35:23 UTC
Hi Curtis,

Summary of the ext3 cylinder alignment case:
mkfs.ext3 creates file systems rounded to 4K even when they are small
ones using 1K block size.  Not a cylinder alignment issue.  No change
to GParted required.

Haven't got time to look at the ntfs and resizerfs cylinder alignment
cases now.

Thanks,
Mike


(In reply to comment #8)
> EXT3 Differences  (CYLINDER ALIGNMENT)
> ----------------
> 
> The resized block value (96390) was 2 blocks larger than the
> originally created partition (96388).  With shrinking, I would have
> expected a value the same or smaller than the originally created
> partition.  Interestingly, using the old code which subtracted 1 kiB,
> the resized block value (92689) was still 1 block larger than the
> originally created partition.
> 
> Q1)  Perhaps we should subtract 2 blocks (2 x 1024 byte blocks) when
>      resizing ext2/3/4 using cylinder alignment?

It's a consequence of how mkfs.ext3 and resize2fs behave.  Mkfs.ext3 is
creating the file system size in multiples of 4K.  Resize2fs maximises
in multiples of 4K.  Smaller ext[234] file systems use a 1K block size,
rather than a 4K block size, and resize2fs to specific size resizes to
that size.  (96388K is the largest 4K multiple below 96390K).

This is not a cylinder alignment issue.  We shoud not patch GParted for
this.


Repeating your testing ...  GParted ran:
    resize2fs /dev/md0p3 96390K

# sfdisk -s /dev/md0p2
96390
# tune2fs -l /dev/md0p2 | egrep 'Block count|Block size|Filesystem volume name'
Filesystem volume name:   created_at_94M
Block count:              96388
Block size:               1024
# sfdisk -s /dev/md0p3
96390
[root@mfleetwo3 ~]# tune2fs -l /dev/md0p3 | egrep 'Block count|Block size|Filesystem volume name'
Filesystem volume name:   shrunk_to_94M
Block count:              96390
Block size:               1024


Direct testing of mkfs.ext3 and resize2fs in a 96390K partition.
1) Recreate a new file system and get one sized 96388K.
2) Maximise file system, still get 96388K.
3) Resize to 96390K and the file system grows.

# sfdisk -s /dev/md0p2
96390
# dd if=/dev/zero of=/dev/md0p2 bs=1M count=1
# mkfs.ext3 -L recreated /dev/md0p2
...
24192 inodes, 96388 blocks
...
# tune2fs -l /dev/md0p2 | egrep 'Block count|Block size|Filesystem volume name'
Filesystem volume name:   recreated
Block count:              96388
Block size:               1024
# resize2fs /dev/md0p2
resize2fs 1.41.12 (17-May-2010)
The filesystem is already 96388 blocks long.  Nothing to do!
# tune2fs -l /dev/md0p2 | egrep 'Block count|Block size|Filesystem volume name'
Filesystem volume name:   recreated
Block count:              96388
Block size:               1024
# resize2fs /dev/md0p2 96390K
resize2fs 1.41.12 (17-May-2010)
Resizing the filesystem on /dev/md0p2 to 96390 (1k) blocks.
The filesystem on /dev/md0p2 is now 96390 blocks long.
# tune2fs -l /dev/md0p2 | egrep 'Block count|Block size|Filesystem volume name'
Filesystem volume name:   recreated
Block count:              96390
Block size:               1024


Creating different sized ext3 file systems to show 4K size rounding in
mkfs.ext3 for small file systems which use a 1K block size.

# truncate -s 10240K /tmp/10M0K.img
# truncate -s 10241K /tmp/10M1K.img
# truncate -s 10242K /tmp/10M2K.img
# truncate -s 10243K /tmp/10M3K.img
# truncate -s 10244K /tmp/10M4K.img
# losetup /dev/loop0 /tmp/10M0K.img
# losetup /dev/loop1 /tmp/10M1K.img
# losetup /dev/loop2 /tmp/10M2K.img
# losetup /dev/loop3 /tmp/10M3K.img
# losetup /dev/loop4 /tmp/10M4K.img
# mkfs.ext3 /dev/loop0
# mkfs.ext3 /dev/loop1
# mkfs.ext3 /dev/loop2
# mkfs.ext3 /dev/loop3
# mkfs.ext3 /dev/loop4
# tune2fs -l /dev/loop0 | egrep 'Block count|Block size'
Block count:              10240
Block size:               1024
# tune2fs -l /dev/loop1 | egrep 'Block count|Block size'
Block count:              10240
Block size:               1024
# tune2fs -l /dev/loop2 | egrep 'Block count|Block size'
Block count:              10240
Block size:               1024
# tune2fs -l /dev/loop3 | egrep 'Block count|Block size'
Block count:              10240
Block size:               1024
# tune2fs -l /dev/loop4 | egrep 'Block count|Block size'
Block count:              10244
Block size:               1024
Comment 11 Mike Fleetwood 2013-05-29 15:02:36 UTC
(In reply to comment #8)
> REISERFS DIFFERENCES (CYLINDER ALIGNMENT)
> --------------------
>
> The resized block count (24097) is one block larger than the
> originally created block count (24096).  I am baffled by this one
> because even the previous code which subtracted 1 byte had the same
> 24097 block count after resizing.
>
> Q2)  Perhaps we should subtract 1 block (1 x 4096 byte blocks) when
>      resizing reiserfs using cylinder alignment?

mkreiserfs creates file systems in multiples of 64K even though the file
system uses a 4K block size.  resize_reiserfs maximises to multiples of
64K too, but it resizes to specific sizes in multiples of 4K.

This is not a cylinder alignment issue.


Create reiserfs in a 40M+63K and 40M+64K device.

# expr 40 \* 1024 + 63
41023
# truncate -s 41023K /tmp/40M63K.img
# losetup /dev/loop0 /tmp/40M63K.img
# mkreiserfs -f /dev/loop0
# debugreiserfs /dev/loop0 2> /dev/null | egrep 'Blocksize|Count of blocks on the device'
Count of blocks on the device: 10240
Blocksize: 4096

# truncate -s 41024K /tmp/40M64K.img
# losetup /dev/loop1 /tmp/40M64K.img
# mkreiserfs -f /dev/loop1
# debugreiserfs /dev/loop1 2> /dev/null | egrep 'Blocksize|Count of blocks on the device'
Count of blocks on the device: 10256
Blocksize: 4096


Maximise those file systems.  No change.

# resize_reiserfs /dev/loop0
resize_reiserfs 3.6.21 (2009 www.namesys.com)
/dev/loop0 already is of the needed size. Nothing to be done

# resize_reiserfs /dev/loop1
resize_reiserfs 3.6.21 (2009 www.namesys.com)
/dev/loop1 already is of the needed size. Nothing to be done


Resize those file systems to the partition size.
1) First file system grows from 40M to 40M+60K leaving 3K unusable with
   a 4K block size.
2) Second file system 40M+64K doesn't change in size.

# sfdisk -s /dev/loop0
41023
# expr 41023 \* 1024
42007552
# resize_reiserfs -s 42007552 /dev/loop0
...
block count           10255 (10240)
...
# debugreiserfs /dev/loop0 2> /dev/null | egrep 'Blocksize|Count of blocks on the device'
Count of blocks on the device: 10255
Blocksize: 4096

# sfdisk -s /dev/loop1
41024
[root@mfleetwo3 ~]# expr 41024 \* 1024
42008576
[root@mfleetwo3 ~]# resize_reiserfs -s 42008576 /dev/loop1
/dev/loop1 already is of the needed size. Nothing to be done
Comment 12 Curtis Gedak 2013-05-29 16:06:50 UTC
Thank you Mike for your work on this bug report.

Your detailed analysis explains the discrepancies between creation and resizing are not due to cylinder alignment.  I was planning to do more testing today; however, your testing got the root of the discrepancy better than what I was planning.  Thank you for doing this.

When testing, I was only using the patch from comment #4.  I did not use the patch from comment #3.  If I understand Jean-Pierre correctly, the patch in comment #3 will set the backup boot sector when NTFS is called with the size in bytes.  As such I did not focus on testing if the NTFS backup boot sector was actually set.


Jean-Pierre,

Q1)  Which version of ntfsresize do you anticipate will contain your patch
     from comment #3 (or a similar patch)?



The patch in comment #4 has been committed to the git repository for inclusion in the next release of GParted.

The relevant git commits can be viewed at the following links:

Shrink file systems to exact size (#701075)
https://git.gnome.org/browse/gparted/commit/?id=3461413d283f1bac77e541b1054e775ec105212f

Place options before device name for the ntfsresize command
https://git.gnome.org/browse/gparted/commit/?id=28d41d0d6a5dcbea0630f63d0c3c5a53892227e3
Comment 13 Jean-Pierre André 2013-05-29 16:54:46 UTC
Hi Curtis,

> When testing, I was only using the patch from comment #4.  I did not use the
> patch from comment #3.  If I understand Jean-Pierre correctly, the patch in
> comment #3 will set the backup boot sector when NTFS is called with the size in
> bytes.  As such I did not focus on testing if the NTFS backup boot sector was
> actually set.

While fixing the backup boot sector issue, I saw that ntfsresize was erroneously
reserving a full cluster for the backup boot sector. A patch in comment #3 fixes this.

This does not explain why you do not get the same values in sdd2 and sdd3.

> Which version of ntfsresize do you anticipate will contain your patch
> from comment #3 (or a similar patch)?

The next one, ... expected in several months (except emergency). If you do not want to compile ntfs-3g, you may use a binary version just uploaded to http://jp-andre.pagesperso-orange.fr/ntfsresize This contains its own libntfs-3g and only uses legacy versions of standard libraries, so you should only have to put it in your test path and make it executable.
Comment 14 Curtis Gedak 2013-05-29 17:19:24 UTC
Hi Jean-Pierre,

Thank you for responding, and also for providing a link to a patched binary of ntfsresize.

I retested using your patched ntfsresize binary, and now the values are the same for initial partition creation (sdd2) and for shrinking (sdd3).  

TEST USING NTFSRESIZE FROM COMMENT #13
--------------------------------------

$ sudo ntfsresize -P -i -f -f /dev/sdd2 | \
  egrep 'Cluster size|Current volume size|Current device size'
Cluster size       : 4096 bytes
Current volume size: 98701824 bytes (99 MB)
Current device size: 98703360 bytes (99 MB)
$ sudo ntfsresize -P -i -f -f /dev/sdd3 | \
  egrep 'Cluster size|Current volume size|Current device size'
Cluster size       : 4096 bytes
Current volume size: 98701824 bytes (99 MB)
Current device size: 98703360 bytes (99 MB)


TEST USING NTFSRESIZE FROM KUBUNTU 12.04
----------------------------------------

When using the kubuntu 12.04 ntfs-3g package version 2012.1.15AR.1-1ubuntu1.2 I get different results for the resized partition (sdd3):

$ sudo ntfsresize -P -i -f -f /dev/sdd2 |   egrep 'Cluster size|Current volume size|Current device size'
Cluster size       : 4096 bytes
Current volume size: 98701824 bytes (99 MB)
Current device size: 98703360 bytes (99 MB)
$ sudo ntfsresize -P -i -f -f /dev/sdd3 | \
  egrep 'Cluster size|Current volume size|Current device size'
Cluster size       : 4096 bytes
Current volume size: 98697728 bytes (99 MB)
Current device size: 98703360 bytes (99 MB)


My guess is that there have been some changes to ntfs-3g since the version used by Ubuntu 12.04.
Comment 15 Jean-Pierre André 2013-05-29 20:05:16 UTC
Hi again,

> I retested using your patched ntfsresize binary, and now the values are the
> same for initial partition creation (sdd2) and for shrinking (sdd3).

Great. I did not know what actions you had applied to sdd2 and sdd3.

> $ sudo ntfsresize -P -i -f -f /dev/sdd3 | \
>   egrep 'Cluster size|Current volume size|Current device size'
> Cluster size       : 4096 bytes
> Current volume size: 98697728 bytes (99 MB)
> Current device size: 98703360 bytes (99 MB)
>
> My guess is that there have been some changes to ntfs-3g since the version
> used by Ubuntu 12.04.

This appears to me to be the consequence of ntfsresize reserving a full cluster for the backup boot sector (a bug older than Ubuntu 12.04) :

usable clusters = (98703360 - 4096)/4096 = 24096
volume size = (24096*4096 + 512) = 98697728

and unusable sectors = (98703360 - 98697728)/512 = 11 this is more than the cluster size, and the purpose of the patch was to use one more cluster.

Note : a partition of 98800128 bytes would be a better value for checking the missing byte in gparted. This is a multiple of 63 sectors, and is also a multiple of 8 sectors plus one sector. Hence a missing byte would lead to an unusable cluster.

If you agree that adding one byte to the size gparted specifies to ntfsresize leads to correct results with the patched ntfsresize, I suggest you close the issue (or at least the part of it related to ntfs).

Thank you for your cooperation.
Comment 16 Curtis Gedak 2013-05-30 18:05:00 UTC
Hi Jean-Pierre,

Thank you for providing an explanation for the difference between the
values for the initial partition creation versus resized partition.

(In reply to comment #15)
> Note : a partition of 98800128 bytes would be a better value for
> checking the missing byte in gparted. This is a multiple of 63
> sectors, and is also a multiple of 8 sectors plus one sector. Hence
> a missing byte would lead to an unusable cluster.

GParted supports two main alignment schemes, MiB and Cylinder.
Unfortunately 98800128 bytes is neither a multiple of MiB nor cylinder
size.

Calculations Follow:

On a modern disk device with 512 byte sectors, the geometry is assumed
to be 255 heads and 63 sectors per track.  (NOTE: This is most often a
fake geometry due to the constraints of representing these values in
the MSDOS partition table).

Hence cylinder size is typically:

Cylinder Size = (Heads) * (Sectors per track) * (Bytes per sector)
              = 255 * 63 * 512 = (16,065 sectors) * (512 bytes per sector)
              = 8,225,280 bytes


Cylinder Alignment Check:

# Cylinders = (98,800,128 bytes) / ( 8,224,280 bytes per cylinder)
            = 12.0117647059 cylinders
            = (not an integer multiple)

Hence GParted does not "normally" create partitions of this size.  I
say "normally" because by using another partition tool we could
pre-create a partition such that gparted would resize the partition to
this size (no change to start, only resize end partition boundary).


Ideally for testing we need a value that is a multiple of cylinder
size, and also a (multiple of 8 sectors) plus one sector.  A single
cylinder meets this criteria.

1 cylinder = 16,065 sectors = (2008 * 8) + 1 sectors


TEST WITH OLD GPARTED CODE AND PATCHED NTFSRESIZE
-------------------------------------------------

Legend:  sdd2 is created 8 MiB cylinder aligned
         sdd3 is shrunk to 8 MiB cylinder aligned

Resize command:  ntfsresize -P --force --force /dev/sdd3 -s 8225279

$ sudo ntfsresize -P -i -f -f /dev/sdd2 | \
  egrep 'Cluster size|Current volume size|Current device size'
Cluster size       : 4096 bytes
Current volume size: 8225280 bytes (9 MB)
Current device size: 8225280 bytes (9 MB)
$ sudo ntfsresize -P -i -f -f /dev/sdd3 | \
  egrep 'Cluster size|Current volume size|Current device size'
Cluster size       : 4096 bytes
Current volume size: 8221184 bytes (9 MB)
Current device size: 8225280 bytes (9 MB)
$

Note the discrepancy in Current volume size which is expected and the
reason for this bug report.


TEST WITH PATCHED GPARTED CODE AND PATCHED NTFSRESIZE
-----------------------------------------------------

Resize command:  ntfsresize -P --force --force /dev/sdd3 -s 8225289

$ sudo ntfsresize -P -i -f -f /dev/sdd2 | \
  egrep 'Cluster size|Current volume size|Current device size'
Cluster size       : 4096 bytes
Current volume size: 8225280 bytes (9 MB)
Current device size: 8225280 bytes (9 MB)
$ sudo ntfsresize -P -i -f -f /dev/sdd3 | \
  egrep 'Cluster size|Current volume size|Current device size'
Cluster size       : 4096 bytes
Current volume size: 8225280 bytes (9 MB)
Current device size: 8225280 bytes (9 MB)
$

Note there is no discrepancy meaning that this bug has been fixed.


> If you agree that adding one byte to the size gparted specifies to
> ntfsresize leads to correct results with the patched ntfsresize, I
> suggest you close the issue (or at least the part of it related to
> ntfs).

I agree that this issue is resolved for ntfs file systems.

In an effort to reduce the creation of duplicate bug reports, we hold
off closing the bug reports until an official release contains the
code.  For a list of bug reports already fixed for the next GParted
release see:

Development Plans for the Next Release of GParted (0.17.0)
http://gparted-forum.surf4.info/viewtopic.php?id=16872

We plan to close this report when the next version of GParted is released.


> Thank you for your cooperation.

You're welcome.  And thank you for all your work on ntfs-3g.  :-)

Sincerely,
Curtis Gedak
Comment 17 Curtis Gedak 2013-09-18 16:51:27 UTC
The patches to address this report have been included in GParted 0.16.2 released on September 18, 2013.