GNOME Bugzilla – Bug 688883
Improve SSD handling by using TRIM
Last modified: 2020-11-13 10:41:21 UTC
The aim is to maximise performance of SSDs without requiring any additional operations or actions by users of GParted. This will also minimise the use of backing storage required by thin provisioning enterprise storage systems and virtual machines which support it. Also see the discussion in the GParted forum thread: Idea to improved SSD handling by using TRIM http://gparted-forum.surf4.info/viewtopic.php?id=16524 Plan to use hdparm command to TRIM the blocks immediately before the creation of a new file system in a partition. Ordering will be: 1) Wipe existing file system signatures; 2) Use hdparm to TRIM the blocks in the partition; 3) Create the file system. Plan to add execution of hdparm as a displayed step in the creation of a new file system so users can see its execution and it will be recorded in the operation results. No user requirement will be required. TRIM will just automatically be used as and when needed. Need to use the above ordering to ensure old file system signatures are fully erased as I have seen information suggesting some SSDs report the original data after it being trimmed, rather than zeros. I'll code this, Mike
Hi Mike, you might take note that at least e2fsprogs already takes care of this automatically when formatting ext[234], and you can use e2fsck -E discard to trim the free blocks on an existing filesystem.
It would be better to TRIM all blocks in the newly created partition (before running mkfs), because not all use ext4 file system. But hdparm doesn't recommend to use --trim-sector-ranges option I'm pasting from hdparm manual: --trim-sector-ranges For Solid State Drives (SSDs). EXCEPTIONALLY DANGEROUS. DO NOT USE THIS OPTION!! Tells the drive firmware to discard unneeded data sectors, destroying any data that may have been present within them. This makes those sectors available for immediate use by the firmware's garbage collection mechanism, to improve scheduling for wear-leveling of the flash media. This option expects one or more sector range pairs immediately after the option: an LBA starting address, a colon, and a sector count (max 65535), with no intervening spaces. EXCEPTIONALLY DANGEROUS. DO NOT USE THIS OPTION!! E.g. hdparm --trim-sector-ranges 1000:4 7894:16 /dev/sdz See http://fossies.org/linux/misc/hdparm-9.43.tar.gz:a/hdparm-9.43/hdparm.8 http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/saucy/hdparm/saucy/view/head:/hdparm.8
Or perhaps the other filesystems just need to grow TRIM support, and this falls outside of the scope of gparted.
hdparm's warnings are appropriate for disks that are in use. --trim-sector-ranges does exactly what it says it does, which is very dangerous if you've got the sector numbers wrong. That's no reason for gparted not to use it.
Survey of what's available: hdparm >= 9.17 (released 2009-07-29) has this usage: hdparm --please-destroy-my-drive --trim-sector-ranges START:COUNT /dev/HD It also includes wiper.sh which requires hdparm >= 9.17. hdparm 9.16 (released 2009-07-16) has this usage: hdparm --please-destroy-my-drive --yes-i-know-what-i-am-doing \ --trim-sectors START COUNT /dev/HD On CentOS 6.4 this core dumps! Util-linux >= 2.23 (released 2013-04-25) has this usage: blkdiscard /dev/PTN Linux kernel >= 2.6.27 (released 2008-10-09) received BLKDISCARD support with commit d30a260. Also mkfs.ext2/3/4, mkfs.xfs and mkfs.btrfs try to discard the block for a recent enough version of the tools. Xfs and btrfs just do a single ioctl(fd, BLKDISCARD, &range) to discard the whole partition. Mkfs.ext2/3/4 discards in 2 GiB amounts optionally showing a progress bar. e2fsprogs >= 1.41.10 (released 2010-02-10) xfsprogs >= 3.0.5 (released 2009-10-23) btrfs-progs commit 43a06f0 (committed 2011-10-25) Distribution Trim/discard support hdparm util-linux kernel RHEL / CentOS 5.9 No (6.6) No (2.13) No (2.6.18) RHEL / CentOS 6.4 Crash (9.16) No (2.17.2) Yes (2.6.32) Debian 6 Yes (9.32) No (2.17.2) Yes (2.6.32) Debian 7 Yes (9.39) No (2.20.1) Yes (3.2) Kubuntu 12.04 LTS Yes (9.37) No (2.20.1) Yes (3.2) e2fsprogs xfsprogs btrfs-progs RHEL /CentOS 5.9 No (1.39) No (2.9.4) (None) RHEL /CentOS 6.4 Yes (1.41.12) Yes (3.1.1) (None) Debian 6 Yes (1.41.12) Yes (3.1.4) No (20100601) Debian 7 Yes (1.42.5) Yes (3.1.7) Yes (20120328) Kubuntu 12.04 LTS Yes (1.42) Yes (3.1.7) No (20100601) Want to provide the benefits of trim to all file systems and if at all possible all GParted supported distributions. Relying of file system trim support doesn't achieve the goal. blkdiscard from util-linux isn't available on any long term distributions yet. hdparm is available in most distributions but not available in any current Red Hat ones. There is no suitable command. Looking at the code of blkdiscard and mkfs.* they mostly just open an fd to the partition and call ioctl(fd, BLKDISCARD, &range) of the whole partition and let the kernel do it. Seems relatively easy. Plan to use this method. This will allow all but RHEL / CentOS 5.x to be supported.
My mistake. It is actually linux >= 2.6.28 (released 2008-12-25) which contains that BLKDISCARD commit.
Something to consider is that discarding twice may waste a not insignificant amount of time. May need to have a per filesystem flag to know if the mkfs tool discards, then have gparted not duplicate it.
Created attachment 267456 [details] [review] Trim support (draft 1) Hi Curtis and Larry, Here's draft 1 patch to add TRIM (aka DISCARD) support. I've tested this on a spinning hard drive, one SSD that I have and 2 USB keys. Results are: (any) HDD [100GiB ptn] Operation not supported, takes < 0.001 secs Samsung 830 SSD [29GiB ptn] works, takes between 0.009 and 0.018 secs SanDisk Cruzer [2GiB ptn] Operation not supported, takes < 0.001 secs Micro USB key Patriot USB key [30GiB ptn] Operation not supported, takes < 0.001 secs This is what the debugging output looks like when it works on my Samsung 830 SSD: 0.000000 +0.000000 STOPWATCH_RESET 0.000000 +0.000000 discard_partition() open("/dev/sda9", O_WRONLY) 0.000300 +0.000300 discard_partition() open() -> 8 0.000547 +0.000247 discard_partition() range[2]={0, 29535240192} 0.000594 +0.000047 discard_partition() ioctl(fd, BLKDISCARD, &range) 0.013254 +0.012659 discard_partition() ioctl() -> 0 0.013473 +0.000220 discard_partition() return 1 and when DISCARD isn't supported on my HDD: 0.000000 +0.000000 STOPWATCH_RESET 0.000000 +0.000000 discard_partition() open("/dev/sda14", O_WRONLY) 0.000090 +0.000090 discard_partition() open() -> 8 0.000115 +0.000024 discard_partition() range[2]={0, 133413470208} 0.000120 +0.000005 discard_partition() ioctl(fd, BLKDISCARD, &range) 0.000135 +0.000015 discard_partition() ioctl() -> -1 0.000168 +0.000032 discard_partition() errno=95 "Operation not supported" 0.000190 +0.000022 discard_partition() return 0 I suspect that my Samsung 830 SSD can't actually discard 29 GiB in a fraction of a second, but actually does it in background, but it doesn't delay subsequent use of the drive while it does it so it doesn't really matter. Could you test reformatting or creating partitions on any SSDs you have and report the outcome. Thanks, Mike
Hi Mike, Testing with the patch set from comment #8 by creating NTFS partitions on IDE HDDs, SATA HDDs, and a generic USB Flash Drive produced no ill effects. DEVICE CLEAR OLD FS SIGNATURES TIME ------------------------------ ---------------------------- 160 GB SATA Intel Fake RAID 0 seconds 160 GB IDE HDD 0 seconds 2 GB Generic USB Flash Drive 0 seconds In all cases the debug code indicated "Operation not supported". Unfortunately my only SSD is in production use containing the OS for a Mythbuntu PVR, so it is not easily available for testing. I will look into acquiring an inexpensive SSD so that I can test further. Larry also lacks an SSD for testing. ;-( Curtis
Hi Curtis, Larry has let me know he is shopping for an SSD too. (I didn't intend to get you two to spend money ;-). My primary concern is that some SSDs take a long time to do a trim and users end up waiting without any recorded details as to what is happening. (So far I only have a single example on my SSD which does it in background returning immediately. I suspect nearly all will behave this way). For the final code I will make trimming a displayed operation detail always reporting success. This way if any user's SSD does take a long time at least they can see what step it is doing. Thanks, Mike
Just in case, my SSDs (KINGSTON SNVP325S2128GB and SanDisk SSD U100 252GB) use to fstrim in a few to several seconds, something like <20s for any partitions up to ~90Gb and usually 3..5s.
Created attachment 268523 [details] Log file from creating very first ext4 partition that spans new SSD Hi Mike and Larry, This week my local computer store had some SSDs on sale so I picked one up. :-) Kingston SSDnow 300V (120 GB) SV300S37A/120GCR Following are the results of creating my first partition on the device, and also reformatting said partition. A) PARTITION CREATION --------------------- 1) Create a new MSDOS partition table on the SSD 2) Create an ext4 partition that spans the entire SSD (111.79 GiB). 3) Apply Note that the Applying pending operation window hung for about 25 seconds (no bounce back and forth in the pulse bar, and no response to clicking on details expander) before finally unfreezing and continuing to display results. 0.000000 +0.000000 STOPWATCH_RESET 0.000000 +0.000000 discard_partition() open("/dev/sdb1", O_WRONLY) 0.000047 +0.000047 discard_partition() open() -> 10 0.000060 +0.000013 discard_partition() range[2]={0, 120032591872} 0.000063 +0.000003 discard_partition() ioctl(fd, BLKDISCARD, &range) 27.598086 +27.598023 discard_partition() ioctl() -> 0 27.598143 +0.000057 discard_partition() return 1 The gparted_details.htm log file for this step is attached. B) PARTITION REFORMAT --------------------- 4) Queue an NTFS partition reformat 5) Apply The GParted main window hung for about 25 seconds and the Applying pending operation window did not display until after about 25 seconds. When the Applying operation window did display, it indicated that all operations had successfully completed. 0.000000 +0.000000 STOPWATCH_RESET 0.000000 +0.000000 discard_partition() open("/dev/sdb1", O_WRONLY) 0.000102 +0.000102 discard_partition() open() -> 10 0.000129 +0.000027 discard_partition() range[2]={0, 120032591872} 0.000136 +0.000007 discard_partition() ioctl(fd, BLKDISCARD, &range) 27.625729 +27.625593 discard_partition() ioctl() -> 0 27.625791 +0.000062 discard_partition() return 1 The hang in the GUI is disconcerting. If the delay is caused by the SSD trim then we might need to investigate setting up the SSD trim operation as a separate thread. Please note that my experience with SSDs is very limited so I cannot draw too many inferences from what occurred. Curtis
Hi Curtis, Thanks for this testing, it was very valuable. So for your Kingston SSD DISCARD is performed in foreground and takes 27 seconds for 120G [1]. Also having GParted DISCARD and then mkfs.btrfs/ext234/xfs DISCARD too wastes another 27 seconds [2]. So if I'm to add support to GParted for this it needs to not block the GUI thread and ideally also not do it for recent versions of mkfs.btrfs/ ext234/xfs which already do it. Thanks, Mike [1] Kernel spends 27.598023 seconds running ioctl(BLKDISCARD) on a 120 GB range: 0.000063 +0.000003 discard_partition() ioctl(fd, BLKDISCARD, &range) 27.598086 +27.598023 discard_partition() ioctl() -> 0 [2] Creation of ext4 on a 120 GB partition takes 28 seconds, 27 seconds of which will have come from the DISCARD (from attached gparted_details.htm): Create Primary Partition #1 (ext4, 111.79 GiB) on /dev/sdb 00:00:57 ( SUCCESS ) ... create new ext4 file system 00:00:28 ( SUCCESS ) mkfs.ext4 -L "" /dev/sdb1 Discarding device blocks: done ...
Hi Mike, I believe your interpretation of the results is correct. When I manually format an ext4 file system from the command line, the process spends about 25 seconds (I didn't time this precisely) in the initial section showing the progress while discarding blocks: Discarding device blocks: #######/####### Eventually finishing with: Discarding device blocks: done Since the mkfs.ext4 process is run asynchronously in GParted, this portion is viewable in the Applying pending operation window. When you need more testing just let me know. Curtis
Created attachment 269073 [details] screenshot of gparted operations Hi Curtis and Mike I just tested my new SSD, 60 GO ssd. Gparted is running from within usb key, so it takes a bit more time to perform operation. Here are debug details copied from terminal : root@ubuntu:~/gparted# src/gpartedbin ====================== libparted : 2.3 ====================== /dev/sda: unrecognised disk label /dev/sda: unrecognised disk label 0.000000 +0.000000 STOPWATCH_RESET 0.000000 +0.000000 discard_partition() open("/dev/sda1", O_WRONLY) 0.000432 +0.000432 discard_partition() open() -> 6 0.000641 +0.000209 discard_partition() range[2]={0, 41295020032} 0.000728 +0.000086 discard_partition() ioctl(fd, BLKDISCARD, &range) 9.354402 +9.353674 discard_partition() ioctl() -> 0 9.354632 +0.000229 discard_partition() return 1 11.497735 +2.143103 STOPWATCH_RESET 0.000000 +0.000000 discard_partition() open("/dev/sda5", O_WRONLY) 0.000854 +0.000854 discard_partition() open() -> 6 0.001013 +0.000159 discard_partition() range[2]={0, 4194304000} 0.001081 +0.000068 discard_partition() ioctl(fd, BLKDISCARD, &range) 0.977129 +0.976048 discard_partition() ioctl() -> 0 0.977426 +0.000296 discard_partition() return 1 1.662346 +0.684920 STOPWATCH_RESET 0.000000 +0.000000 discard_partition() open("/dev/sda6", O_WRONLY) 0.001264 +0.001264 discard_partition() open() -> 6 0.001947 +0.000683 discard_partition() range[2]={0, 1048576000} 0.002453 +0.000506 discard_partition() ioctl(fd, BLKDISCARD, &range) 0.243313 +0.240860 discard_partition() ioctl() -> 0 0.243617 +0.000303 discard_partition() return 1 1.707776 +1.464160 STOPWATCH_RESET 0.000000 +0.000000 discard_partition() open("/dev/sda7", O_WRONLY) 0.000326 +0.000326 discard_partition() open() -> 6 0.000488 +0.000162 discard_partition() range[2]={0, 13479444480} 0.000572 +0.000084 discard_partition() ioctl(fd, BLKDISCARD, &range) 3.113847 +3.113275 discard_partition() ioctl() -> 0 3.114186 +0.000339 discard_partition() return 1 root@ubuntu:~/gparted# And here are gparted-details saved after operations are completed : GParted 0.17.0-git Libparted 2.3 Create Primary Partition #1 (ntfs, 38.46 GiB) on /dev/sda 00:00:12 ( SUCCESS ) create empty partition 00:00:01 ( SUCCESS ) path: /dev/sda1 start: 2048 end: 80656383 size: 80654336 (38.46 GiB) clear old file system signatures in /dev/sda1 00:00:00 ( SUCCESS ) write 68.00 KiB of zeros at byte offset 0 00:00:00 ( SUCCESS ) write 4.00 KiB of zeros at byte offset 67108864 00:00:00 ( SUCCESS ) write 4.00 KiB of zeros at byte offset 41295015936 00:00:00 ( SUCCESS ) flush operating system cache of /dev/sda 00:00:00 ( SUCCESS ) set partition type on /dev/sda1 00:00:00 ( SUCCESS ) new partition type: ntfs create new ntfs file system 00:00:02 ( SUCCESS ) mkntfs -Q -v -L "" /dev/sda1 Cluster size has been automatically set to 4096 bytes. Creating NTFS volume structures. Creating root directory (mft record 5) Creating $MFT (mft record 0) Creating $MFTMirr (mft record 1) Creating $LogFile (mft record 2) Creating $AttrDef (mft record 4) Creating $Bitmap (mft record 6) Creating $Boot (mft record 7) Creating backup boot sector. Creating $Volume (mft record 3) Creating $BadClus (mft record 8) Creating $Secure (mft record 9) Creating $UpCase (mft record 0xa) Creating $Extend (mft record 11) Creating system file (mft record 0xc) Creating system file (mft record 0xd) Creating system file (mft record 0xe) Creating system file (mft record 0xf) Creating $Quota (mft record 24) Creating $ObjId (mft record 25) Creating $Reparse (mft record 26) Syncing root directory index record. Syncing $Bitmap. Syncing $MFT. Updating $MFTMirr. Syncing device. mkntfs completed successfully. Have a nice day. ======================================== Create Extended Partition #2 (extended, 17.44 GiB) on /dev/sda 00:00:00 ( SUCCESS ) create empty partition 00:00:00 ( SUCCESS ) path: /dev/sda2 start: 80656384 end: 117229567 size: 36573184 (17.44 GiB) ======================================== Create Logical Partition #3 (linux-swap, 3.91 GiB) on /dev/sda 00:00:02 ( SUCCESS ) create empty partition 00:00:00 ( SUCCESS ) path: /dev/sda5 start: 80658432 end: 88850431 size: 8192000 (3.91 GiB) clear old file system signatures in /dev/sda5 00:00:00 ( SUCCESS ) write 68.00 KiB of zeros at byte offset 0 00:00:00 ( SUCCESS ) write 4.00 KiB of zeros at byte offset 67108864 00:00:00 ( SUCCESS ) write 4.00 KiB of zeros at byte offset 4194299904 00:00:00 ( SUCCESS ) flush operating system cache of /dev/sda 00:00:00 ( SUCCESS ) set partition type on /dev/sda5 00:00:01 ( SUCCESS ) new partition type: linux-swap(v1) create new linux-swap file system 00:00:00 ( SUCCESS ) mkswap -L "" /dev/sda5 Setting up swapspace version 1, size = 4095996 KiB LABEL=, UUID=1a7d50e5-49e7-4054-980e-c0d948a1ca69 ======================================== Create Logical Partition #4 (ext3, 1000.00 MiB) on /dev/sda 00:00:01 ( SUCCESS ) create empty partition 00:00:00 ( SUCCESS ) path: /dev/sda6 start: 88852480 end: 90900479 size: 2048000 (1000.00 MiB) clear old file system signatures in /dev/sda6 00:00:00 ( SUCCESS ) write 68.00 KiB of zeros at byte offset 0 00:00:00 ( SUCCESS ) write 4.00 KiB of zeros at byte offset 67108864 00:00:00 ( SUCCESS ) write 4.00 KiB of zeros at byte offset 1048571904 00:00:00 ( SUCCESS ) flush operating system cache of /dev/sda 00:00:00 ( SUCCESS ) set partition type on /dev/sda6 00:00:01 ( SUCCESS ) new partition type: ext3 create new ext3 file system 00:00:00 ( SUCCESS ) mkfs.ext3 -L "" /dev/sda6 Discarding device blocks: done Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks 64000 inodes, 256000 blocks 12800 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=264241152 8 block groups 32768 blocks per group, 32768 fragments per group 8000 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376 Allocating group tables: done Writing inode tables: done Creating journal (4096 blocks): done Writing superblocks and filesystem accounting information: done mke2fs 1.42.8 (20-Jun-2013) ======================================== Create Logical Partition #5 (ext4, 12.55 GiB) on /dev/sda 00:00:09 ( SUCCESS ) create empty partition 00:00:01 ( SUCCESS ) path: /dev/sda7 start: 90902528 end: 117229567 size: 26327040 (12.55 GiB) clear old file system signatures in /dev/sda7 00:00:00 ( SUCCESS ) write 68.00 KiB of zeros at byte offset 0 00:00:00 ( SUCCESS ) write 4.00 KiB of zeros at byte offset 67108864 00:00:00 ( SUCCESS ) write 4.00 KiB of zeros at byte offset 13479440384 00:00:00 ( SUCCESS ) flush operating system cache of /dev/sda 00:00:00 ( SUCCESS ) set partition type on /dev/sda7 00:00:00 ( SUCCESS ) new partition type: ext4 create new ext4 file system 00:00:05 ( SUCCESS ) mkfs.ext4 -L "root" /dev/sda7 Discarding device blocks: done Filesystem label=root OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks 824160 inodes, 3290880 blocks 164544 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=3372220416 101 block groups 32768 blocks per group, 32768 fragments per group 8160 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208 Allocating group tables: done Writing inode tables: done Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done mke2fs 1.42.8 (20-Jun-2013) ======================================== Hope this help Mike :) Larry
bugzilla.gnome.org is being replaced by gitlab.gnome.org. We are closing all old bug reports and feature requests in GNOME Bugzilla which have not seen updates for a long time. If you still use gparted and if you still see this bug / want this feature in a recent and currently supported version, then please feel free to report it at https://gitlab.gnome.org/GNOME/gparted/-/issues/ by following the guidelines at https://wiki.gnome.org/Community/GettingInTouch/BugReportingGuidelines Thank you for creating this report and we are sorry it could not be implemented so far (volunteer workforce and time is limited).