GNOME Bugzilla – Bug 735525
Replace NTFS preset with UDF
Last modified: 2018-05-24 10:31:04 UTC
The current format dialog offers presets of FAT32, NTFS, and Ext4. I suggest that NTFS is replaced with UDF. Reasoning: - UDF supports large files just like NTFS - NTFS is proprietary, whereas UDF is a formal standard - OSX can only read NTFS without 3rd party software - Performance under Linux is better because write the kernel supports UDF (NTFS requires FUSE) Windows XP is the only somewhat relevant OS that only supports UDF up to 2.01 (later versions via drivers included in BluRay and HDDVD drives), all other operating systems (incl Vista and newer) support the latest UDF version, 2.60, out of the box. OSX assigns UDF-formatted flash drives a DVD icon but they work just fine. According to my tests on various platforms the benefits outweigh the downsides.
In https://bugzilla.gnome.org/show_bug.cgi?id=770738 we did not decide for UDF as preset, but the patch considers it in the custom FS page.
I think the bottom line one size fits all is going to be FAT32. There's just no ambiguity of support there, and with 4K blocks it supports up to 16TB. The big drawback for FAT is the lack of a journal, so the bigger the drive, the slower the drive, and the more file churn, the more I'd want a journaled file system. The drawback is, macOS only has read-only support. So that's back to FAT32, or sure UDF is reasoanble too with the small drawback that I can't find an fsck for UDF on any platform. The unfortunate reality is either the user needs to know about file systems to make a choice. Or the software needs to know the use case, to alter the default. If you don't know anything at all about the use case, probably just format it FAT32. Suboptimal in many technical ways, but it will definitely work as intended. UDF is a little bit weird in that I'm pretty sure at least one of the platforms, maybe macOS or Linux, does not like (does not recognize) UDF on a partition. The whole drive is to be formatted UDF, no partition map. So that'd need to be tested.
(In reply to Chris Murphy from comment #2) > I think the bottom line one size fits all is going to be FAT32. There's just > no ambiguity of support there, and with 4K blocks it supports up to 16TB. > > The big drawback for FAT is the lack of a journal, so the bigger the drive, > the slower the drive, and the more file churn, the more I'd want a journaled > file system. The drawback is, macOS only has read-only support. So that's > back to FAT32, or sure UDF is reasoanble too with the small drawback that I > can't find an fsck for UDF on any platform. Windows can repair UDF (has GUI + chkdsk). For Linux fsck with repair support is in progress, look for udftools project. > The unfortunate reality is either the user needs to know about file systems > to make a choice. Or the software needs to know the use case, to alter the > default. That is truth. > If you don't know anything at all about the use case, probably just > format it FAT32. Suboptimal in many technical ways, but it will definitely > work as intended. > > UDF is a little bit weird in that I'm pretty sure at least one of the > platforms, maybe macOS or Linux, does not like (does not recognize) UDF on a > partition. This can be deduced from UDF specification. And Mac OS X enforce it as it can detect & automount UDF file system only if is on whole disk device (not partition)... But manually call "mount" on Mac OS X can be used to detect & mount UDF also from partition. > The whole drive is to be formatted UDF, no partition map. So > that'd need to be tested. Testing was already done, search for format-udf project on github. I will write results with compatibility information about UDF into mkudffs man page. So everything would be at one place -- in Linux tool for formatting disk as UDF.
By the way, both FAT32 and UDF have the same 2TB limitation. So a different choice must be made above that size, and realistically it ought to be made before 2TB for data safety and efficiency reasons. I'd maybe pick 500GB as the cutoff. How to choose? User has to know, or the software has to ask about use case in order to infer the format. Linux only = ext4, XFS, Btrfs. Linux + Windows = NTFS. Linux + macOS = HFS+ (non-journaled) Linux + Windows + macOS = exFAT
(In reply to Chris Murphy from comment #4) > By the way, both FAT32 and UDF have the same 2TB limitation. Not fully truth. UDF has upper limit at (2^32-1)*blocksize and blocksize must match logical sector size. For 512 disks it is 2TB-512. But for 4k disks it is 16TB-4k. So when deciding based on disk capacity and fact that 4k disks are not rare anymore, this should be used in choosing sane defaults. > So a different > choice must be made above that size, and realistically it ought to be made > before 2TB for data safety and efficiency reasons. I'd maybe pick 500GB as > the cutoff. > > How to choose? User has to know, or the software has to ask about use case > in order to infer the format. > > Linux only = ext4, XFS, Btrfs. > Linux + Windows = NTFS. > Linux + macOS = HFS+ (non-journaled) > Linux + Windows + macOS = exFAT Linux has native read-only support for NTFS for a long time. R/W is supported by ntfs-3g and it starts to be available on more distributions. But does not have to be preinstalled. exFAT is not supported on Linux and more linux distributions does not have exfat fuse driver installed by default. Therefore I would not say that exFAT is available for Linux in same way as NTFS. I would say exFAT support on Linux is comparable to ext2 support on Windows. ext2fsd can be installed on Windows and supports R/W for ext2 disks, similarly like exfat can be installed on Linux userspace and supports R/W for exFAT. Therefore if user is going to choose FS based on OS support I would write: Linux only = ext4, XFS, Btrfs Linux + Windows = NTFS Linux + Mac = HFS+ Windows + Mac = exFAT Linux + Windows (with 3rd app) = ext2 Linux (with 3rd app) + Windows = exFAT
I could not in good conscience recommend the 3rd party ext2 driver on Windows. The reliability of NTFS as the interchange volume format between Linux and Windows is much more reliable. And the bigger the drive gets, the more we need journaling, regardless of the logical block size.
(In reply to Chris Murphy from comment #6) > I could not in good conscience recommend the 3rd party ext2 driver on > Windows. I think it is comparable with exfat Linux one. And it does not contradict that (non-)recommendation :-) > The reliability of NTFS as the interchange volume format between > Linux and Windows is much more reliable. Yes. > And the bigger the drive gets, the more we need journaling, regardless of > the logical block size. Just because of fast crash recovery (without need to traverse whole disk)? Or do you have in mind another reason?
>Just because of fast crash recovery (without need to traverse whole disk)? Or do you have in mind another reason? Both fast crash recovery and better repair tools. Yes those repair tools only exist on Windows, and that's not good, but at least the data can be saved. Even NTFS is suboptimal, but we don't have cross platform Btrfs, ZFS, or ReFS. Any of those is the better option at this volume size, in particular with USB removable HDDs which are notorious flakey connections and not flushing to disk but claiming they have; and CoW tolerates this better. Yes you lose some recent data not fully committed to disk but at least the file system still mounts and you don't lose previously committed data.
With crash recovery we are in poor state on Linux when it comes to those NTFS and exFAT. Basically those fuse drivers can damage FS even more when FS is in inconsistent state... I do not know what could be good default option for large disks, but probably user should be informed about those problems for every one FS and more-power-users then should be able to choose FS which fits for their needs.
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/gnome-disk-utility/issues/26.