GNOME Bugzilla – Bug 160787
lvm support
Last modified: 2012-02-22 00:57:29 UTC
It would be a real good feature to have be able to have this functionality in gparted for LVM volumes.
Eurhm, what feature? support for lvm (heh, i hardly know what it is :P ) Could you be a bit more specific? thnx :)
I am running Fedora Core 3 which by default will take the remaining space (after /boot is created) and put it into LVM. Under the LVM partition the installer creates the / partition and the swap. LVM allows a user to extend a partition across multiple physical disks. It also allows for easy management of the space. It essentially virtualizes the disk so you can easily change it. It has many commands to change/move/get metadata from the LVM partition. It would be a great compliment to your gparted interface. I don't know of any LVM managers that are even remotely as easy to use and graphically pleasing as gparted. Joe
Created attachment 34640 [details] screen shot of LVM Here is a screen shot of a lvm partition. This actually has 2 partitions inside of it.
ok... thanks for your explanation. Could you please post your partitionlayout (preferably from fdisk AND parted) ? That way i can see how these tools handle lvm and maybe get some ideas on how to implement this in gparted. thnx so far.. :)
Here are the outputs you asked for. I am also including information that you might find useful from lv commands including what command I ran. I am also including my /etc/fstab file that defines the labels. I have also attached all the available commands for lvm at the end. If you don't have access to a system with LVM I can give you the man pages and other assistance. fdisk is: Disk /dev/hda: 60.0 GB, 60011642880 bytes 255 heads, 63 sectors/track, 7296 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System /dev/hda1 * 1 13 104391 83 Linux /dev/hda2 14 7296 58500697+ 8e Linux LVM parted: (parted) print Disk geometry for /dev/hda: 0.000-57231.562 megabytes Disk label type: msdos Minor Start End Type Filesystem Flags 1 0.031 101.975 primary ext3 boot 2 101.975 57231.562 primary lvm lvdisplay (lvm tool) [root@myhost ~]# lvdisplay --- Logical volume --- LV Name /dev/VolGroup00/LogVol00 VG Name VolGroup00 LV UUID bC5rYN-IB0f-KSPv-8PHc-m5DL-WxW9-G754cs LV Write Access read/write LV Status available # open 1 LV Size 53.81 GB Current LE 1722 Segments 1 Allocation inherit Read ahead sectors 0 Block device 253:0 --- Logical volume --- LV Name /dev/VolGroup00/LogVol01 VG Name VolGroup00 LV UUID QlxLUq-7TC1-Qyo8-FMdp-UhtN-3Hlx-3wOzwy LV Write Access read/write LV Status available # open 1 LV Size 1.94 GB Current LE 62 Segments 1 Allocation inherit Read ahead sectors 0 Block device 253:1 lvs : lvs LV VG Attr LSize Origin Snap% Move Copy% LogVol00 VolGroup00 -wi-ao 53.81G LogVol01 VolGroup00 -wi-ao 1.94G lvm commands: lvchange lvextend lvmdiskscan lvm.static lvrename lvscan lvcreate lvm lvmsadc lvreduce lvresize lvdisplay lvmchange lvmsar lvremove lvs
Sorry I forgot my fstab. # This file is edited by fstab-sync - see 'man fstab-sync' for details /dev/VolGroup00/LogVol00 / ext3 defaults 1 1 LABEL=/boot /boot ext3 defaults 1 2 none /dev/pts devpts gid=5,mode=620 0 0 none /dev/shm tmpfs defaults 0 0 none /proc proc defaults 0 0 none /sys sysfs defaults 0 0 /dev/VolGroup00/LogVol01 swap swap defaults 0 0
More info, because I really would like this feature. From a user point of view: A lvm partition (aka a logical volume) is like a normal partition, that you never need to move. Its parent is not a normal partition, but a volume group, made up of several partitions. The main advantage is that you can always resize it provided there is enough room in the underlying volume group, which avoids the common problem of having to move around all partitions when one is full. This part uses commands like lvm lvresize, lvm lvcreate, lvm lvremove... Then there is another part to it, which is that partitions can be added to a (possibly new) volume group. Commands are: lvm pvcreate (sets up a partition for lvm use), lvm vgcreate, lvm vgextend (to add physical volumes to the volume group). Finally there are other manipulations that would require adding some sort of volume groups list to the UI: merging and spitting volume groups with lvm vgmerge... As far as I know the only API is the command line.
Well, since lots of people request this feature, i've moved it up the priority list :) However, i wont have much time the next few weeks. So don't expect lvm support this month, sorry.
i do request this feature too :)
What is LVM : http://www.tldp.org/HOWTO/LVM-HOWTO/whatislvm.html
other docs : http://sourceware.org/lvm2/ http://www.suse.de/cgi-bin/print_page_www.pl?NPSPath=/webredesign/htdocs/en/whitepapers/lvm/lvm1.html http://www-128.ibm.com/developerworks/library/l-lvm2.html?n-l-4121 and http://www.redhat.com/magazine/009jul05/features/lvm2/ good luck plors :)))
there is a interesting graphical tool included in rhel4 and FC enabling to handle VolGroup and LogVol : system-config-lvm, unfortunately it is impossible to use it since you cannot umount / it would be nice to implement it under Gparted LiveCD
SuSE/Novell's YaST supports LVMs too. Here's a short and incomplete list, how to implement GParted's features using the lvm userpace-tools: VolumeGroups -------------------- detect vgscan / vgdisplay read ? create (create partition +) pvcreate + vgcreate grow vgextend shrink vgreduce move ? copy dd ? check ? (delete) vgremove (merge) vgmerge LogicalVolumes -------------------- detect lvscan / lvdisplay read ? create lvcreate + create FS grow lvextend + resize FS shrink resize FS + lvreduce move n/a copy dd ? check check FS (delete) lvremove Maybe this helps...
thanks, but what really should help is some kind of timemachine :)
This feature would be greatly appreciated by me, as well.
I would like to also make a feature request for LVM in Gparted. Also, I wonder why Gparted is included by default in Fedora 7 when the default file system is lvm... and furthermore if one installs from the F7 livecd, the user does not have the choice to change the file system.... i guess this is a question for the fedora maintainers :-)
(In reply to comment #16) > I would like to also make a feature request for LVM in Gparted. > > Also, I wonder why Gparted is included by default in Fedora 7 when the default > file system is lvm... and furthermore if one installs from the F7 livecd, the > user does not have the choice to change the file system.... i guess this is a > question for the fedora maintainers :-) > Yes I think you could mention it to FC7. I gonna mail them about that point right now :)
Any news on how support for lvm partitions is going? Anyway us mortal non coders can help out? :-)
Thank you to all that have posted here with tips and offers of help. :-) Detection of lvm2 physical volumes has been added to the Gnome SVN repository for inclusion in the next release of GParted (0.4.2). Please note that Logical Volume Management is not yet supported by GParted. I am still trying to get my head around how LVM will be graphically presented, and what is involved in re-engineering the core architecture of GParted to support LVM. If you have some ideas, I am open to suggestions.
Created attachment 124285 [details] Screenshot of possible LVM UI Attached a quick idea of a possible UI for LVM support - basically copied from the partitioning bit of Anaconda.
(In reply to comment #19) > I am still trying to get my head around how LVM will be graphically presented, > and what is involved in re-engineering the core architecture of GParted to > support LVM. If you have some ideas, I am open to suggestions. Have you looked at 'system-config-lvm' from Red Hat?
Created attachment 124551 [details] Screenshot of RH's LVM tool
Thank you for the ideas and screen shots Rob and Wil. The pictures are especially useful. I particularly like Rob's mock-up of GParted that displays the lvm logical volumes for a volume group. An extension to this idea would be to have volume groups listed in the device drop down list. Then when a person selects a volume group, the logical volumes could be displayed in the lower section with a graphic bar representing the usage within the volume group, and a list of all the logical volumes in the volume group. Physical disk drive and partition display would remain the same. I still have to figure out how to handle the different operations for lvm. Perhaps extra PV and/or LV menus would help out.
*** Bug 567018 has been marked as a duplicate of this bug. ***
SabayonLinux defaults to creating LVM partitions. Fixing partitions off a livecd would be easier if gparted supported LVM.
Even if gparted doesn't support the whole range of LVM operations, it would be very nice if it could at least detect logical volumes and allow the user to edit partitions on them as it currently does for normal block devices.
I would appreciate if the application "GParted" could let me increase the size of an existing partition (of the type "LVM2") on the operation level "fdisk". I could adapt my growing storage needs by the command "pvresize" manually until LVM support will really become convenient. A cooperation with the command "pvdisplay" would be nice to provide potential safety checks for further adjustments of physical volumes.
(In reply to comment #27) > I would appreciate if the application "GParted" could let me increase the size > of an existing partition (of the type "LVM2") on the operation level "fdisk". I > could adapt my growing storage needs by the command "pvresize" manually until > LVM support will really become convenient. > > A cooperation with the command "pvdisplay" would be nice to provide potential > safety checks for further adjustments of physical volumes. Hmm. I usually allocate partitions in 20 GiB blocks. If I need more space for LVM I create new 20 GiB partition, make it PV and add it to VG. Unless you mean shrinking PV. I guess that bigger problem is actual UI for creating LV on VG and adding PV to VG (especially if gparted is used in installers).
(In reply to comment #28) > If I need more space for LVM I create new 20 GiB partition, > make it PV and add it to VG. I do not want to clutter volume groups and make my storage allocation potentially confusing in this way. A partition resize is not supported by a command like "cfdisk" in one step so far. (A partition can only be separately deleted and created.) I hope that contiguous storage extensions will become easier by the corresponding tools.
Yes; providing an enlarge & shrink facility for LVM-physical volumes should be implemented before any fancy log. volume creation & management facilities. This is a basic and important feature. Log vol. management is already provided by many other tools. No immediate need to have a gparted support for it.
In order to support resizing LVM Physical Volumes, GParted would need ways to do the following: 1) Determine if an LVM PV is active or not This is needed because a partition must not be active when changing it's size. 2) De-activate an LVM-PV This is needed to de-activate an active LVM PV partition so that it might be operated upon. Does anyone know of a way to perform these actions, either through the command line, or through a library?
Depending on what you are doing, it is not always necessary to have a PV offline to change it. pvresize can be used to safely resize an existing and in-use PV, and a combination of creating a new PV and mirroring/moving existing LVs to it could be used to "copy" or "move" one. This may actually be a better idea than directly copying the partition, as LVM does not handle duplicate PV uuids very well. As far as detecting if a PV is in use...I don't know of a direct way. I can determine what PV(s) a given LV is on, and I can determine if an LV is in use (thought I think it would require some output parsing). If that could be useful, let me know and I'll write up the details.
Thanks Kevin for your response. The challenge I foresee is not with running the pvresize command, but rather in trying to change the partition table when the partition is active. That is why I am seeking ways to do the two things identified in comment #31.
(In reply to comment #31) > 1) Determine if an LVM PV is active or not > This is needed because a partition must not be active when changing > it's size. You can use vgdisplay -v which will give you a list of the active LVs for each VG (or the VG you specify) as well as the PVs which make up the VG. I don't think it's possible for only "some" LVs to be active in the same VG, which would mean that if any LV is active in a VG, you should be able to treat the entire VG as active and consequently all PVs used by that VG as active / in use as well. > 2) De-activate an LVM-PV > This is needed to de-activate an active LVM PV partition so that it might > be operated upon. If you deactive the all VGs which use the PVs of interest, then there should be nothing using these PVs and your goal should be achived. E.g. vgchange -an <vg-name> If an LV is mounted (in use) and you try to deactivate the PV, you will get an error which you can trap in your GUI and inform the user that either the PV can't be operated on or provide some instructions for how the user can remedy this invalid state (e.g. unmount the LVs). > Does anyone know of a way to perform these actions, either through the command > line, or through a library? A small bash-example to illustrate finding the active PVs and how you can hook deactivation logic to the discovery: find_values(){ gawk -vfield="$1" --re-interval -F ' {2,}' '$2 == field {print $3}' ;} value_present(){ gawk -vfield="$1" -vvalue="$2" --re-interval -F ' {2,}' '$2 == field && $3 == value {print "true"}' ;} find_pvs_for_vg(){ vgdisplay -v "$1" 2>/dev/null | find_values "PV Name" ;} is_vg_active(){ vgdisplay -v "$1" 2>/dev/null | value_present 'LV Status' 'available' ;} vgs=$(vgdisplay|find_values "VG Name") for vg in $vgs;do if [ ! -z "$(is_vg_active $vg)" ];then for pv in $(find_pvs_for_vg "$vg");do echo "PV $pv is active, deactivate with: vgchange -an $vg" done fi done
Created attachment 204810 [details] [review] Read-only LVM2 PV support patch Hi, Just to let everyone know that I am working on adding support for LVM2 Physical Volumes to GParted. This does not include support for the management of Volume Groups or Logical Volumes, hence is only a subset of full LVM2 support. PVs will be treated like any other file system in GParted. (Chose this subset as it fits into GParted's existing way of working without requiring any user interface or major internal changes, plus it will provide useful functionality for users managing disks with mixed file systems and LVM2 PVs). Status so far is that I have implemented read-only support for LVM2 PVs. It reports free space and busy status of PVs. Still outstanding is all the read-write features like: creation, toggle active status, resizing, "file system" checking and correctly removing PVs. The attached patch is just to show progress and if any one would like to test it. (It's merges 14 small changes to date). The patch applies on top of the latest GParted from git as of today. $ git describe GPARTED_0_11_0-7-ge52384b I do not yet consider this code ready or suitable for committing to GParted. To see/review all the individual commits have a look at my public repository on my home machine: https://rockover.homeip.net/cgit/gparted/ lvm2-devel branch. I use a self signed certificate so if you want to clone it you'll have to make git accept it: $ git config --global http.sslverify false $ git clone https://rockover.homeip.net/cgit/gparted/ If nothing else some of the commit messages could probably do with tidying up. Reviews, testing and comments welcome. Thanks, Mike
Hi Mike Good to see that lvm support is being added to GParted. It is certainly one of the major things that are still lacking. I had a look at the patch, and tested it. As you mention that you are still working on write support, I'll give some feedback on the readonly part. I have tested the patch on a test system with a number of disks. on which I created a few VGs distributed over a collection of PVs. In general, it gives a good view of the PVs. I like the idea of reusing the mountpoint information for the VG name. I have some minor remarks / suggestions, which you may already be aware of, or have thought of yourself: As one of the VGs on my test system is exported, I noticed that the patch currently does not distinguish between 'volume group inactive' and 'volume group exported'. It would be useful to have such a distinction. I would even consider 'exported' to be a more important state than the current 'inactive'. The 'vgchange -a n' and 'vgchange -a y' commands are currently invoked from Win_GParted.cc (not surprisingly, as swapon is too). Would it be an idea to add a (de)activate (and im/export - see above) function to the FileSystem class, and implement the operation there ? Although I haven't given the idea some deep thought, it seems a useful abstraction to me, and it keeps Win_GParted much cleaner ? A last thing, which pertains more to the write part, and which you may have noticed already, is that a volume group has to be active (actually: imported) for its physical volumes to be manipulated, and that currently, deactivating the volume group really only deactivates the *logical* volumes, and leaves the volume group itself very much active (which makes the deactivation redundant). I hope you find my remarks to be of use to you, and I look forward to seeing the patch complete, and integrated into GParted. Kind Regards, Rogier.
Thank you very much Mike for this initial patch. You have made a good starton on a key component that is missing from GParted. And your detection of active LVM partitions (partition busy) is very useful for other file systems too due to the interaction among logical partitions. And thank you Rogier for the code review too. I think you have some some good ideas worth considering. Mike, if you desired to finalize a "read-only" patch then I would suggest the following: - Add the LVM_PV_Info.h and lvm_pv.h to the include/Makefile.am - Add the LVM_PV.Invo.cc and lvm_pv.cc files are added to po/POTFILES.in - Ensure that trailing whitespace is removed from the patch - Ensure that 2012 is included for the copyright year on the files added. Great work on this initial patch!
(In reply to comment #36) Hi Rogier, > As one of the VGs on my test system is exported, I noticed that the patch > currently does not distinguish between 'volume group inactive' and 'volume > group exported'. It would be useful to have such a distinction. I would even > consider 'exported' to be a more important state than the current 'inactive'. I was previously putting Volume Group import/export as out of scope of my current support for PVs only. Gut reaction is that it seems useful and not too much work to add reporting and control of a VG's import/ export status similarly to how activate/deactivate is handled so I will add it to my to do list. > The 'vgchange -a n' and 'vgchange -a y' commands are currently invoked from > Win_GParted.cc (not surprisingly, as swapon is too). Would it be an idea to add > a (de)activate (and im/export - see above) function to the FileSystem class, > and implement the operation there ? Although I haven't given the idea some deep > thought, it seems a useful abstraction to me, and it keeps Win_GParted much > cleaner ? I can see that adding activate and deactivate methods into the lvm2_pv class is logical, especially if added to the parent FileSystem class and implemented for the linux_swap class too. However it doesn't simplify Win_GParted.cc as it still has to handle all the UI and translatable text differences. Since my activate / deactivate patch follows how swap is currently handled I don't plan on changing this. > A last thing, which pertains more to the write part, and which you may have > noticed already, is that a volume group has to be active (actually: imported) > for its physical volumes to be manipulated, ... [Short answer] GParted requires a partition to be unmounted and inactive before performing any operations on it. I am sticking with this for my implementation of LVM2 PV only support. Specifically the VG will have to be inactive before allowing GParted to move one of its PVs. Also PV resizing 'lvm pvresize --setphysicalvolumesize SIZE /dev/DEVICE' and PV checking 'lvm pvck /dev/DEVICE' can be performed with the VG active or not. [Long answer] Resolving the dichotomy between GParted only working on inactive partitions and one of LVMs key aspects being the ability to perform online administration was the most import question for me to answer in adding LVM2 PV only support to GParted. For me the points were: * GParted's current approach is to only operate on unmounted and inactive partitions. * LVM is all about being able to manage storage online and it would limit GParted's usefulness if it couldn't do the same. * GParted appears to be often used from recovery boot media meaning the partitions are already unmounted. * System-config-lvm was initially a Red Hat (and Fedora, CentOS etc) only tool for managing LVM2 online, but is now available for Debian and therefore presumably can be made available for other distributions. * GParted has the capability to move a partition where it's source and destination sectors overlap. This can only ever be done with a PV offline. This last point about moving a partition overlapping with itself, along with my need to limit the size of the task I take on at any point in time (it's only a free time hobby), means means that the LVM2 PV only support I am adding will have the PVs inactive for all supported operations, as GParted expects. After this full support for LVM VGs and LVs can be considered. This will require presenting new concepts to the user with significant UI changes, probably with the ability to perform a number of LVM operations online. > ... and that currently, deactivating > the volume group really only deactivates the *logical* volumes, and leaves the > volume group itself very much active (which makes the deactivation redundant). My understanding of LVM is a little different. After some investigation and experimentation I found that a VG being active is exactly the same as one or more of its LVs being active. Nothing in LVM or device mapper, using the dmsetup command, reported a VG as being active when it contained no LVs or none of them were active. -- Is this difference academic anyway? > I hope you find my remarks to be of use to you, and I look forward to seeing > the patch complete, and integrated into GParted. Thankyou for your review and testing. Mike
(In reply to comment #38) > [Short answer] > GParted requires a partition to be unmounted and inactive before > performing any operations on it. I am sticking with this for my > implementation of LVM2 PV only support. Specifically the VG will have > to be inactive before allowing GParted to move one of its PVs. Also PV > resizing 'lvm pvresize --setphysicalvolumesize SIZE /dev/DEVICE' and PV > checking 'lvm pvck /dev/DEVICE' can be performed with the VG active or > not. This short answer is right on the mark. Disk partition boundaries cannot be safely changed while the partition is in use. This is the main reason for GParted requiring partitions to be inactive/unmounted when changing partition boundary sizes. There is some ongoing work by Phillip Susi to enable the ability to change partition boundaries while the partition is active. See the following mailing list thread: Add partition resize function to BLKPG ioctl http://www.gossamer-threads.com/lists/engine?do=post_view_printable;post=1461595;list=linux I think that focusing on providing the ability to resize LVM Physical Volumes is where GParted can provide the biggest benefit for users. My reasoning is that resizing PV's also requires adjusting partition boundaries, which is not a trivial task to do from the command line.
Created attachment 206351 [details] [review] LVM2 PV read-only patch set (v2) Hi Curtis, Here is the patch set implementing LVM2 PV read-only support ready for submission. Addresses your 4 points. I hope you don't mind multiple patches, but I think that smaller logical pieces makes reviewing easier and along with the commit messages improves understanding between us and adds to the knowledge stored in the git history. As before apply with: git am gparted-lvm2-pv-read-only-v2.mbox Git am complains that patch "Report space usage of LVM2 PVs (#160787)" adds 2 lines of whitespace errors. I have looked really closely and I can't see any, only possibly a few in diff context. The commit dates are all over the place because some of the commits have been re-created to resolve conflicts with changes in the master branch, where as other have merely been cherry-picked, edited and squashed as I prepared the patch set for submission. Learnt a new git technique, squashing multiple commits into one using git rebase -i followed by pick, squash, squash, ... for each SHA1. Google "git rebase squash". Thanks, Mike
Hi Mike, (In reply to comment #38) > This last point about moving a partition overlapping with itself, along > with my need to limit the size of the task I take on at any point in > time (it's only a free time hobby), means means that the LVM2 PV only > support I am adding will have the PVs inactive for all supported > operations, as GParted expects. > > After this full support for LVM VGs and LVs can be considered. I totally agree, and I think I wouldn't have done it any differently. LV operations are definitely of lesser importance, and having support for them as well will not provide the kind of benefit to GParted users that the ability to resize and move PVs and the underlying partitions will. > > ... and that currently, deactivating > > the volume group really only deactivates the *logical* volumes, and leaves the > > volume group itself very much active (which makes the deactivation redundant). > My understanding of LVM is a little different. After some investigation > and experimentation I found that a VG being active is exactly the same > as one or more of its LVs being active. Nothing in LVM or device > mapper, using the dmsetup command, reported a VG as being active when it > contained no LVs or none of them were active. -- Is this difference > academic anyway? I did some more experimentation, and it seems indeed that even disk IO related to mirror or raid5 or raid6 synchronisation stops when an LV is inactive. Actually, the /dev/mapper device nodes are even removed, suggesting (implying?) that the kernel mapping no longer exists, which would preclude LVM-related disk IO to the underlying partitions (except for meta-data updates). So your understanding is correct. Sorry about that. However, I did manage to start and run a pvmove on a VG with no active LVs. In the process, the LV got re-activated. I also tried it on a raid5-LV, and only the one stripe (out of 4) that was moved ended up reactivated. So, apparently it is unsafe to expect LVM to leave LVs inactive after LVM operations that require data movement (although that may be considered a bug). Obviously, that is not something a 'vgchange -a y' followed by a 'vgchange -a n' won't fix (just a 'vgchange -a n' didn't work for my raid LV). A complication, and the reason why I first made the comment, is that currently, once a partition's VG is inactive, GParted assumes that the partition is free to be repurposed, and allows it to be reformatted or deleted. At that time though, a partition is still part of the VG, and a reformat or deletion damages the VG, and destroys part of the data that the user (probably) believes to be safe on the other PVs that are still in the VG. Rogier.
(In reply to comment #40) Hi Mike, > Git am complains that patch "Report space usage of LVM2 PVs (#160787)" > adds 2 lines of whitespace errors. I have looked really closely and > I can't see any, only possibly a few in diff context. In case it is of any use: I have found 'git diff --color' to be quite helpful at locating whitespace errors. I have also looked at your latest patch. Apart from the issue mentioned in the last paragraph of my previous post, I found that a volume group name containing a '-' confuses the code, and the VG is reported to be inactive, even when it isn't. I assume that having a second VG, with an identical name up to the first '-' would have some interesting effects as well. Kind Regards, Rogier.
Hi Rogier, (In reply to comment #41) > So, apparently it is unsafe to expect LVM to leave LVs inactive after LVM > operations that require data movement (although that may be considered a > bug). Obviously, that is not something a 'vgchange -a y' followed by a > 'vgchange -a n' won't fix (just a 'vgchange -a n' didn't work for my raid LV). I don't see this as an issue we have to handle. If someone uses GParted to manipulate an inactive LVM PV and at the same time initiates a pvmove via LVM directly, which activates the VG, they are responsible for their own corruption. > A complication, and the reason why I first made the comment, is that > currently, once a partition's VG is inactive, GParted assumes that the > partition is free to be repurposed, and allows it to be reformatted > or deleted. At that time though, a partition is still part of the VG, > and a reformat or deletion damages the VG, and destroys part of the data > that the user (probably) believes to be safe on the other PVs that are > still in the VG. One of the items I have for read-write support is to handle deletion of file systems, probably as a new FileSystem specific method. I think this is needed because: 1) each PV stores a full copy of the LVM metadata, including all the PVs, and 'lvm pvremove /dev/DEVICE' must be run to cleanly remove the PV rather than just allowing creation of a new file system over the top to effectively corrupt LVM setup, even in the PV is free; 2) I have seen a couple of combinations of mkfs and pvcreate resulting in 2 signatures in the partition and parted and blkid report different contents. As long as the pvremove is not forced it doesn't succeed, even when it contains an empty VG. So as long I make GParted call this for Delete and Format to it will at least fail and report an error to the user. (In reply to comment #42) > Apart from the issue mentioned in the last paragraph of my previous post, I > found that a volume group name containing a '-' confuses the code, and the VG > is reported to be inactive, even when it isn't. I assume that having a second > VG, with an identical name up to the first '-' would have some interesting > effects as well. I will have a closer look when I have some free time in a day or 2. Thank you for your ongoing testing, Mike
(In reply to comment #42) > Apart from the issue mentioned in the last paragraph of my previous post, I > found that a volume group name containing a '-' confuses the code, and the VG > is reported to be inactive, even when it isn't. I assume that having a second > VG, with an identical name up to the first '-' would have some interesting > effects as well. What is happening is that LVM (and libdevmapper) are constructing the name to given to the kernel device-mapper drive as 'VGNAME-LVNAME', but additionally it is encoding hyphens (-) in the VGNAME and LVNAME as double hyphens (--), so VG-NAME and LV-NAME becomes 'VG--NAME-LV--NAME' as the name given to the kernel device-mapper driver. The LVM commands perform this translation when necessary but using dmsetup to query the kernel directly just shows the raw name with double hyphen encoding. At the moment I would prefer to find an LVM command which can display the active status of a VG or (combined LV and VG name), in preference to adding this double dash encoding knowledge to GParted, but I guess it depends on if there is such an LVM command and how easy its output is to parse. Mike
Created attachment 206634 [details] [review] LVM2 PV fix hyphens patch Here's a patch to fix GParted detecting active status of VGs with hyphens in the name. Applies on top of existing "LVM2 PV read-only patch set (v2)" (gparted-lvm2-pv-read-only-v2.mbox).
Thank you both Mike and Rogier for your work on LVM. Mike, thanks to your tip about editing midstream git commits, I have located and fixed the trailing whitespace in patch 2/8. Following is a link on how to display trailing whitespace in emacs: Life Is Too Short For Bad Code http://trey-jackson.blogspot.com/2008/03/emacs-tip-12-show-trailing-whitespace.html I still need to do some more testing, but so far this patch set looks good. If all goes well I will commit these patches for inclusion in the next release of GParted (tentative release planned for mid-Feb). And I do prefer to have the history involved in all the commits as opposed to one big commit. Rogier, if you wish to do this too, then if you have all of your commits on a branch separate from "master", you can generate the complete commit set in mbox format with the following command (assuming you are on your separate branch): git format-patch origin/master --stdout > mypatchset.mbox Hope that helps. I've been learning more and more with git and am amazed at what a person can do. And I think I've only scratched the surface too. :-)
Mike, one quick question. In looking at Utils::tokenize() and Utils::split(), these two methods appear to be doing essentially the same thing. Perhaps one handles an edge case better than another? Am I missing something?
(In reply to comment #47) > Mike, one quick question. > > In looking at Utils::tokenize() and Utils::split(), these two methods appear to > be doing essentially the same thing. > > Perhaps one handles an edge case better than another? > > Am I missing something? Utils::tokenize() is about extracting a list of tokens from any number of background separator characters. E.g. tokenize(str=" word1 word2 ", tokens, delimiters=" ") -> tokens=["word1","word2] where as Utils::split() splits on every separator character. E.g. split(str=",,word1,,word2,,", result, delimiters=",") -> result=["","","word1","","word2","",""] I needed both different behaviours. Perhaps tokenize() needs its comment improving a little.
Thank you for the clarification Mike. I agree that the comment for tokenize() does need improving, and I shall do so with an example like you outlined.
The patches to provide read-only support for LVM PVs have been committed to the Gnome git repository for inclusion in the next release of GParted. The series of relevant git commits can be viewed at the following links: Report space usage of LVM2 PVs (#160787) http://git.gnome.org/browse/gparted/commit/?id=abd0dd04268c873f101c5850f16399d7e5b903b4 Allow unused space in a partition to equal its size http://git.gnome.org/browse/gparted/commit/?id=0acb6237255435013c65777d81609df9bac24b79 Cache results from querying all LVM2 PVs (#160787) http://git.gnome.org/browse/gparted/commit/?id=608d992e822a6ea0a8b5612c134ab76ca5a394d9 Lazy initialize the cache from querying LVM2 PVs (#160787) http://git.gnome.org/browse/gparted/commit/?id=ff8ad0412057dd6644cd290a8affc71786b5d89f Display LVM2 VGNAME as the PV's mount point (#160787) http://git.gnome.org/browse/gparted/commit/?id=8083f11d84dbd4f186271a3cdbf5170db259f8b8 Display busy status of LVM2 PVs (#160787) http://git.gnome.org/browse/gparted/commit/?id=aa085a3caab45554fb88af9dec3c93334f11df54 Show LVM2 PVs as supported, read-only (#160787) http://git.gnome.org/browse/gparted/commit/?id=72ac712024e65597319faa65b07001f2f6d263b1 Switch to using lvs to identify active LVM LVs (#160787) http://git.gnome.org/browse/gparted/commit/?id=820242635a9e02ce8af67854dd15cb9b32af485f
Created attachment 206996 [details] [review] Two minor fixes relative to 51650ed Hi Mike / Curtis, I have tested the latest git version of GParted, and I have found a few opportunities for improvement. The first two are just minor things I happened to notice, and which probably nobody would care about if left as is. In case you want to change them, I attached two patches. 1) A minor cosmetic thing: The README still mentions dmsetup is required for LVM. It no longer is. 2) Also a minor thing: Currently, the test for a busy partition is: partition_is_busy = ped_partition_is_busy( lp_partition ) || lvm2_pv_info .has_active_lvs( partition_path ) ; This means the LVM cache is consulted for each and every partition on the system. This can be optimised by checking the cache for known LVM partitions only. The LVM cache is currently also consulted for extended partitions, which would normally not contain anything but logical partitions. 3) detection of busy lvm partitions fails when a volume group is exported. The result is that *all* PVs on the system are marked as not-busy. The cause is that 'lvm lvs' exits with code 5 in that case. An alternative might be to use 'lvm pvs' instead, which (AFAICT) does not exit with non-zero exit code. It also has the added benefit that it reports the contents of exported VGs as well. 4) GParted allows a physical volume to be deleted or reformatted while the physical volume is part of a volume group. This is what I also mentioned in an earlier post. I mention it again, as I think this is a serious issue. The volume group may contain important data, distributed over several PVs. Removal of one of the PVs will not only destroy the data, it contains but may in fact damage all LVs of the VG, and make the entire contents of the VG unusable. That is most likely not the user's intention. Even if it is the user's intention, this kind of operation belongs with read-write LVM support. IMO, read-only LVM support should *prevent* this kind of damage. Also, after deletion of the PV, the LVM commands will start complaining that they can't find the PV, and manual repair of the VG metadata is needed. In fact, from LVM point of view, deletion of a PV that is part of a VG, is comparable to a failed disk. That is not a situation GParted should knowingly allow to be created. In the case of LVM, I think it should not be possible to delete or reformat a PV if it is part of a VG. My suggestion would be to mark lvm physical volumes as not-busy only if they are *not* part of a VG, and to mark them as busy if they are. This ensures that only PVs that are not used (not just 'not active') can be deleted and reformatted. It also fits well with the idea that a PV that is in a VG is considered 'mounted' on the VG. For read-write support, a vgreduce (iow: 'unmount') would be needed on a PV before allowing the partition to be deleted or reformatted. Let me know what you think. Kind Regards, Rogier.
Hi Mike, While testing, I have been thinking about the issues involved in adding LVM write support, and it occurred to me that the ability for GParted to perform certain operations on partitions that are busy might be of great use to you. If you agree that it would facilitate your implementation of LVM write support, and if you would like to have the feature, but are not planning to add it yourself, I'd like to propose that I look into it. If there are no major roadblocks, I would provide you / Curtis with a patch. Let me know if you're interested. Kind Regards, Rogier
Hi Rogier, (In reply to comment #51) > Created an attachment (id=206996) [details] [review] > Two minor fixes relative to 51650ed > > Hi Mike / Curtis, > > I have tested the latest git version of GParted, and I have found a few > opportunities for improvement. The first two are just minor things I > happened to notice, and which probably nobody would care about if left > as is. > In case you want to change them, I attached two patches. > > 1) A minor cosmetic thing: The README still mentions dmsetup is required > for LVM. It no longer is. Still need to keep the line "And device-mapper support in the kernel." Each active LV is actually an active map loaded into the kernel's device-mapper driver. > 2) Also a minor thing: Currently, the test for a busy partition is: > partition_is_busy = ped_partition_is_busy( lp_partition ) || > lvm2_pv_info .has_active_lvs( partition_path ) ; > This means the LVM cache is consulted for each and every partition on > the system. This can be optimised by checking the cache for known LVM > partitions only. > > The LVM cache is currently also consulted for extended partitions, > which would normally not contain anything but logical partitions. Reviewed and tested. Passed by me. > 3) detection of busy lvm partitions fails when a volume group is > exported. The result is that *all* PVs on the system are marked as > not-busy. The cause is that 'lvm lvs' exits with code 5 in that case. > An alternative might be to use 'lvm pvs' instead, which (AFAICT) does > not exit with non-zero exit code. It also has the added benefit that it > reports the contents of exported VGs as well. Nice catch. Definitely needs fixing. Note that 'lvm pvs' is not a replacement for 'lvm lvs' because the pv_attributes don't include an active flag where as the lv_attributes do. As well as fixing this bug, really need the lvm2 class to be able to report these errors encountered in the LVM2_PV_Info cache back to the users. I'll have a look at these over the next few days. > 4) GParted allows a physical volume to be deleted or reformatted while > the physical volume is part of a volume group. > > This is what I also mentioned in an earlier post. I mention it again, > as I think this is a serious issue. The volume group may contain important > data, distributed over several PVs. Removal of one of the PVs will not only > destroy the data, it contains but may in fact damage all LVs of the VG, > and make the entire contents of the VG unusable. That is most likely > not the user's intention. Even if it is the user's intention, this kind > of operation belongs with read-write LVM support. IMO, read-only LVM > support should *prevent* this kind of damage. > > Also, after deletion of the PV, the LVM commands will start complaining > that they can't find the PV, and manual repair of the VG metadata is > needed. In fact, from LVM point of view, deletion of a PV that is part > of a VG, is comparable to a failed disk. That is not a situation GParted > should knowingly allow to be created. > > In the case of LVM, I think it should not be possible to delete or > reformat a PV if it is part of a VG. My suggestion would be to mark > lvm physical volumes as not-busy only if they are *not* part of a VG, > and to mark them as busy if they are. This ensures that only PVs that > are not used (not just 'not active') can be deleted and reformatted. It > also fits well with the idea that a PV that is in a VG is considered > 'mounted' on the VG. > > For read-write support, a vgreduce (iow: 'unmount') would be needed on a > PV before allowing the partition to be deleted or reformatted. Even before any LVM patches were added, GParted has always allowed a user to delete or format a partition containing an LVM PV, or any other type of file system. So the question is whether extra measures are needed beyond the existing GParted warnings now LVM PVs are supported read-only? I don't think so. Curtis is the final arbiter though. Yes, just deleting an LVM PV is comparable to disk failure and corrupts the LVM configuration. I think that my proposal for partition deletion handles this case, as outlined in comment #43 above, but obviously it isn't implemented yet for read-only support. Marking PVs as busy based on whether they are part of a VG is not a long term solution, because this would prevent GParted moving or resizing any PV that wasn't free. This removes one of GParted's key benefits for PVs, the ability to resize and move PVs containing data. I can see having a partition menu option to evacuate the data from a PV using vgreduce as being a useful independent action. Thank you for your ongoing testing, patches and review, Mike
(In reply to comment #53) > (In reply to comment #51) >> 1) A minor cosmetic thing: The README still mentions dmsetup is required >> for LVM. It no longer is. > >Still need to keep the line "And device-mapper support in the kernel." >Each active LV is actually an active map loaded into the kernel's >device-mapper driver. Right. Sorry for overlooking that. > > 3) detection of busy lvm partitions fails when a volume group is > > exported. The result is that *all* PVs on the system are marked as > > not-busy. The cause is that 'lvm lvs' exits with code 5 in that case. > > An alternative might be to use 'lvm pvs' instead, which (AFAICT) does > > not exit with non-zero exit code. It also has the added benefit that it > > reports the contents of exported VGs as well. > > Nice catch. Definitely needs fixing. Note that 'lvm pvs' is not a > replacement for 'lvm lvs' because the pv_attributes don't include an > active flag where as the lv_attributes do. You can use lv_ (and vg_) fields with pvs. E.g.: lvm pvs -o pv_name,pv_attr,vg_name,vg_attr,lv_name,lv_attr There are some limitations, but I don't think they affect your needs. > Marking PVs as busy based on whether they are part of a VG is not a long > term solution, because this would prevent GParted moving or resizing any > PV that wasn't free. This removes one of GParted's key benefits for > PVs, the ability to resize and move PVs containing data. Agree. I think both options are not optimal, and kind of break LVM support in some way. Hence my offer to look into implementing the basic infrastructure to perform operations on busy partitions in the short term, so that you can use it in the implementation of LVM write support. IMO, that would solve both issues at once (or rather, solve one without introducing the other) ? Kind Regards, Rogier.
Thank you both Mike and Rogier for you continued work on LVM. The patches in comment #51, with some minor editing to keep the one sentence, have been committed to the Gnome git repository for inclusion in the next release of GParted. The relevant git commits can be viewed at the following links: README file: dmsetup is not required for lvm http://git.gnome.org/browse/gparted/commit/?id=ed51f4b28fff4c0b4f25e9e2534b527c417155fa Check LVM cache only for LVM physical volumes http://git.gnome.org/browse/gparted/commit/?id=50befd62a321aaac8ae85778ba05f31216e7e696 (In reply to comment #53) > (In reply to comment #51) > > 4) GParted allows a physical volume to be deleted or reformatted while > > the physical volume is part of a volume group. > > > Even before any LVM patches were added, GParted has always allowed a > user to delete or format a partition containing an LVM PV, or any other > type of file system. So the question is whether extra measures are > needed beyond the existing GParted warnings now LVM PVs are supported > read-only? I don't think so. Curtis is the final arbiter though. With GParted we try to empower the user to have complete control over their disk devices. As such if GParted does not recognize the format of a partition on the disk, then we at least permit the user to delete the partition. The work to add read-only support for LVM is a step forward towards properly handling LVM2 PVs. Since GParted does not yet provide any other way to remove a partition from a volume group, I think it is good to keep the delete option enabled. When further LVM2 PV knowledge and support is built into GParted, such as removing a partition from an LVM volume group, and resizing PVs, then we can try to provide a safer way to delete/remove partitions that are part of volume groups. In the meantime I think GParted has to provide some method to delete any partition that is on a disk device.
Created attachment 207220 [details] [review] LVM2 PV fix exported VGs patch set Here's a 2 patch set which does the following: 1) Fix detection of LVM2 PV busy status when exported VGs exist 2) Display any errors from querying LVM2 PVs to the user Thanks to Rogier for making me look closer at "lvm pvs" to get LV attrs. Turned out to be a neat solution. Testing and reviews welcome. Thanks, Mike
(In reply to comment #56) > Here's a 2 patch set which does the following: > 1) Fix detection of LVM2 PV busy status when exported VGs exist Works fine for me now ! > 2) Display any errors from querying LVM2 PVs to the user Works fine too. > Testing and reviews welcome. When trying to get lvm to report some errors, I deleted a PV, and GParted crashed :-(. It seems to be caused by the fact that the pvs output is split on one or more of ' ' and/or '\n': Utils::tokenize( output, lvm2_pv_cache, " \n" ) ; However, when a PV is missing, the output contains lines like: unknown device,37748736,vg00,raid0-01,-wi--- These turn into two cache entries, the first of which is invalid: unknown device,37748736,vg00,raid0-01,-wi--- which results in a SEGV when GParted searches the cache. Kind regards, Rogier.
Good catch on this problem Rogier. Mike from looking at the code I'm guessing that you used tokenize() to perform two functions: a) To get rid of the leading whitespace b) To break the output into separate strings based on '\n' If this is the case, then you might consider using Utils::tokenize('\n') to break the output into separate strings. Then before you use the individual strings, use Utils::trim() to remove the leading (and trailing) whitespace. 'Just a thought. :-) For now I will hold off on the new release announcement until we address this problem. After resolution I will create a beta tar ball and announce an upcoming release. If you are curious, the list of bugs and enhancements in the next release can be found in the Development Forum posting. Development Plans for the Next Release of GParted (0.12.0? or 1.0.0?) http://gparted-forum.surf4.info/viewtopic.php?id=16231 If you have some thoughts for the next release I would like to hear these. The General Development Forum is a good place for this discussion.
Created attachment 207384 [details] [review] LVM2 PV fix exported VGs patch set (v2) (In reply to comment #57) > When trying to get lvm to report some errors, I deleted a PV, > and GParted crashed :-(. Now it's a 3 patch set. 1) Fix detection of LVM2 PV busy status when exported VGs exist (#160787) 2) Display any errors from querying LVM2 PVs to the user (#160787) 3) Prevent crash in the LVM2 PV information cache (#160787) First 2 patches are the same as per comment #56, except for adding bug number to the comments. 3rd patch is to fix the crash found by Rogier. (In reply to comment #58) > If this is the case, then you might consider using Utils::tokenize('\n') to > break the output into separate strings. Then before you use the individual > strings, use Utils::trim() to remove the leading (and trailing) whitespace. Great minds think alike ;-) Thanks Mike
Created attachment 207393 [details] [review] VG export status & damaged warning patch set Heres 2 more patches which do the following: 1) Display VG export status with an LVM2 PVs busy status (#160787) 2) Warn when an LVM2 PV is a member of a damaged VG (#160878) as in the case Rogier identified causing the crash before. Need to stop trying to squeeze more patches in before the next GParted release cutoff ;-) Mike
The 3 patches in comment #59, and the 2 patches in comment #60 have been committed to the Gnome git repository for inclusion in the next release of GParted (0.12.0). Two minor changes to the text have been made: a) Changed "may" to "might" in LVM2_PV_Info::load_lvm2_pv_info_cache() "Some or all of the details may be missing or incorrect." The word "may" implies permission, whereas "might" does not have the same connotation of permission. I remember reading about this somewhere but can't seem to find the link. b) Changed "are" to "is" in LVM2_PV_Info::get_error_messages() This is a grammar fix since the primary noun is singular. This is a bit picky on my part but my English teacher did drill this into me. ;-) "One or more Physical Volumes belonging to the Volume Group are missing." breaks down to noun-verb: One are ...missing. One is ...missing. is the correct match. The relevant git commits can be viewed at the following links: Fix detection of LVM2 PV busy status when exported VGs exist (#160787) http://git.gnome.org/browse/gparted/commit/?id=4b30a2ddbfd8837af5663ec6a85992f73381b163 Display any errors from querying LVM2 PVs to the user (#160787) http://git.gnome.org/browse/gparted/commit/?id=c24170927d545d9f3cde4ed2d98dfc122505920d Prevent crash in the LVM2 PV information cache (#160787) http://git.gnome.org/browse/gparted/commit/?id=3fbdb8055f449d53834ed4a08b0dadd43b88f07c Display VG export status with an LVM2 PVs busy status (#160787) http://git.gnome.org/browse/gparted/commit/?id=6d665d669db14c1fce8db7e924bb903e1fd79b4f Warn when an LVM2 PV is a member of a damaged VG (#160878) http://git.gnome.org/browse/gparted/commit/?id=ea1c0c023e4a9c10605a74c7e756453c0e2e16fd With regards to this bug report, what do you think of renaming this one to "Add LVM read-only Support"? Then a new bug report could be created called something like "Add LVM PV resize support"
(In reply to comment #61) > With regards to this bug report, what do you think of renaming this one to "Add > LVM read-only Support"? > Then a new bug report could be created called something like "Add LVM PV resize > support" How about: Add LVM PV read-only support Add LVM PV read-write support (As well as adding PV resizing I will also be adding: PV creation, PV checking, VG activation/deactivation and the big one general file system wiping to implement PV specific removal).
Second thoughts, will it be a little confusing with every patch footer saying: "Bug #160787 - lvm support" but having renamed this bug to "Bug #160787 - Add LVM PV read-only support"? Just have the last entry say this bug only added LVM2 PV read-only support. Ongoing development happening under new "Bug ####### - Add LVM PV read-write support".
Thanks for your feedback Mike. I agree with going ahead with the suggestions in comment #63. Please feel free to open a new bug report for "Bug ####### - Add LVM PV read-write support". I will include a link to the new bug report when I close this report after the actual GParted 0.12.0 release on Feb 21, 2012.
The enhancement to add read-only support for LVM Physical Volumes has been included in GParted 0.12.0 released on February 21, 2012. A new bug report has been created to continue with further development of LVM PV support. See: Bug #670171 - Add LVM PV read-write support