GNOME Bugzilla – Bug 731036
Plenty of temporary files/directories are not removed
Last modified: 2014-06-06 12:04:53 UTC
This is exacerbated on osx/ios btw During the various phases (building recipes and doing packaging), plenty of temporary files and directories are created that are never cleaned up. git grep tempfile shows these modules creating temporary files: bootstrap/osx.py bootstrap/windows.py build/oven.py build/recipe.py ide/xcode/fwlib.py packages/debian.py packages/osx/buildtools.py packages/osx/bundles.py packages/osx/packager.py packages/rpm.py packages/wix_packager.py tools/osxuniversalgenerator.py utils/shell.py Let's make temporary files/directories exactly that, temporary. And avoid spamming computers with sometimes (on osx) several hundred megabytes of leaked files for each packaging
Created attachment 277614 [details] [review] cerbero: Remove more temporary directories
Review of attachment 277614 [details] [review]: ::: cerbero/ide/xcode/fwlib.py @@ +194,3 @@ shell.call('ar -cqS %s %s-%s' % (libname, libprefix, obj_f), tmpdir_thinarch) object_files_md5.append(md5) + shutil.rmtree(lib_tmpdir) In this same function tmpdir should also be deleted. It's created in: 162 tmpdir = tempfile.mkdtemp() ::: cerbero/packages/osx/packager.py @@ +306,3 @@ pb = ProductBuild() pb.create_package(distro_path, output_file, [self.package.relative_path('.')]) + shutil.rmtree(distro_path) distro_path should be os.path.join(self.tmp, "Distrobition.xml") so that's it's deleted with the temporary directory if keep_temp = False
Created attachment 277798 [details] [review] cerbero: close temporary file to delete them
Created attachment 277799 [details] [review] cerbero: close temporary file to delete them
(In reply to comment #0) > build/oven.py > build/recipe.py > tools/osxuniversalgenerator.py > utils/shell.py > bootstrap/osx.py NamedTemporaryFile not closed properly (fixed in previous patch) > packages/rpm.py > packages/debian.py Already deleted in packages/linux.py if keep_temp = True > packages/wix_packager.py > packages/osx/bundles.py Already deleted correctly > packages/osx/buildtools.py Unused code > packages/osx/packager.py > ide/xcode/fwlib.py Deleted correctly with your patch > cerbero/packages/osx/bundles.py Needs to be more review > bootstrap/windows.py Delete with new patch
Created attachment 277800 [details] [review] bootstrap: remove temporary file correctly
Created attachment 277801 [details] [review] osx: remove unused code
Review of attachment 277614 [details] [review]: ::: cerbero/ide/xcode/fwlib.py @@ +194,3 @@ shell.call('ar -cqS %s %s-%s' % (libname, libprefix, obj_f), tmpdir_thinarch) object_files_md5.append(md5) + shutil.rmtree(lib_tmpdir) we can't, that's the target of the final calls (either lipo or cp). When/how does *that* directory get used/cleaned up ?
This directory is created and used here: 181 if len(archs) > 1: #should be a fat file, split only to the arch we want 182 libprefix += '_%s_' % thin_arch 183 lib_tmpdir = self._split_static_lib(lib, thin_arch) 184 else: 185 lib_tmpdir = self._split_static_lib(lib) 186 187 obj_files = shell.ls_files(['*.o'], lib_tmpdir) 188 for obj_f in obj_files: It's taking a static library, and extracting the object files for the given architecture and adding it to the archive if they were not added previously (based on the md5) This step is done for each architecture, storing the results in tmpdir_thinarch After having created the archives for all the architecture, a fat library is created with lipo and tmpdir can be deleted afterwards
Pushed with the initial patch fixed. Thanks ! commit 6c096a855cd055b43baf7a676879afea631ca6ac Author: Andoni Morales Alastruey <ylatuya@gmail.com> Date: Tue Jun 3 16:11:01 2014 +0200 osx: remove unused code https://bugzilla.gnome.org/show_bug.cgi?id=731036 commit 43f515509d6cf627f515c98e70b9b3b6d1ea979b Author: Andoni Morales Alastruey <ylatuya@gmail.com> Date: Tue Jun 3 16:22:30 2014 +0200 bootstrap: remove temporary file correctly https://bugzilla.gnome.org/show_bug.cgi?id=731036 commit 773654e7732850642b779d7665f5259d4ca0986d Author: Andoni Morales Alastruey <ylatuya@gmail.com> Date: Tue Jun 3 16:01:06 2014 +0200 cerbero: close temporary file to delete them https://bugzilla.gnome.org/show_bug.cgi?id=731036 commit 4b89b87c933a9cddf73cfc1af11853698a5a96e2 Author: Edward Hervey <edward@collabora.com> Date: Sat May 31 11:53:05 2014 +0200 osx: Avoid leaking temporary file/directories Remote temporary directories, or use existing temporary directory to store temporary files https://bugzilla.gnome.org/show_bug.cgi?id=731036