GNOME Bugzilla – Bug 726947
Simplify command line batch processing
Last modified: 2016-10-09 22:07:50 UTC
Created attachment 272718 [details] [review] Patch to simplify batch processing The attached patch adds a macro to Script-fu that simplifies invocations of GIMP to perform batch processing. The macro is named 'with-files' and allows executing a command line such as: gimp -i -b '(with-files "*.png" <body>)' or on Windows: gimp -i -b "(with-files \"*.png\" <body>)" where <body> is the code that handles whatever processing you want to perform on the files. There are three variables that are available within the <body>: 'basename', 'image', and 'layer'. The 'basename' is the name of the file with its extension removed, while the other two variables are self-explanatory. You basically write your code as though it were processing a single image and the 'with-files' macro applies it to each of the files matching the pattern. While it is currently possible to perform identical batch operations from command line invocations of GIMP, it typically requires several lines of boilerplate code to find the files matching the pattern and loop through them, opening each of the files in turn, and then deleting each image when completed. This patch eliminates the need for this boilerplate code and reduces the complexity of batch processing to a more manageable level. NOTES Only a single pattern can be passed to the macro (as its first argument). Handling multiple filename patterns would require a more complicated syntax definition (that might need to be adjusted for different platforms). The original filename is not exposed directly within the <body>; if needed, it can readily be retrieved by using the 'gimp-image-get-filename' PDB procedure. This macro deviates a bit from the conventional Scheme approach of defining 'with-*' operations. Normally, a Scheme 'with-*' procedure would expect a lambda for its second argument. For example: (with-input <item> (lambda () <body>)). Since the 'with-files' macro is intended mainly for usage "external" to Script-fu (e.g., a BASH shell), It seemed reasonable to disregard normal convention in favor of greater simplification and understandability (the reason for introducing the macro). The macro is not registered with the PDB and therefore does not need the "script-fu-" namespace prefix. Including such a prefix would only serve to make the macro less easy to use. The macro is not intended as a substitute for traditional batch invocation of GIMP. It offers a simplified approach that should meet the needs of the bulk of such processing, but it does not cover all expected usages. In situations where more complicated operations need to be specified, the original functionality is still available.
Wow, nice :) I judge the elegance of the code, but shouldn't it be named "foreach-file"? "with-files" doesn't say much IMO.
Err, I *can't* judge...
I don't have a problem with 'foreach-files' being substituted. One minor argument against it, however, is that Script-fu uses the name 'for-each' for its mapping procedure and there may be a small degree of uncertainty about the inconsistency in hyphenation used. Despite this, I personally would still prefer 'foreach-file' to 'for-each-file'; though my strongest preference is for 'with-files'.
Actually, it doesn't really matter and I don't overly care :) I'll leave that to you and Kevin who maintains script-fu. Setting milestone to 2.10 as a reminder, enhancement bugs easily get lost otherwise...
I will start by saying I support the idea of adding something to Script-Fu that will make it easier for users to do batch processing. We just have to decide on the best way to provide some feature(s) to support that goal. The macro uses features of Scheme I don't use as I haven't written macros. As I think about this issue several thoughts and questions come to mind. 1. Would it be better to use a procedure instead of a macro? Which might be faster? 2. I would suggesting using a a name of for-each-file. It is consistent with 'for-each' and in keeping with foreach functions in other languages (e.g. foreach name in names). 3. To have body on command line can make command line complicated. Have it call a procedure, or do we need one version that accepts a body and another that calls a procedure? 4. I could see a use case where it would be useful to provide both basename and extension. 5. The general purpose strbreakup and unbreakupstr function are written in Scheme. The code could be made a little faster if new functions were added to ftx to extract the file name and extension. 1G. Discussion of this topic can be continued via via IRC or the developer mailing list.
I will briefly respond to Kevin's first point and carry the discussion on the remainder of the issues to the developer mailing list. The macro will be expanded once to produce a procedure that performs the iteration. The expansion takes about 200-300 milliseconds on a 2GHz machine. This delay is likely insignificant relative to the load time for the GIMP instance, and it only occurs once. Converting the 'with-files' functionality to a procedure would require that the <body> be placed within a 'lambda'. While not particularly onerous, many Script-fu authors have gone years without ever learning about lambda (and just the mere mention of it has been known to cause some Pythonistas' eyes to glaze over).
Do we really have to discuss the performance of expanding a scheme macro when the body of the iteration is image manipulation? Can we get this in for 2.10 please, in whatever form? This is badly missing and one of the most asked questions on IRC ("how do I process multilple files").
1 year minus 1 day later: could we get this into master? :-)
I think so.
Everybody agrees here that it is a useful macro. So I reviewed this, tested it and can say it works. I only did very minor trailing whitespace fixes to the commit before pushing it (I'm very sorry, I completely forgot to amend the commit message to make it more in line with our usual commit formatting!): commit b6611b94ad036b25fad7f5aaa10778d8056aa55d Author: saul <saul@crazyauntgail.com> Date: Sun Mar 23 20:36:06 2014 -0400 Simplify command line processing by adding a 'with-files' macro. Some comments: (1) I have no idea about naming (foreach-files vs with-files) so I let it as-is. If anyone wants to change this and comes up with good arguments, we can still do it. I personally don't care. (2) I removed the call to gimp-quit at the end of the loop. The macro should not quit GIMP. The user may still have processing to do: commit ce2e187b08750d6e6deefd7c889572ee22432b38 Author: Jehan <jehan@girinstud.io> Date: Mon May 30 18:55:37 2016 +0200 plug-ins: script-fu bash processing should not finish with gimp-quit. This is not the macro business to quit GIMP. The user may still have processing to do after looping over images. (3) About point 4. of Kevin in comment 5: > 4. I could see a use case where it would be useful to provide both basename and extension. I agree that a direct access to the filename being processed is useful (well Kevin was asking for the extension, but this is easily obtained from the filename). So I added the access to this variable: commit 4d23939c3dff241bd87e80b2e277e2742a7d7233 Author: Jehan <jehan@girinstud.io> Date: Mon May 30 19:20:50 2016 +0200 plug-ins: make "filename" accessible to the loop of "with-files" macro. The filename of the current bash file being processed is a pretty useful data to access in a script. (4) About optimization discussion, same as Mitch (comment 7), I feel like this is not really the biggest of our worry here. Yet now that the feature is in GIMP, everyone is free to commit better implementations! (5) Finally about the proposition to have an alternative macro which accepts a procedure instead of a body. Why not! Patches welcome. In any cases, I see no good reasons to block the merge of this new macro when everyone is saying that it's welcome. Improvements, optimizations and alternative versions welcome. But for now let's push this and avoid bitrot. :-) Should we close or do someone still want to improve things before GIMP 2.10, then?
Review of attachment 272718 [details] [review]: Committed as: commit b6611b94ad036b25fad7f5aaa10778d8056aa55d Author: saul <saul@crazyauntgail.com> Date: Sun Mar 23 20:36:06 2014 -0400 Simplify command line processing by adding a 'with-files' macro.
By the way, I left it opened because if anyone wants to change something, either in the name, or in the way it works, or whatever, this is the time to do it and propose something! After GIMP 2.10 release, this will have to be considered as "stable" and probably used in many scripts out there. So it would be better that we don't change this again (even though the API can be changed for the next major, GIMP 3.0). If nobody speaks up by the time we release 2.10, we can simply close this report and consider it as FIXED.
We should add to our Basic Batch tutorial and explain how this is achieved, otherwise there will simply be no feedback.
The one issue I have with the attached script is it called (gimp-quit). That is fine if it will only ever be used in batch mode and never called from within GIMP. If it might get called/used within GIMP you don't want it calling the quit function.
From comment 10: > (2) I removed the call to gimp-quit at the end of the loop. The macro should > not quit GIMP. The user may still have processing to do:
Is there anything left to do here? Jehan?
Not really. Some people were voicing disagreements about the naming and/or the API. So I was only keeping this opened for them to provide an improvement patch. Let's just close as FIXED. If anyone still thinks it should be changed, please reopen or open a new bug report with a patch before it hits stable.