GNOME Bugzilla – Bug 144352
Perspective transforms add extra row of transparent pixels to source image
Last modified: 2004-12-22 21:47:04 UTC
the new preview option shows this clearly: if interpolation is set to 'none', the result should be nearly identical with the preview. when two of the corners are close together, this is not so. the preview shows something that is about as i expect, but after clicking 'transform' i get a result which is nothing like it (roughly as if the entire selection were scaled to fit between the two smallest points). steps to reproduce: 1. select perspective-transform tool 2. enable preview, set interpolation to none 3. move two adjacent corners towards each other, so that the distance between them halves. 4. hit 'transform' and check if the result looks much like the preview did. 5. if not, repeat 3+4, reducing the distance between the corners further. this bug occurs with all interpolation modes, interpolation = none just makes it extremely obvious.
it appears to relate to the distance that the circular 'centre-point' is from the actual centre of the shape. it only appears when the 'centre-point' becomes too close to one of the edges. this can be through excessive expansion or excessive contraction of one side of the original selection
I don't see what interpolation has to do with this. What you're seeing is actually the correct behavior of the perspective tool - it creates a three-dimensional perspective effect, i.e. the distance between things diminishing as they get farther away. So, if you squish two endpoints together, the transformed image will accordingly look like it has been stretched out. The grid is actually a much more accurate representation of this. The effect of the preview is more like moving the corners of the image around while it stays flat on the screen, rather than a 3D effect. If you want a tool that does this, that's one thing - but this isn't a bug in the perspective tool.
interpolation has nothing to do with this (other than making it easy to see something's wrong). actually try this out, and you will see, at a certain point it STOPS looking like the polygon that describes the edges. it should produce a result like this: ___ / \ / \ ._______. but actually produces a result like this: ___ | | | | |__| (that is, the bottom edge gets much smaller even though i never changed it.) the preview behaves correctly , always showing something like the first case. the result doesn't. the easiest way to see this is try to perspectivise a filled box so that it looks like a triangle.
I can't reproduce this. Could you provide a screenshot of the conditions where this happens?
looks like the bug is different from what i thought, and actually offsets the unchanged edge like this: ___ / \ / \ ._______. result is shaped like ___ / \ ._____. (the source area still gets fully mapped.. just that it is mapped wrongly) the offset seems to be in proportion to the distance of the circle from the centre of the polygon. attaching two images.
Created attachment 28704 [details] the chosen perspectivization
Created attachment 28705 [details] the result (note offset)
Created attachment 28708 [details] transformed black area with a single row of white pixels I believe I understand. I made a comparison of linear interpolation and no interpolation, which I have attached. The area transformed was a black square with a single row of white pixels at the bottom, and should extend to the bottom of the image. On the left you can see that the white pixels in the last row fade into transparency (but stops in the center of the transparent row), and on the right it looks as if there is an extra row of transparent half-pixels. The transparent pixels should not be there at all - the last row of the source image should be the last row sampled.
Changing summary to better reflect the problem.
Actually these two cases represent two different problems. I've fixed the bug when no interpolation is used: 2004-06-15 Philip Lafleur <plafleur@cvs.gnome.org> * app/core/gimpdrawable-transform.c (gimp_drawable_transform_tiles_affine): Don't round texture coordinates when not using interpolation. Fixes bug #144352 for the nearest neighbor case only. The problem when interpolating is that it blends the texels surrounding a point specified by the texture coordinates, so it has to blend the edge pixels with transparent ones outside the texture. The simple solution to this would be to bound the texture coordinates one pixel away from the edges for linear interpolation, and (2, 3?) for cubic. A small amount of the picture would be lost, but it's better than having blurred edges.
This fix should probably go into the stable branch (gimp-2-0) as well.
Applied to gimp-2-0.
"The problem when interpolating is that it blends the texels surrounding a point specified by the texture coordinates" [...] "The simple solution to this would be to bound the texture coordinates one pixel away from the edges for linear interpolation, and (2, 3?) for cubic." meh. that would introduce error in the opposite direction, unless you adjusted the source rectangle, which would make it less intuitive :( is it possible to extend rather than crop the mapping? like, for linear, 0 0012344 1 2 3 4 4 (row/column map-- each number equates to a horiz/vertical line, in grid fashion) so that the extra row/column needed for interpolation is a copy of the last nearby row/column. this is consistent with the fix for linear interpolation, and is IMO the best solution for the general cases, since preservation of maximum data is usually most desirable. maybe doable using subdivision. actually, this seems like a problem with the interpolation code. in most interpolation cases, there are three ways to interpolate the edges: + wrap (best in the case of textures and tiles) + extend (best for highly detailed images) + crop (best for isolated objects with transparent areas) my investigations suggest that the fix would belong in pixel_surround_lock and introduce a new parameter 'edge_mode'. i don't fully grasp how the interpolation and pixel_surround code interacts, though.. is pixel_surround_lock called once for each source pixel?
If you're referring to the fix I just made, it is totally unrelated to the interpolation problem, so I'm not sure what you mean by consistent. I removed some code that rounded the texture coordinates up (so they pointed outside the texture halfway through the last row) that had no business being there. Currently the interpolation code checks if the texture coordinates are outside the source image, and if they are it substitutes a background color with 0 alpha. Extending the image would be possible, but it would be a bit more complicated and the extension of the edge pixels would certainly be visible in the result in many cases. If it were cropped, with linear interpolation half of the cropped pixels would still be visible because they would be blended with the adjacent pixels. With cubic interpolation you would probably lose maybe a whole row, but it's still pretty insignificant. I'm not familiar with pixel_surround_lock... but all the relevant texture-mapping code is contained in gimp_drawable_transform_tiles_affine(). There are separate functions for the bicubic and bilinear sampling, but these wouldn't need to be changed.
slight misunderstanding: what i call 'cropping' is the current behaviour: pixels outside the image are treated as empty. i regard 1pixel of error as too much. < 1 pixel is acceptable, but more is noticable especially if you need to draw things at the finished size. anyway, i see that code now. looks like for either wrap or extend the solution is easy: wrap or clamp UV coordinates respectively. I'll attempt this tomorrow.
I'm definitely opposed to wrapping (why would you prefer that to losing a pixel?) but I can agree with clamping/extending. I actually already attempted this today, and it won't work the way you mention. In this case the texture coordinates are calculated for every pixel in the destination area, not just the polygon you see. If you clamp the UV coordinates, the entire area will be filled.
"why would you prefer that to losing a pixel?" it's the most appropriate option for textures and tiles, since it makes the transform results consistent with the tileability of the texture. eg. if you place another polygon using the same texture immediately adjacent, there will be continuity.
Tileable images are certainly not a general case, and I think asking the user what to do with edge pixels would be confusing and unnecessary. I don't want pixels from one side of an image to appear on the other side if I'm transforming it, and I can't imagine why anyone would expect that to happen.
ok, i agree that 'extend' is the best general case. i just figured out what you meant by "If you clamp the UV coordinates, the entire area will be filled." .. so they need to be clamped only when they are <= 1 pixel (linear) or 3 pixels (cubic) outside of the image dimensions. i'll go try that.
Created attachment 28748 [details] [review] implements extension around texture edges this reduces blurring significantly. error is <= 80% of a pixel (this amount of error was noted along the longest side). works as described in preceding comment.
Created attachment 28749 [details] [review] modified patch I'm not sure why you think it worked for you because you used 'i' inside the clamp loop instead of 'b'. However, I've modified your patch slightly and it seems good to me - there's actually no need for the threshold value or to change the test for pixels that are completely outside the polygon. I think this should be applied but I wouldn't mind a second opinion.
Created attachment 28750 [details] [review] there are 5 coordinate pairs, not 4 Unfortunately these patches introduce a strange line parallel to the smallest side in some cases. I'll attach an example.
Created attachment 28751 [details] example of unwanted line
Created attachment 28758 [details] [review] fixed patch This seems to fix it. Testing would be appreciated.
for the original test case, it works perfectly. however, try this: + bring up the perspective tool + drag one corner almost to the centre. hit 'transform'. same as before, the amount of erroneous empty space created is proportionate to the distance corner->center (less distance == more error -- up to about 4 pixels) my testing has not broken this patch, so it seems an improvement on the previous behaviour (though it doesn't fully fix this bug :(
Sorry, but I don't see any problem when doing what you said; there is no empty space at all. I've also tested this patch thoroughly myself. Are you sure you're running the patched version and not one you installed somewhere else?
Actually, it does get offset a little, but only from the upper left. I'll take a look.
Created attachment 28777 [details] [review] fixed fixed patch This should fix it, please test. It was not a problem with the patch, but the result of more seemingly pointless rounding.
i haven't managed to break it, seems good to me :)
Applied to main branch: 2004-06-17 Philip Lafleur <plafleur@cvs.gnome.org> * app/core/gimpdrawable-transform.c (gimp_drawable_transform_tiles_affine): Make transforms (most notably perspective transforms) conform exactly to specified edges. Includes a patch by David Gowers. Fixes bug #144352.