Bug 83682 – [test case attached] Memory corruption (?) occurring in art_svp_vpath_stroke_raw - causes massive memory allocations.

After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.

Bug 83682 - [test case attached] Memory corruption (?) occurring in art_svp_vpath_stroke_raw - causes massive memory allocations.


Summary:	[test case attached] Memory corruption (?) occurring in art_svp_vpath_stroke_...


Status:	RESOLVED FIXED

Product:	libart
Classification:	Deprecated
Component:	Other
Version:	unspecified
Hardware:	Other Linux

Importance:	Normal critical
Target Milestone:	---
Assigned To:	Nautilus Maintainers
QA Contact:	Nautilus Maintainers

URL:
Whiteboard:

Depends on:
Blocks:

Reported:	2002-05-31 12:10 UTC by Richard Kinder
Modified:	2004-12-22 21:47 UTC

See Also:
GNOME target:	---
GNOME version:	2.0

Attachments
Tarball containing a test case that causes problems in libart_lgpl. (90.58 KB, tar/gz) 2002-05-31 12:13 UTC, Richard Kinder	Details

Description Richard Kinder 2002-05-31 12:10:54 UTC

Warning: Very long bugreport!

Version used gnome cvs head from a few days ago.

OK, in the process of porting the seti_applet to GNOME 2, I decided to move
the gnomecanvas I use for drawing the various pieces of information to use
anti-aliasing. It all works kind of nice, except one piece where I try to
create a graph type image on the canvas.

I have traced the problem to the innards of libart, my analysis up until
now is below this intro.

The short of it is that when the aa canvas object is being rendered, due to
corruption of a couple of data points in the path, massive amounts of
memory are allocated, which causes system instability (OOM kills the
application after first killing every other process :).

Attached is the smallest test case I could put together in short time. Note
that this case segfaults, but if the #if 0 is changed to #if 1, the systems
gets into a very bad state (starts allocating > 200meg). This is a VERY bad
thing to happen. Please review the gnome canvas stuff and tell me if I'm
doing anything bone-headed.

Half complete analysis follows (real-life prevents completion of this in
any realistic time-frame):

Analysis:

Tracing the code executed when the main graph canvas item (polygon item) is
created, the problem seems to lie in art_svp_vpath_stroke_raw - when points
are added to the forward path. The two listings at the end show the forward
path at two points in the execution of art_svp_vpath_stroke_raw. There is
also a raw memory dump of the appropriate memory location.

An alternative analysis is that I am doing something REALLY bone-headed in
creating the gnome canvas item. I hope this is the case, but due to the
observed corruption deep inside libart_lgpl, I doubt this.

---------------------------------
Corrupted 'forw' pointer listings:

'forw' prior to corruption:
(gdb) list_p 40 forw
$1555 = {code = ART_LINETO, x = -0.5, y = 5.9767486668125569}
$1556 = {code = ART_LINETO, x = 1.75, y = 5.9767486668125569}
$1557 = {code = ART_LINETO, x = 1.75, y = 0}
$1558 = {code = ART_LINETO, x = 1.25, y = 0}
$1559 = {code = ART_LINETO, x = 0.75, y = 0}
$1560 = {code = ART_LINETO, x = 0.75, y = 2.6387510281897835}
$1561 = {code = ART_LINETO, x = 3, y = 2.6387510281897835}
$1562 = {code = ART_LINETO, x = 3, y = 0}
$1563 = {code = ART_LINETO, x = 2.5, y = 0}
$1564 = {code = ART_LINETO, x = 2, y = 0}
$1565 = {code = ART_LINETO, x = 2, y = 2.0813832126404614}
$1566 = {code = ART_LINETO, x = 4.25, y = 2.0813832126404614}
$1567 = {code = ART_LINETO, x = 4.25, y = 0}
$1568 = {code = ART_LINETO, x = 3.75, y = 0}
$1569 = {code = ART_LINETO, x = 3.25, y = 0}
$1570 = {code = ART_LINETO, x = 3.25, y = 4.769385839597045}
$1571 = {code = ART_LINETO, x = 5.5, y = 4.769385839597045}
$1572 = {code = ART_LINETO, x = 5.5, y = 0}
$1573 = {code = ART_LINETO, x = 5, y = 0}
$1574 = {code = ART_LINETO, x = 4.5, y = 0}
$1575 = {code = ART_LINETO, x = 4.5, y = 3.2627927506602856}
$1576 = {code = ART_LINETO, x = 6.75, y = 3.2627927506602856}
$1577 = {code = ART_LINETO, x = 6.75, y = 0}
---Type <return> to continue, or q <return> to quit---
$1578 = {code = ART_LINETO, x = 6.25, y = 0}
$1579 = {code = ART_LINETO, x = 5.75, y = 0}
$1580 = {code = ART_LINETO, x = 5.75, y = 3.1285054091445343}
$1581 = {code = ART_LINETO, x = 8, y = 3.1285054091445343}
$1582 = {code = ART_LINETO, x = 8, y = 0}
$1583 = {code = ART_LINETO, x = 7.5, y = 0}
$1584 = {code = ART_LINETO, x = 7, y = 0}
$1585 = {code = ART_LINETO, x = 7, y = 3.8733555068065755}
$1586 = {code = ART_LINETO, x = 9.25, y = 3.8733555068065755}
$1587 = {code = ART_LINETO, x = 9.25, y = 0}
$1588 = {code = ART_LINETO, x = 8.75, y = 0}
$1589 = {code = ART_LINETO, x = 8.25, y = 0}
$1590 = {code = ART_LINETO, x = 11.25, y = 0}
$1591 = {code = ART_LINETO, x = 11.25, y = 0}
$1592 = {code = ART_LINETO, x = 11.25, y = 5.6231801011759677}
$1593 = {code = ART_LINETO, x = 12.5, y = 5.6231801011759677}
$1594 = {code = ART_LINETO, x = 12.5, y = 0}
$1685 = {code = ART_LINETO, x = 12.5, y = 0}
$1686 = {code = ART_LINETO, x = 12.5, y = 3.0967901874241432}
$1687 = {code = ART_LINETO, x = 13.75, y = 3.0967901874241432}
$1688 = {code = ART_LINETO, x = 13.75, y = 0}
$1689 = {code = ART_LINETO, x = 13.75, y = 0}
$1690 = {code = ART_LINETO, x = 13.75, y = 6.1439993600049112}
---Type <return> to continue, or q <return> to quit---
$1691 = {code = ART_LINETO, x = 15, y = 6.1439993600049112}
$1692 = {code = ART_LINETO, x = 15, y = 0}
$1693 = {code = ART_LINETO, x = 15, y = 0}
$1694 = {code = ART_LINETO, x = 15, y = 3.343588293392167}
$1695 = {code = ART_LINETO, x = 16.25, y = 3.343588293392167}
$1696 = {code = ART_LINETO, x = 16.25, y = 0}
$1697 = {code = ART_LINETO, x = 16.25, y = 0}
$1698 = {code = ART_LINETO, x = 16.25, y = 2.0293425655105435}
$1699 = {code = ART_LINETO, x = 17.5, y = 2.0293425655105435}
$1700 = {code = ART_LINETO, x = 17.5, y = 0}
$1701 = {code = ART_LINETO, x = 17.5, y = 0}
$1702 = {code = ART_LINETO, x = 17.5, y = 4.3053157964136552}
$1703 = {code = ART_LINETO, x = 18.75, y = 4.3053157964136552}
$1704 = {code = ART_LINETO, x = 18.75, y = 0}
$1705 = {code = ART_LINETO, x = 18.75, y = 0}
$1706 = {code = ART_LINETO, x = 18.75, y = 5.1822439502208466}
$1707 = {code = ART_LINETO, x = 20, y = 5.1822439502208466}
$1708 = {code = ART_LINETO, x = 20, y = 0}
$1709 = {code = 1074861017, x = 749.4414062500465, y = -1.0000002403892001}
$1710 = {code = 3786165403, x = 2.6530476935189564e-314,
  y = 9.0498870944156119e-270}
$1711 = {code = 135471136, x = 0, y = 5.3114652946463512e-315}
$1712 = {code = 9, x = 7.3208854788301895e-312, y = 749.60559182930865}
(gdb)


'forw' post corruption - see ***CORRUPTION*** in the listing below

(gdb) list_p 70 forw
$1852 = {code = ART_LINETO, x = -0.5, y = 5.9767486668125569}
$1853 = {code = ART_LINETO, x = 1.75, y = 5.9767486668125569}
$1854 = {code = ART_LINETO, x = 1.75, y = 0}
$1855 = {code = ART_LINETO, x = 1.25, y = 0}
$1856 = {code = ART_LINETO, x = 0.75, y = 0}
$1857 = {code = ART_LINETO, x = 0.75, y = 2.6387510281897835}
$1858 = {code = ART_LINETO, x = 3, y = 2.6387510281897835}
$1859 = {code = ART_LINETO, x = 3, y = 0}
$1860 = {code = ART_LINETO, x = 2.5, y = 0}
$1861 = {code = ART_LINETO, x = 2, y = 0}
$1862 = {code = ART_LINETO, x = 2, y = 2.0813832126404614}
$1863 = {code = ART_LINETO, x = 4.25, y = 2.0813832126404614}
$1864 = {code = ART_LINETO, x = 4.25, y = 0}
$1865 = {code = ART_LINETO, x = 3.75, y = 0}
$1866 = {code = ART_LINETO, x = 3.25, y = 0}
$1867 = {code = ART_LINETO, x = 3.25, y = 4.769385839597045}
$1868 = {code = ART_LINETO, x = 5.5, y = 4.769385839597045}
$1869 = {code = ART_LINETO, x = 5.5, y = 0}
$1870 = {code = ART_LINETO, x = 5, y = 0}
$1871 = {code = ART_LINETO, x = 4.5, y = 0}
$1872 = {code = ART_LINETO, x = 4.5, y = 3.2627927506602856}
$1873 = {code = ART_LINETO, x = 6.75, y = 3.2627927506602856}
$1874 = {code = ART_LINETO, x = 6.75, y = 0}
---Type <return> to continue, or q <return> to quit---
$1875 = {code = ART_LINETO, x = 6.25, y = 0}
$1876 = {code = ART_LINETO, x = 5.75, y = 0}
$1877 = {code = ART_LINETO, x = 5.75, y = 3.1285054091445343}
$1878 = {code = ART_LINETO, x = 8, y = 3.1285054091445343}
$1879 = {code = ART_LINETO, x = 8, y = 0}
$1880 = {code = ART_LINETO, x = 7.5, y = 0}
$1881 = {code = ART_LINETO, x = 7, y = 0}
$1882 = {code = ART_LINETO, x = 7, y = 3.8733555068065755}
$1883 = {code = ART_LINETO, x = 9.25, y = 3.8733555068065755}
$1884 = {code = ART_LINETO, x = 9.25, y = 0}
$1885 = {code = ART_LINETO, x = 8.75, y = 0}
$1886 = {code = ART_$1887 = {code = ART_LINETO, x = 8.25, y =
2.6299557468512451} <--forw[34]
$1887 = {code = ART_LINETO, x = 8.25, y = 2.6299557468512451}
$1888 = {code = ART_LINETO, x = 10.5, y = 2.6299557468512451}  ***CORRUPTION***
$1889 = {code = ART_LINETO, x = 1.8446744073709552e+19, y = 0} ***CORRUPTION***
$1890 = {code = ART_LINETO, x = 12.5, y = 5.6231801011759677}
$1891 = {code = ART_LINETO, x = 12.5, y = 0}
$1892 = {code = ART_LINETO, x = 12.5, y = 0}
$1893 = {code = ART_LINETO, x = 12.5, y = 3.0967901874241432}
$1894 = {code = ART_LINETO, x = 13.75, y = 3.0967901874241432}
$1895 = {code = ART_LINETO, x = 13.75, y = 0}
$1896 = {code = ART_LINETO, x = 13.75, y = 0}
$1897 = {code = ART_LINETO, x = 13.75, y = 6.1439993600049112}
---Type <return> to continue, or q <return> to quit---
$1898 = {code = ART_LINETO, x = 15, y = 6.1439993600049112}
$1899 = {code = ART_LINETO, x = 15, y = 0}
$1900 = {code = ART_LINETO, x = 15, y = 0}
$1901 = {code = ART_LINETO, x = 15, y = 3.343588293392167}
$1902 = {code = ART_LINETO, x = 16.25, y = 3.343588293392167}
$1903 = {code = ART_LINETO, x = 16.25, y = 0}
$1904 = {code = ART_LINETO, x = 16.25, y = 0}
$1905 = {code = ART_LINETO, x = 16.25, y = 2.0293425655105435}
$1906 = {code = ART_LINETO, x = 17.5, y = 2.0293425655105435}
$1907 = {code = ART_LINETO, x = 17.5, y = 0}
$1908 = {code = ART_LINETO, x = 17.5, y = 0}
$1909 = {code = ART_LINETO, x = 17.5, y = 4.3053157964136552}
$1910 = {code = ART_LINETO, x = 18.75, y = 4.3053157964136552}
$1911 = {code = ART_LINETO, x = 18.75, y = 0}
$1912 = {code = ART_LINETO, x = 18.75, y = 0}
$1913 = {code = ART_LINETO, x = 18.75, y = 5.1822439502208466}
$1914 = {code = ART_LINETO, x = 20, y = 5.1822439502208466}
$1915 = {code = ART_LINETO, x = 20, y = 0}
$1916 = {code = 1074861017, x = 749.4414062500465, y = -1.0000002403892001}
$1917 = {code = 3786165403, x = 2.6530476935189564e-314,
  y = 9.0498870944156119e-270}
$1918 = {code = 135471136, x = 0, y = 5.3114652946463512e-315}
$1919 = {code = 9, x = 7.3208854788301895e-312, y = 749.60559182930865}
---Type <return> to continue, or q <return> to quit---

mem dump of corruption above (from forw[34]) (<-- indicated the corruption
'x' point):

0x8131d08:      0x00000003      0x00000000      0x40208000      0x00000000
0x8131d18:      0x00000000      0x00000003      0x00000000      0x40208000
0x8131d28:      0x3d153a11      0x40050a26      0x00000003      0x00000000
0x8131d38:      0x40250000      0x3d153a11      0x40050a26      0x00000003
0x8131d48:      0x00000000      0x43f00000<--   0x00000000      0x00000000
0x8131d58:      0x00000003      0x00000000      0x40290000      0xeca84667
0x8131d68:      0x40167e22      0x00000003      0x00000000      0x40290000
0x8131d78:      0x00000000      0x00000000      0x00000003      0x00000000
0x8131d88:      0x40290000      0x00000000      0x00000000      0x00000003
0x8131d98:      0x00000000      0x40290000      0xef0c7bb6      0x4008c639
0x8131da8:      0x00000003      0x00000000      0x402b8000      0xef0c7bb6
0x8131db8:      0x4008c639      0x00000003      0x00000000      0x402b8000
0x8131dc8:      0x00000000      0x00000000      0x00000003      0x00000000
0x8131dd8:      0x402b8000      0x00000000      0x00000000      0x00000003
0x8131de8:      0x00000000      0x402b8000      0x917776cf      0x40189374
0x8131df8:      0x00000003      0x00000000      0x402e0000      0x917776cf

element 37 - the main corruption:
(gdb) x/64xw (forw+37)
0x8131d44:      0x00000003      0x00000000      0x43f00000      0x00000000
0x8131d54:      0x00000000      0x00000003      0x00000000      0x40290000
0x8131d64:      0xeca84667      0x40167e22      0x00000003      0x00000000
0x8131d74:      0x40290000      0x00000000      0x00000000      0x00000003
0x8131d84:      0x00000000      0x40290000      0x00000000      0x00000000
0x8131d94:      0x00000003      0x00000000      0x40290000      0xef0c7bb6
0x8131da4:      0x4008c639      0x00000003      0x00000000      0x402b8000
0x8131db4:      0xef0c7bb6      0x4008c639      0x00000003      0x00000000
0x8131dc4:      0x402b8000      0x00000000      0x00000000      0x00000003
0x8131dd4:      0x00000000      0x402b8000      0x00000000      0x00000000
0x8131de4:      0x00000003      0x00000000      0x402b8000      0x917776cf
0x8131df4:      0x40189374      0x00000003      0x00000000      0x402e0000
0x8131e04:      0x917776cf      0x40189374      0x00000003      0x00000000
0x8131e14:      0x402e0000      0x00000000      0x00000000      0x00000003
0x8131e24:      0x00000000      0x402e0000      0x00000000      0x00000000
0x8131e34:      0x00000003      0x00000000      0x402e0000      0x381b4332

Comment 1 Richard Kinder 2002-05-31 12:13:25 UTC

Created attachment 8884 [details]
Tarball containing a test case that causes problems in libart_lgpl.

Comment 2 Richard Kinder 2002-07-14 23:16:14 UTC

Update: if I turn optimisations off in the build of the library, the
code now dies in art_uta_add_line - stack trace attached.

---------------------------------------------------

(gdb) run
Starting program:
/home/rkinder/test_libart_lgpl_bug/src/./test_libart_lgpl_bug 
[New Thread 1024 (LWP 3256)]

Program received signal SIGSEGV, Segmentation fault.

+ Trace 25055

Thread 1024 (LWP 3256)

#0 art_uta_add_line
at art_uta_vpath.c line 93
#1 art_uta_from_vpath
at art_uta_vpath.c line 328
#2 art_uta_from_svp
at art_uta_svp.c line 51
#3 gnome_canvas_update_svp
at gnome-canvas-util.c line 470
#4 gnome_canvas_item_update_svp
at gnome-canvas-util.c line 536
#5 gnome_canvas_item_update_svp_clip
at gnome-canvas-util.c line 573
#6 gnome_canvas_shape_update
at gnome-canvas-shape.c line 1176
#7 gnome_canvas_polygon_update
at gnome-canvas-polygon.c line 252
#8 gnome_canvas_item_invoke_update
at gnome-canvas.c line 465
#9 gnome_canvas_group_update
at gnome-canvas.c line 1562
#10 gnome_canvas_item_invoke_update
at gnome-canvas.c line 465
#11 gnome_canvas_group_update
at gnome-canvas.c line 1562
#12 gnome_canvas_item_invoke_update
at gnome-canvas.c line 465
#13 do_update
at gnome-canvas.c line 3162
#14 idle_handler
at gnome-canvas.c line 3189
#15 g_idle_dispatch
at gmain.c line 3129
#16 g_main_dispatch
at gmain.c line 1617
#17 g_main_context_dispatch
at gmain.c line 2161
#18 g_main_context_iterate
at gmain.c line 2242
#19 g_main_loop_run
at gmain.c line 2462
#20 gtk_main
at gtkmain.c line 922
#21 main
at main.c line 174
#22 __libc_start_main
at ../sysdeps/generic/libc-start.c line 129

$1 = {x0 = -67108864, y0 = -2, width = 1, height = 1, utiles = 0x819f4f0}

Comment 3 Richard Kinder 2003-07-21 12:31:51 UTC

This bug is probably fixed thanks to this commit:

http://cvs.gnome.org/bonsai/cvsview2.cgi?diff_mode=context&whitespace_mode=show&root=/cvs/gnome&subdir=libart_lgpl&command=DIFF_FRAMESET&file=art_uta_vpath.c&rev2=1.8&rev1=1.7

I cannot reproduce the bug using the 'test case' attached (the graph
actually renders!)

I will now close this bug - to avoid crashes like this, use
libart_lgpl with version >= 2.3.12