Created attachment 179378 [details] [review]
fix length limitation of vsprintf() by using two pass vsnprintf() approach

While attempting to build doxygen documentation for a large software project I encountered memory corruption in realloc() that caused execution to abort.

A debugging session showed that doxygen was trying to print a warning in doxygen.cpp:685 for a path that is longer than about 200 characters.

A further analysis showed that the problem is rooted in SCString::sprintf( const char*, ...) in qtools/scstring.cpp:152. The function temporarily increases the string object size to at least 256 characters and then calls vsprintf() for the given arguments.

In my case the string argument was longer than 200 characters causing the "minlen" of 256 characters to be overflowed and the heap being corrupted. The following resize operation using realloc() then causes the program to crash.

I've attached a simple bugfix for SCString::sprintf( const char*, ...) that adds a two-pass operation using vsnprintf(). The first pass determines the number of characters that the given format string will require, then the string is resized accordingly. The second pass then performs the actual printing.

In my tests this fixed the crash and building documentation was possible again. The patch might, however, require more tuning. I couldn't determine whether the two-pass operation is supported under all circumstances by vsnprintf. The man pages are a tad unclear about that. Also the MS Windows CRT traditionally has got limitations regarding such C99 functionality.

Best regards,

Matthias