Here is a f23 crash report:

https://retrace.fedoraproject.org/faf/problems/bthash/?bth=3e843c8b22e93b1c6be557a86b6694c61c0161fb&bth=68aba2555a4c1308bd8fac8ec694c1e12f20963e&bth=7739bf887bd56ac532b04fbcab40e54f2730d953

I think what is happening here is that remove_operations_button_attention_style removes the timeout_id *without* removing the actual timeout. I guess you assumed that returning G_SOURCE_REMOVE is good enough. And it is, but only if you return it from an actual timeout. This function is also called from add_operations_button_attention_style (why twice in a row, btw ?). And this leads to the timeout_id being reset without the scheduled timeout getting cleared. And if you  are then quick enough to close the window, the dispose() handler will not see the timeout_id, thus not remove the timeout either. And then the timeout gets eventually run and accesses the already-finalized toolbar.