While working on bug #656377, I realized the underlying .source files are not removed properly for collection backends, the whole UI knows the source was removed, still it's back after restart.

I'm not sure if the easiest steps, but at least what I do:
a) grab EWS with the change for bug #656377
b) In Mail view pick from account's context menu "Manage Subscriptions" to
   get list of EWS account's Public Folders
c) subscribe to a calendar source
d) close the dialog, switch to Calendar view and see it's there
e) back to Mail view, and Manage Subscriptions - see the calendar is checked
f) uncheck the calendar - no error claim, no console warning
g) close the dialog and go to Calendar view
   - either the public folder is still there or it's gone
h) close evolution and all other evolution-related processes
i) run evolution again - the public calendar is still there
j) right-click it and "Delete", confirm about sure to delete from server
   - the source registry may show "Removing source (xxx)" eventually
k) close evolution and all other evolution-related processes
l) run evolution again - the public calendar is still there

When checking corresponding folder in
   ~/.cache/evolution/sources/<ews-uid>/
I see the corresponding .source file of the public Calendar is left there after delete attempts.