Thanks for taking the time to report this bug.
However, you are using a version that is too old and not supported anymore. GNOME developers are no longer working on that version, so unfortunately there will not be any bug fixes for the version that you use.

By upgrading to a newer version of GNOME you could receive bug fixes and new functionality. You may need to upgrade your Linux distribution to obtain a newer version of GNOME.
Please feel free to reopen this bug if the problem still occurs with a newer version of GNOME.

Now... if you'd possibly have read my report... than you'd have seen that it still applies to all current versions...

I wonder now whether I should ask reopen that bug... it seems to be a general Evolution policy to corrupt their user's mail... so why trying to get it fixed?

I mean come on.. it's only the 3rd line where I mention that it still hapens in 3.5 and I guess so in 3.6 too...

*crashes his head against the table*

(In reply to comment #1)
> Thanks for taking the time to report this bug.
> However, you are using a version that is too old and not supported anymore.
> GNOME developers are no longer working on that version, so unfortunately there
> will not be any bug fixes for the version that you use.

Now would you please be so kind to re-open the bug? Or do I have to report it again with version (illogically set to the most recent version where the problem was found, and not the first one)?

Oh and one thing for the records...

Even IF this issue would just affect old versions... it is unbelievable outrageous that it's being simply closed... instead of at least warning all users via announcement-mailing lists and release notes... so that people at least know where they are.


This is just the same as it likely already happened some years ago (during the time with all the bugs on index corruptions), when there was likely one (or even many?) bugs that caused corruptions on the mails themselves... there was never a clear statement on whether this was the case or not... and it seemed as if things were swept under the carpet.

Just saw that reopening works now... before an error occurred.

So... one night after...

I did not want to personally insult anyone ("fucking idiots")... well I did want to... but I take it back now.


Anyway... for the reasons I wrote in later comments,... I hope it should be clear that there is some problem in Evolution, with respect to tests and such things.
If there were proper tests... such an issue could likely not have remained un-noted for so many years.

Further, there is a problem in how the bug tracking is handled. It's simply ridiculous that any issue is closed without reading... just because a (in your guys opinion) "wrong" version is set.

Last but not least, there is a general problem in design philosophies, which I guess mainly come from the evil GNOME philosophies (i.e. drop as much features as possible... make software for users that are assumed to be dumb).
Also there are some rather critical things missing (just look at the other bugs I have opened)... some which even impact users’ security (with respect to crypto) on some higher level.
A similar big problem is when you decided on your side to simply drop support from the mbox style archive you were using all the time, breaking many setups (and there is simply no single excuse for this).


Right now, evolution misses a lot of needed features... some of them are obvious like support for most mailbox formats (the different mbox-subformats, maildir, MH, etc.) others, I've reported already.

I hope you guys would understand, that the stupid end user on which GNOME targets all its software... and which some of them actually don't need all those advanced features.... also never use a MUA that is a binary.
These people typically always use some web-based MUA.
So to some extent,.. either Evolution turns more into an MUA for professionals,... or it is anyway rather useless because of the web-based MUAs.

But, to be honest, Thunderbird is no a real alternative either and I wouldn't know any other real one (well unless KMail perhaps).



Now back to that issue...
So far I informed the Debian community, by posting to debian-user and debian-user-german, as well as opening a critical bug there against Evolution, thus users upgrading or freshly installing Evolution have a chance to see it (when they use apt-listbugs).
Further I proposed to add an entry about the corruption in Debians NEWS file for Evolution.

I personally won't inform the other major distros' communities.
But Matthew is from Redhat... so could you please take appropriate steps for Fedora/Redhat?


Cheersm
Chris.

Oh and we need someone from SUSE... is there anywone amongst the Evolution developers who could do it for SUSE?


btw: From all my mails... at least (!) 16754 are unrecoverably corrupted due to this issue :( :( :(

@Christoph Anton Mitterer:

Maybe come to the point instead of writing novels and show some basic manners, and your bug reports might get even completely read?

Please read https://live.gnome.org/CodeOfConduct before any further Bugzilla activity. If I see such behavior again your account might be disabled.

> Oh and we need someone from SUSE

Then contact SUSE.

*** This bug has been marked as a duplicate of bug 529215 ***

@André:
I'm really not much surprised that this issue is again closed and hidden behind a duplicate (even though it was not even one, as I explain in the split out bug #686363), without not even solving the issue itself or going into the design/maintenance/philosophy issues I mentioned above in some comments.

Now you accuse me of bad conduct,... fine... but _I_ took it back above (even though many people on some mailing lists were I sent warnings around replied that they quite understand my reaction).
I wonder where the code of conduct for developers is, that should prevent them from silently letting their users mails corrupt for years.
Even if you guys think this was an unsolvable issue, as noted in bug #529215 (which it is not - every other MUA or mail server I know does it right)... this is simply outrageous against your users to not even warn them.


That you guys know about this bug for 4 years now without doing anything on it is simply ridiculous... I wonder what happens to non-public security issues that are reported here. 


Now if that's GNOME's way to deal with justified criticism and technical major problems... just go ahead... block my account.
By this issue I learned once again (as countless people - just ask Linus - did before) that contribution in any form (here: reporting issues) is not desired.


It seems to me that upstream is not going to fix this issue... I'll reopen the original bug in a few minutes though, posting the widespread solutions used by all other MUAs and mail servers... just in case you really didn't know about them (though even Wikipedia tells them).
Close it again if you want,... I'm out here unless being directly asked technical questions.


I've already opened Debian bug #690741 yesterday, cause I feel it my duty to warn people about such big problem.
As it seems the issue denied by upstream, I'll ask the Debian maintainer on what one can do to permanently warn users (e.g. by putting a note in the package description) or by disabling all mbox related functionally.


Cheers,
Chris.

(In reply to comment #10)
> Now you accuse me of bad conduct

It's hard to "accuse" you of facts. Reread your first sentence maybe?
If you want to interact with developers show basic manners. Just because you're on the internet doesn't mean to act like a jerk.
Do you take users/students seriously that start a conversation by "Hey Christian, dear fucking idiot"?

Bring up issues, discuss them in a civil tone with developers: Fine and welcome.
"I know better what you should do, you do it all wrong, you don't care enough, and my issue is the most important one EVAH!!!!" attitudes: I'm sick of it.

> but _I_ took it back above

Where?

> I wonder where the code of conduct for developers is,

It is here: https://live.gnome.org/CodeOfConduct

> Now if that's GNOME's way to deal with justified criticism and 
> technical major problems...

Hell yeah, let's make this big by generalizing! It must be GNOME's problem, not Evolution's problem! No wait, that's the reason why open source in general will never win!

> I learned once again that contribution in any form (here: reporting issues)
> is not desired.

See above. It's your personal tone and attitude that is not welcome, not "contribution in any form".

> As it seems the issue denied by upstream

Citation needed. Where was your contributed patch refused?

Ah I can't resist and answer one last time.


> > Now you accuse me of bad conduct
> It's hard to "accuse" you of facts. Reread your first sentence maybe?
> If you want to interact with developers show basic manners. Just because
> you're
> on the internet doesn't mean to act like a jerk.
Honestly,... when looking at the situation, that such an issue is known since 4 years... not being fixed,... and not even warned users about it's existence... then I wonder which sides acts like a jerk.


> Do you take users/students seriously that start a conversation by "Hey
> Christian, dear fucking idiot"?
Surely I'm not happy, but when I find out that I made such a big mistake, I probably try to understand why people are mad on me.


> > but _I_ took it back above
> Where?
comment 7 ... 2nd paragraph.


> > I wonder where the code of conduct for developers is,
> It is here: https://live.gnome.org/CodeOfConduct
It also says "Assume people mean well"... well after the first report 4 years ago was simply closed the next one, too, and mine now twice... I can hardly believe that it's meant well against users if they're silently left vulnerable to silent data corruption.
Does the code of conduct still apply then?


> Hell yeah, let's make this big by generalizing! It must be GNOME's
> problem, not Evolution's problem!
Well I know the same at least from NM, where we had similar discussions... what I wanted to achieve was better integration of NM in Debian's and other native network management tools... so that people has less (valid) reasons to flame on NM, cause it would at least not cause much trouble anymore.
The general tone was "we won't do it and even if someone came up with patches... we actually don't quite want it... because NM should be THE network configurator for anyone"...
The same holds true when GNOME decided to come up with GNOME Shell. Some people may like it... fine.. but the long term plan is to abolish the old way (which is now called legacy/fallback)... in other words, GNOME comes along and tells it's user to know it better than an interface style that has now proven usable and functional for more then 15 years.

So again, I really wonder which side is arrogant and believes to know it better per se.


> > As it seems the issue denied by upstream
> Citation needed. Where was your contributed patch refused?
The original bug report was alread marked as "NOTABUG"... which most people probably recognise as "we do not consider this to be an issue, therefore we won't do anything with respect to it".


Regarding "my patch":
- Reporting issues does not necessarily mean that one wants to dive deeply in a complex program like Evolution. I personally am already into too many projects, sorry.

- Regarding me personally,... it's now the 2nd time that Evolution caused me massive loss and corruption of mails, it made me an awful lot of work, when it break valid setups by removing it's own local mbox format. And there are the "minor" issues where I saw potential for improvement or to prevent meta-security attacks (all of which I reported)...
After so much troubles... do you really think I'll stay at Evolution? Or actually start to contribute on it in form of patches? Especially when (after this and other issues) I must assume, that such critical things are generally swept under the carpet and therefore must wonder how many further such things exist?!

The main reason I reported this, was the hope, that ultimately other users might be spared from silent mail corruption.

(In reply to comment #12)
> that such an issue is known since 4 years

Priorities differ.

> It also says "Assume people mean well"... well after the first report 4 years
> ago was simply closed the next one, too, and mine now twice... I can hardly
> believe that it's meant well against users if they're silently left vulnerable
> to silent data corruption.
> Does the code of conduct still apply then?

Does that mean you imply bad intentions?

> I wanted to achieve was better integration of NM
> The general tone was "we won't do it
> So again, I really wonder which side is arrogant and believes to know it better
> per se.

"We won't do it" with LONG EXPLANATIONS WHY. You sound like you were ignored. You were not. Maintainers decide which way a codebase takes. If they disagree with you, just do it. Nobody blocks you from making NM work better in Debian. Or fork. Take the code, get it into Debian and ship it, and convince upstream in case it's good.

> After so much troubles... do you really think I'll stay at Evolution?

Your personal preferences are up to you, and there will never be an application that fits well for every single user out there.
We welcome bug reports if they are well-written. https://bugzilla.gnome.org/page.cgi?id=bug-writing.html might tell you how to achieve that in the future. In case you write novels instead of well-structured bug reports I can understand though when your reports are ignored or not getting properly read, as time and manpower is extremely limited.

(In reply to comment #13)
> > that such an issue is known since 4 years
> Priorities differ.
Evolution is still also MUA (aka Mail User Agent), right? So one thing it mainly goes about is mail?

In that context, I seriously wonder what can probably have higher priority than irrecoverably damaging those mails?! The only thing I can imagine are security issues, e.g. those that allow remote exploits.

 
> > ago was simply closed the next one, too, and mine now twice... I can hardly
> > believe that it's meant well against users if they're silently
> > left vulnerable
> > to silent data corruption.
> > Does the code of conduct still apply then?
> Does that mean you imply bad intentions?
Do I claim that someone went along and intentionally added or removed code that caused breakage? No.

But given that this issue has been reported the 3rd time now,.. what options remain.
- Either the bug(s) were simply never read and closed (at least the case when my bug was closed the first time)... that's quite close to bad intentions, isn't it?

- Or it was read, but not understood (and in all cases, the issue was not formulated in a language that could not be understood)... well then either we're back to the "idiot"[0] thingy... or we must accuse the people closing the issues away of not at least asking someone who knows better.
Now the particular issue is not very difficult to understand... so I would hope that this is not the case.

- Last but not least,.. it was read, was understood, but simply not cared about... again... quite close to bad intentions.


And now, when were things escalated due to my insulting around (and let's be honest... if I wouldn't have done so... this issue would now also be marked as a duplicated of a bug that was already closed as notabug),... what happened?
Any announcements on mailing lists like some gnome-security or -announce lists, where hopefully many users can be reached, warning them about the serious corruption?
Any plans to add notes to the release files of Evolution? Any efforts in communication with the major distros (apart from what I did for Debian), so that they might try to communicate things to their users?
Still feels somewhat like "better not spread bad publicity" which is against the users in this case... again quite close to bad intentions.

And even if the reporter (I) was the biggest jerk on earth... that doesn't justify that the other users are not warned if he reports a serious issue.


> Your personal preferences are up to you, and there will never be an
> application
> that fits well for every single user out there.
Agreed,... and one could argue on that with respect to the local-emaisl-mbox-format dropping... but not with respect to this very issue.
I guess every user of a MUA has a "personal preference" of data integrity.


> We welcome bug reports if they are well-written.
Well one sees what has happened to bug 529215...


> https://bugzilla.gnome.org/page.cgi?id=bug-writing.html might tell you how to
> achieve that in the future.
Given that I write thousands of tickets,.. I guess I know how to do already, and usually my input is quite appreciated.
That I become as offensive as with the initial report here happens rarely (actually I guess it was the first time ever)... and even if, not without reason.


> In case you write novels instead of well-structured
> bug reports I can understand though when your reports are ignored or not
> getting properly read
When you refer to the other ticket's I've opened... well some of them might have been lengthy, but for many, elaborate explanation is simply necessary.


> as time and manpower is extremely limited.
Limited manpower doesn't justify not communicating such sever issues in form of release notes/mailinglists/etc. to users... one could also add a small pop up window on the next version of Evolution that tells anyone what likely has happened over the past years.

And if man-power is so limited that even this and core-functionality can't be assured... on should be so honest to mark a product as unmaintained/broken/experimental-state or similar.


Before you wrote:
> > As it seems the issue denied by upstream
> Citation needed.

Now some messages afterwards,... the tickets are still NOTABUG/DUPLICATE... doesn't seem my assumption is that wrong, does it?


Cheers,
Chris.


[0] Oh yeah,... and if you feel now that is reason to block me, than please take care that I'm also removed from any mail sending (already tried to remove myself from this ticket, but guess that's not possible as reporter).

(In reply to comment #14)
> Given that I write thousands of tickets,.. I guess I know how to do already

I respectfully disagree. I'd appreciate less "..." and some structure for longer  / more complicated issues, like steps to reproduce, actual outcome, expected outcome, reproducibility.

> Limited manpower doesn't justify not communicating such sever issues in form of
> release notes/mailinglists/etc. to users... one could also add a small pop up
> window on the next version of Evolution that tells anyone what likely has
> happened over the past years.

Do you volunteer to write that up? 
Why do you think that average users care about software updates?
Why do you think average users won't just click away these pop-ups?

> And if man-power is so limited that even this and core-functionality can't be
> assured... on should be so honest to mark a product as
> unmaintained/broken/experimental-state or similar.

Sure, let's do that. 
How exactly to identify such products? And where to mark that?

> Now some messages afterwards,... the tickets are still NOTABUG/DUPLICATE...
> doesn't seem my assumption is that wrong, does it?

We probably have different definitions of denying. Let me assure you that 99% of open source software projects with a certain codebase size deny issues.

(In reply to comment #15)
> like steps to reproduce, actual outcome,
> expected outcome, reproducibility.
Most of what I reported has no steps to reduce or outcome, because it's just not implemented.
And I guess for the From_ issue it should be quite clear to the educated reader, what's happening.


> Do you volunteer to write that up?
Usually I do in such cases, but not for Evolution.
We were having now a lengthy discussion and basically all my general/philosophical issues were simply ignored.
All the points where I accused that something severely goes wrong in the handling of this project. Ignored.
Rather you go into nit-picky details like me using too often an ellipsis ("...").

Not to talk about any technical aspects on this issue,... still no single word from any Evolution developer about it, it's being closed.

I asked several times whether anyone could seriously believe that I'm not angry. Ignored.


Now just before I found Matthew closing bug #681706. "Wontfix". The same typical misbehaviour I decried several times in this discussion. (That bug is clearly not a wontfix - which is generally considered as "I is not possible to fix it" - it's rather a "we don't want to do it", in which case the proper way would be to keep it open till someone takes hold onto it.)
I guess I do not need to again elaborate why bug #681706 would have been security critical for all users relying on any crypto,... whether they know/demand it or not.


> Why do you think that average users care about software updates?
> Why do you think average users won't just click away these pop-ups?
Well but _then_ it's clearly that user's fault. If a user is dump and clicks away important messages - well we can't stop them from shooting himself.
But what's in a software developer's responsibility is to at least give them a good chance to get informed.


> Sure, let's do that. 
> How exactly to identify such products? And where to mark that?
Again.. release notes, announcement mailing lists... the maintainers from distros will pick it up and have the change to add information to things like e.g. NEWS files or release notes, which users can be expected to read.
If they don't - their fault.


> > Now some messages afterwards,... the tickets are still NOTABUG/DUPLICATE...
> > doesn't seem my assumption is that wrong, does it?
> We probably have different definitions of denying. Let me assure you that 99%
> of open source software projects with a certain codebase size deny issues.
Sure. Often this is not a problem. But not if it's core functionality.

Remember the OpenSSL debacle in Debian? That was _really_ severe and it was _really_ embarrassing for Debian (even if upstream was IMHO not completely irresponsible on the whole issue).
But they knew... security is the core of OpenSSL and if that's broken in a way where a normal update won't help (as with the "usual" OpenSSL security bugs) because of weak keys... then they must inform the users on all possible information channels.

Remember the floating point bug Intel had years ago in one of it Pentium (?) CPUs? Serious corruption of data could result out of it... they widely announced it and people had the chance to work around it and check already computed data.




Anyway,.. please let's stop this discussion now.
Seeing #681706 made me fully pissed off and I don't think this will lead to anything. Moreover, everything has already been said, especially with respect to the administrative/philosophical/design problems in this project.

Should anyone need my technical advise on this issue, please either via bug #529215 or (if I don't answer cause my account got blocked) directly via email (calestyo AT scientia.net).





For those trying to manually fix the corruption:
- Don't use mbox in Evolution anymore, neither importing nor exporting.
- I'm not sure whether this corruption does not occur in other account types of Evolution (maildir, imap, etc.)... but at a short glance it seems that maildir and IMAP were save.
- If you want to find any places in your mail that were possibly corrupted, that grep and the regexp "^>From " are your friends.
- From_ lines with more than one leading ">", i.e. those matching "^>>>*From " should be save. As far as I could see, Evolution didn't quote them.
- Be aware that all your manual corrections are typically just assumptions (unless you can then verify those mails via signatures)

Hope that's all that needs to be considered.

Bye bye.

(In reply to comment #16)
> All the points where I accused that something severely goes wrong in the
> handling of this project. Ignored.

I don't think I ignored them, I just don't know what else is there to discuss when it comes to differing in priorities and usual outcomes of limited manpower in projects combined with demanding requests in a highly aggressive tone.

You can probably ask yourself why you feel ignored and don't succeed to get your message through if you "hide" important issues in long unstructured comments with lots of noise around.