I'm facing the same problem. Is this bug fixed in 2.30?

It looks to be worse that this or at least more fundamental.  I was able to work around the lookup problem by the nasty kludge of making an entry in the /etc/hosts file for mydomain.com@zimbra.mydomain.com.  Once we also patched Evolution to stop rejecting vcards of mime-type text/vcard (https://bugzilla.gnome.org/show_bug.cgi?id=566330 - thank you whoever provided that), we got our contacts and celebrated.  Our joy was short lived because Evolution failed when we tried to save a new contact with a rather unhelpful "other error" message.

In tracing on both the mail server and the packets, it looks like this problem with the prepended domain shows up here, too.  Here is the packet trace:

PUT /dav/user@mydomain.com/Contacts/?fmt=vcf/16DA32F3-330C054A-437F603E.vcf HTTP/1.1

Host: mydomain.com@zimbra.mydomain.com

User-Agent: Evolution/2.28.3.1

If-None-Match: *

Content-Type: text/x-vcard

Authorization: Basic anN1bGxpdmFuQHBhY2lmZXJhLmNvbTpNYXlhMTB5c2l1cw==

Content-Length: 434



BEGIN:VCARD

VERSION:3.0

TEL;X-EVOLUTION-UI-SLOT=1;TYPE=WORK,VOICE:+1 (207) 123-4567

EMAIL;X-EVOLUTION-UI-SLOT=1;TYPE=WORK:nowhere@example.com

X-MOZILLA-HTML:FALSE

X-EVOLUTION-VIDEO-URL:

FBURL:

CALURI:

X-EVOLUTION-BLOG-URL:

X-EVOLUTION-FILE-AS:Contact\, Evolution

N:Contact;Evolution;;;

FN:Evolution Contact

NOTE:

X-EVOLUTION-SPOUSE:

NICKNAME:

X-EVOLUTION-ASSISTANT:

X-EVOLUTION-MANAGER:

ROLE:

TITLE:

URL:

END:VCARDHTTP/1.1 406 Not Acceptable

Server: nginx

Date: Sat, 08 May 2010 15:40:19 GMT

Transfer-Encoding: chunked

Connection: keep-alive


Notice the host field in the put packet.

On the Zimbra side mailboxlog file we see:

[btpool0-2700://mydomain.com@zimbra.mydomain.com/dav/user@mydomain.com/Contacts/?fmt=vcf/] aname=user@mydomain.com;ip=172.30.10.73;ua=Evolution/2.28.3.1;] FileUploadServlet - Received file: name=null, size=95, id=ab0a860b-d897-49f9-8589-dca6a468556e:144feae3-c0f1-48df-9fa3-d8f3b97dc02c

2010-05-08 11:40:19,511 INFO  [btpool0-2700://mydomain.com@zimbra.mydomain.com/dav/user@mydomain.com/Contacts/?fmt=vcf/] [aname=user@mydomain.com;ip=172.30.10.73;ua=Evolution/2.28.3.1;] dav - DavServlet operation PROPFIND to /home/user@mydomain.com/Contacts/ (depth: one) finished in 16ms

2010-05-08 11:40:19,579 INFO  [btpool0-2700://mydomain.com@zimbra.mydomain.com/dav/user@mydomain.com/Contacts/?fmt=vcf/16DA32F3-330C054A-437F603E.vcf] [aname=user@mydomain.com;ip=172.30.10.73;ua=Evolution/2.28.3.1;] FileUploadServlet - Received file: name=null, size=434, id=ab0a860b-d897-49f9-8589-dca6a468556e:a0854774-b6c4-4fbe-863e-52381ac2dd21

2010-05-08 11:40:19,579 INFO  [btpool0-2700://mydomain.com@zimbra.mydomain.com/dav/user@mydomain.com/Contacts/?fmt=vcf/16DA32F3-330C054A-437F603E.vcf] [aname=user@mydomain.com;ip=172.30.10.73;ua=Evolution/2.28.3.1;] dav - sending http error 406 because: no DAV resource for folder

2010-05-08 11:40:19,579 INFO  [btpool0-2700://mydomain.com@zimbra.mydomain.com/dav/user@mydomain.com/Contacts/?fmt=vcf/16DA32F3-330C054A-437F603E.vcf] [aname=user@mydomain.com;ip=172.30.10.73;ua=Evolution/2.28.3.1;] dav - DavServlet operation PUT to /home/user@mydomain.com/Contacts/ (depth: zero) finished in 1ms

We certainly need this fixed as quickly as possible we are not even developers no less Evolution hackers but we'd be willing to test and do whatever else we can do to help - John

In a desperate attempt to solve this critical problem for us, I stared at the code for many, many hours.  Unable to really understand where the problem was, I simply copied in the logic from the caldev backend that seemed to be working properly.  I added:

	gsize                     len;

and

	if (priv->uri) {
		SoupURI *suri = soup_uri_new (priv->uri);

		/* properly encode uri */
		if (suri && suri->path) {
			gchar *tmp = soup_uri_encode (suri->path, NULL);
			gchar *path = soup_uri_normalize (tmp, "/");

			soup_uri_set_path (suri, path);

			g_free (tmp);
			g_free (path);
			g_free (priv->uri);

			priv->uri = soup_uri_to_string (suri, FALSE);
		}

		soup_uri_free (suri);
	}

	/* remove trailing slashes... */
	len = strlen (priv->uri);
	while (len--) {
		if (priv->uri[len] == '/') {
			priv->uri[len] = '\0';
		} else {
			break;
		}
	}

	/* ...and append exactly one slash */
	if (priv->uri && *priv->uri) {
		gchar *tmp = priv->uri;

		priv->uri = g_strconcat (priv->uri, "/", NULL);

		g_free (tmp);
	}


 to 

static GNOME_Evolution_Addressbook_CallStatus
e_book_backend_webdav_load_source(EBookBackend *backend,
                                  ESource *source, gboolean only_if_exists)

Since this is the only place I could find where priv->uri is set and notes on soup_uri_to_string seem to imply that it knows how to strip out the user name and password.

It did not work :((  Packet traces still show the host with the domain prepended.  Where else can I look?

I can confirm that the@domain in the login uid is contributing to the problem.  When I change my uid from user@mydomain.com to simply user, the host value is properly formed without the prepended domain.

I think I have tracked down the formation of the URI in the e-book-backend-webdav.c file and it looks like it constructing it correctly.  However, the problem may be in the way the URI is stored in %gconf.xml.  I noticed that the URI was stored in the contact %gconf.xml as user@mydomain.com@server.mydomain.com whereas in the calendar %gconf.xml, it was stored as user%40mydomain.com@server.mydomain.com.  I've tried to change this manually but without success yet.

It's bug #578335 for encoding of @ in username for CalDAV, but it has some history under it too (more than this particular bug). I'll try to make this work for WebDAV too, though changes on Evolution side and reconfiguration of the WebDAV source will be probably needed. I will see.

I'll be more than happy to test this moment you have the patch.  I've already had to patch and rebuild to remove the restriction about only accepting the deprecated x-vcard as the mime-type value.

By the way, alluding to bug 578335, we are not using gnome-key-ring.  We rebuilt with it disabled.  We are running Evolution within vservers which preclude access to HAL and gnome-key-ring seems to insist on querying HAL generating endless syslog error messages.  Thanks - John

Created attachment 160712 [details] [review]
eds patch

for evolution-data-server;

I did this within eds only, no need to change evo part. By the way, when I was creating the new WebDAV account in evolution 2.31.1 (actual master), then the '@' in user name was properly escaped. So I changed it in GConf key, to be able to reproduce this. Note the URL was there twice, so I guess it didn't work for you (change from '@' to '%40'), because you missed the second URL. Just a guess. Anyway, with this patch it works for both cases as expected.

This patch also adds debugging facility to WebDAV address books similar to CalDAV, just export on a console of e-addressbook-factory environment variable WEBDAV_DEBUG with value: "all" for full logging, "headers" for only headers, "anything nonempty else" for minimal logging of exchange between evolution's backend and a server.

Created commit 1220c1d in eds master (2.31.2+)
Created commit 3b51add in eds gnome-2-30 (2.30.2+)

Wonderful. I'll try it as soon as I finish the rest of the morning calls and emails.  The same problem manifests itself in free/busy queries.  Will this patch address that, too, or is that separate logic? Thanks.

Alas, something is bad.  We are failing right at the starting line.  I get no contacts.  In fact, I get no packets attempting to retrieve contacts.  With debug=all, I see:

Entity: line 3: parser error : Extra content at the end of the document

Interesting.  I noticed in the debug that my URI was coming out as:
dav/user@domain/Contacts//?fmt=vcf (by the way ?fmt=vcf breaks adding contacts because it seems to be added into the contact name - I had just added it to troubleshoot this problem)

I then changed my source definition from:
dav/user@domain/Contacts/ (which is what I originally put in)
to:
dav/user@domain/Contacts
and that seemed to work.  Could be coincidence as I am doing this empirically rather than analytically but it is working now.  I'll report back on further testing.

Oh, you've right, there is a typo, here:
> if (!suri->path || !*suri->path || suri->path[strlen (suri->path) - 1])
should be check for "!= '/'" at the end. Note you shouldn't add any parameters to the uri, as it really only takes the url and places the name of new item to it.

Created commit 20c5952 in eds master (2.31.2+)
Created commit 030f4c9 in eds gnome-2-30 (2.30.2+)

You mentioned somewhere that you can see contacts created in evolution only in evolution, and those created in zzzimbra's web ui only in zzzimbra? It does that for me at least. From the list returned through the DAV I see that it returns only some items, so it might be on zzzimbra side. Depending on what they'll tell you, please open (or not) a new bug report for it. Note I tried to change "text-x-vcard" to "text/vcard; charset=utf-8" in the PUT request, but it didn't help. I believe it's either something really simple on evolution's side, or on Zzzimbra side.

I am wondering if it is a combination.  In Zimbra's REST documentation (http://wiki.zimbra.com/wiki/ZCS_6.0:Zimbra_REST_API_Reference#Response_Formats), they do state that the vcf interpretor is triggered by appending ?fmt=vcf or it will assume the uploaded file is csv.  I notice that you are appending the name of the file ending in .vcf.  I would assume Zimbra knows what to do with that and doesn't need the fmt parameter.  However, it does seem to be misinterpreting the type as type 8 (document) rather than type 6 (contact).  I assume those types are specific to Zimbra.

I'll manually post a contact using curl and check the mailbox.log file and the packet trace to see how it forms the DAV request.  Thanks - John

Alas, the patch does not solve the terminal "/" problem.  I just repatched, changed my sources to end in "/" and it failed until I removed the "/".

(In reply to comment #15)
> Alas, the patch does not solve the terminal "/" problem.  I just repatched,
> changed my sources to end in "/" and it failed until I removed the "/".

What does it mean "failed" exactly? I tried to reproduce it, and regardless I add a slash at the end of the URL in preferences or not I see the used URL still the same, with exactly one slash at the end. The only difference is that when I change the URL in preferences then I'm asked for a password, but I enter it and everything works as expected.

If I include the terminal "/", opening the address book hangs.  No contacts are returned and the status bar shows a continuous looking up contacts (I forget the exact message and cannot test it right now).  If I removed the "/", it works as expected.  Thanks - John

Somehow I must have mangled the patches.  Even though I had the latest and correct patch, the webdav source had the old patch text.  My apologies - John

Alas, after all this work, Evolution still cannot create new Contacts.  Zimbra keeps misinterpreting them as Type 8 (Documents) instead of Type 6 (Contacts) when it enters them into the mail_items database table.  We fought for hours to get the NGINX web proxy to rewrite the Evolution given URI to include the supposedly required format parameter (?fmt=vcf).  We finally got this to work and could see the properly formated creation request in the Zimbra logs but it still misinterprets it as Type 8.  We have opened an incident with Zimbra.

Ah, I think I found it! This is part of a response from Zimbra tech support:

###
To create a Contact oject of type 6 with vCard, WebDAV client should set the
content-type header to text/vcard in the create request.
###

Evolution is sending type text/x-vcard which is deprecated:

PUT /dav/jsullivan@pacifera.com/Contacts/209318A3-42B63567-2A649F80.vcf HTTP/1.1

Host: zimbra.pacifera.com

User-Agent: Evolution/2.28.3.1

If-None-Match: *

Content-Type: text/x-vcard

Authorization: Basic <something>
Content-Length: 361



BEGIN:VCARD

VERSION:3.0

EMAIL;X-EVOLUTION-UI-SLOT=1;TYPE=WORK:final@mycompany.com
X-MOZILLA-HTML:FALSE

X-EVOLUTION-VIDEO-URL:

FBURL:

CALURI:

X-EVOLUTION-BLOG-URL:

X-EVOLUTION-FILE-AS:Contact\, Final

N:Contact;Final;;;

FN:Final  Contact

NOTE:

X-EVOLUTION-SPOUSE:

NICKNAME:

X-EVOLUTION-ASSISTANT:

X-EVOLUTION-MANAGER:

ROLE:

TITLE:

URL:

END:VCARDHTTP/1.1 201 Created

Server: nginx

Date: Wed, 12 May 2010 19:22:11 GMT

Connection: keep-alive

ETag: "541679-541679"


Thus, this is probably related to https://bugzilla.gnome.org/show_bug.cgi?id=566330

Changing this to text/vcard may redeem all this work.  The only remaining problem is the 75 second delay when creating the contact.  We are still awaiting work from Zimbra on that one - John

That sounds like a Zimbra bug then because there is no such MIME type.  text/x-vcard became text/directory.

Interesting.  I see this comment in the Wikipedia VCard article (http://en.wikipedia.org/wiki/VCard):

Note: The Internet media type text/vcard was used (incorrectly) in some SyncML Documentation to refer to vCard 3.0, and the example was followed in various implementations.

nono, bug #566330 is the other way around, it's what is evolution-data-server's webdav backend willing to read, not what it uses for writing. I tried to change all the "x-vcard" (quotes for clarity only) strings in e-book-backend-webdav.c file of evolution-data-server earlier (there left only one for me, in the upload_contact function), but it didn't help to me. Maybe I've too old Zzzimbra server.

John, could you try to do the change in your sources, and with your server, please? If it'll not help, then try with "text/directory", as stated in [1]? Though draft [2] deprecates text/directory as well, thus maybe my Zzzimbra server is really too old.

[1] http://tools.ietf.org/html/rfc2426
[2] http://tools.ietf.org/html/draft-ietf-vcarddav-vcardrev-06

I'm certainly willing to do that but I'm not particularly familiar with the code nor am I really a developer at all.  Would you mind pointing me to the pertinent portions of the code so I don't mangle it by mistake.  Thanks - John

Created attachment 162009 [details]
eds test patch

for evolution-data-server;

You can try with this test patch. See what file it changes (addressbook/backends/webdav/e-book-backend-webdav.c), and there you can make your other changes as well. I'm not sure how it'll apply to your evolution-data-server sources, but I hope it'll be OK. If not, you can always open that file and do a simple replace of all "text/x-vcard" to "text/vcard" yourself (quotes for clarity only).

The patch worked perfectly and there were no other places where I needed to change the file.  Unfortunately, it still created a mail_item entry  with type=8.  I sent the packet trace showing the correct Content-Type header entry, i.e., set to text/vcard to Zimbra tech support and we'll see what they say.  Thanks very much - John

(In reply to comment #21)
> That sounds like a Zimbra bug then because there is no such MIME type. 
> text/x-vcard became text/directory.

Actually, I think Zimbra is correct here.  I was ready to send them the links referenced further down to show that they were in error but thankfully read them first.  From http://tools.ietf.org/html/draft-ietf-vcarddav-vcardrev-06
3. MIME Type Registration


   To:  ietf-types@iana.org

   Subject:  Registration of media type text/vcard

   Type name:  text

   Subtype name:  vcard

   Required parameters:  none

   Optional parameters:  charset

   Encoding considerations:  The "charset" MIME parameter is interpreted
      as defined in [RFC2046], section 4.1.2.  If it is omitted, the
      default encoding is UTF-8 as defined in [RFC3629].

   Security considerations:  See Section 11.

   Interoperability considerations:  The text/vcard media type is
      intended to identify vCard data of any version.  There are older
      specifications of vCard [RFC2426][oldreference_VCARD] still in
      common use.  While these formats are similar, they are not
      strictly compatible.  In general, it is necessary to inspect the
      value of the VERSION property (see Section 7.7.10) for identifying the standard to which a given vCard object conforms.

      In addition, the following media types are known to have been used
      to refer to vCard data.  They should be considered deprecated in
      favor of text/vcard.

      *  text/directory

      *  text/directory; type=vcard

      *  text/x-vcard

Moreover, the CardDAV spec calls for it.  From http://tools.ietf.org/search/draft-ietf-vcarddav-carddav-10#section-6.2.2

Description:  The CARDDAV:supported-address-data property is used to
      specify the media type supported for the address object resources
      contained in a given address book collection (e.g., vCard version
      3.0).  Any attempt by the client to store address object resources
      with a media type not listed in this property MUST result in an
      error, with the CARDDAV:supported-address-data precondition
      (Section 6.3.2.1) being violated.  In the absence of this property
      the server MUST only accept data with the media type "text/vcard"
      and vCard version 3.0, and clients can assume that.

   Definition:

       <!ELEMENT supported-address-data (address-data-type+)>

       <!ELEMENT address-data-type EMPTY>
       <!ATTLIST address-data-type content-type CDATA "text/vcard"
                             version CDATA "3.0">
       <!-- content-type value: a MIME media type -->
       <!-- version value: a version string -->



   Example:

       <C:supported-address-data
          xmlns:C="urn:ietf:params:xml:ns:carddav">
         <C:address-data-type content-type="text/vcard" version="3.0"/>
       </C:supported-address-data>

I'm not an expert but it looks like we do need to change Evolution to bring it into conformance with the specs - John

(In reply to comment #27)
> I'm not an expert but it looks like we do need to change Evolution to bring it
> into conformance with the specs - John

OK, but even with that and a test patch from comment #25 you said it doesn't work in comment #26, so we are still waiting on their input, if I understand this correctly.

Yes, that is correct.  Yes, they have been responding thankfully.  They reiterated that having ContentType=text/vcard in the header should work.  I sent back the traces showing that, when we set Content-Type: text/vcard, it still did not work.  We'll see how they respond - John

The specific problem of creating Zimbra database entries with the wrong type value has been fixed in Zimbra 6.0.7. I can confirm that in my environment.  The only remaining problem is the 75 second delay when creating new contacts or appointments.

We are getting very close on the 75 second problem, too.  That is the last remaining issue as far as we can see.

The 75 second delay is coming from the NGINX proxy installed with Zimbra.  It defaults keepalive_timeout to 75 seconds.  In contrast, davical's NGINX configuration sets it to 0.  There may be a good reason related to the behavior of Zimbra Web Client why Zimbra defaults their NGINX keepalive_timeout to 75 seconds.

However, the fact that the keepalive_timeout is even kicking in seems to indicate a lower level problem.  Why is the connection not being closed so that it needs to be timed out?

As I reviewed yet again the traces we've taken of Evolution/Zimbra interaction, as expected, it is Evolution which initiates the TCP connection to create a new record.  Why is Evolution not closing the connection after it receives the 201 Creation Successful message from the DAV PUT? I'm quite ignorant on the subject but this smells more like an Evolution problem than a Zimbra problem.  Shouldn't Evolution close the TCP connection when it is finished? I can't imagine Zimbra would send a FIN to a session it has not initiated but I'm not sure.  Thanks - John

We have been able to work around the 75 second delay problem by changing the NGINX settings.  Under /opt/zimbra/conf/nginx/templates we edited nginx.conf.web.https.template and nginx.conf.web.http.template by setting keepalive_timeout 0; within the server section, before the location block and then running zmnginxctl stop ; zmnginxctl start.

With the above Zimbra reconfiguration, these two patches and the patch from https://bugzilla.gnome.org/show_bug.cgi?id=566330, Evolution and Zimbra appear to be working other than Free/Busy.  When I try that, I get an error messaged that the Calendars have failed and I see this on stderr:

(evolution:32409): calendar-gui-WARNING **: Unable to access free/busy url: Operation not supported

(evolution:32409): calendar-gui-WARNING **: Unable to access free/busy url: Operation not supported

(evolution:32409): libebook-WARNING **: e_book_dispose: Exception releasing remote book interface!

Free/Busy retrieval us set to http://zimbra.pacifera.com/home/%u@%d/?fmt=ifb which works perfectly fine in Kontact.

Have the above three patches been included in 2.30? Thanks - John

For the first yes, see comment #12. The second is just a test patch, it's not there yet. But as you confirmed its usability, then I'll commit it for 2.30.3 too. See bug #566330 comment #4, it's in 2.30.2 too.

For the text/vcard content type patch:
Created commit 20aa2b4 in eds master (2.31.5+)
Created commit 531766b in eds gnome-2-30 (2.30.3+)

We received this very helpful feedback from the Zimbra devs:

***
This header needs to be set in HTTP request by WebDAV client.

> > Connection: close

This header tells the server to close connection after handling the current HTTP request.  Absent of this header means the server can use keep-alive, which is part of HTTP/1.1 spec.  

The way they disabled keep-alive was to edit the server configuration.  That will work in that configuration, but won't work against other ZCS install, or other servers that support keep-alive.  keep-alive is ubiquitous on HTTP servers, so I'm somewhat puzzled as for why they are running into open connection issue with ZCS install only.  Perhaps Apple Calendar server does not support keep-alive.  But if keep-alive was the issue, they will surely run into this issue again with other servers.
***

Thus, I suppose we work better with other systems more by accident than by design.  It would seem it is our responsibility to set this in the header to ensure the connection is closed.  Is that a relatively trivial task?

Thanks for the update, but could you open a new bug report for this issue, as we moved far far away from the original report? Feel free to add a link to it in here.

OK - new bug report at https://bugzilla.gnome.org/show_bug.cgi?id=623936
Thanks - John