IMAP IDLE and TCP resetting

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

IMAP IDLE and TCP resetting

Gnome Evolution - General mailing list
Hello everybody!

Today I discovered that Evolution's IMAP IDLE support is unreliable if a server
resets TCP connection. Although in such case, Evolution reconnects to the
server, but no longer uses the IDLE command. So a user loses their instant
notifications of new emails until Evolution is restarted. I see no IDLE commands
in Wireshark after that RST packet. However, periodic updates are performed.

Is this something specific for my config (Evolution 3.34.1 from Ubuntu 19.10)?

---
I am not subscribed to the list, please copy your replies to me directly. I sent
this email twice because first time it was canceled. If you receive a duplicate,
sorry about that.


_______________________________________________
evolution-list mailing list
[hidden email]
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-list
Reply | Threaded
Open this post in threaded view
|

Re: IMAP IDLE and TCP resetting

Gnome Evolution - General mailing list
On Sun, 2020-04-19 at 20:23 +0300, Nicholas Guriev via evolution-list
wrote:
> So a user loses their instant notifications of new emails until
> Evolution is restarted.

        Hi,
as far as I remember, the IDLE is started whenever a connection exists
and it has nothing better to do. That is, you do not need to restart
Evolution, you can simply move to a different folder, select a message,
which had not been downloaded yet, or even have the periodic check run,
after which the IDLE will be started.

> Is this something specific for my config (Evolution 3.34.1 from
> Ubuntu 19.10)?

I think there had been done something about this, but I do not see any
directly related change in the IMAPx code since your version, thus it
can be there are (still) some corner cases, where this misbehaves.

It's also that the account itself can be in offline mode (it is
indicated in the Mail view of Evolution, in the folder list, with an
icon on the right of the account name) and it stays in that until
something asks for an online operation.

        Bye,
        Milan

P.S.: you can see what is going on between the server and the Evolution
when you run it from a terminal like this:

   $ CAMEL_DEBUG=imapx:io evolution

aka no need for wireshark, which might not work with encrypted
connections anyway.

_______________________________________________
evolution-list mailing list
[hidden email]
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-list
Reply | Threaded
Open this post in threaded view
|

Re: IMAP IDLE and TCP resetting

Gnome Evolution - General mailing list
On Пн, 2020-04-20 at 10:44 +0200, Milan Crha wrote:
> as far as I remember, the IDLE is started whenever a connection exists
> and it has nothing better to do. That is, you do not need to restart
> Evolution, you can simply move to a different folder, select a message,
> which had not been downloaded yet, or even have the periodic check run,
> after which the IDLE will be started.

Unfortunately, it seems not enough to continue idling. Although, the
desired actions are performed and even regular pollings occur, at the
end of operation I see no IDLE command. I had reduced number of
concurrent connections from 3 to 1, to eliminate possible interference.
This did not help.

> It's also that the account itself can be in offline mode (it is
> indicated in the Mail view of Evolution, in the folder list, with an
> icon on the right of the account name) and it stays in that until
> something asks for an online operation.

That is not the case. Evolution is in online mode. At least nothing in
GUI indicates opposite.

> P.S.: you can see what is going on between the server and the Evolution
> when you run it from a terminal like this:
>
>    $ CAMEL_DEBUG=imapx:io evolution
>
> aka no need for wireshark, which might not work with encrypted
> connections anyway.

Thank you for the debugging hint. But Wireshark is more powerful in
watching network communications, and it can even decrypt the traffic if
it gets session keys. When the SSLKEYLOGFILE environment variable is
set, NSS writes all the keys into a file that is then read by Wireshark.

https://wiki.wireshark.org/TLS#Using_the_.28Pre.29-Master-Secret

> > Is this something specific for my config (Evolution 3.34.1 from
> > Ubuntu 19.10)?
>
> I think there had been done something about this, but I do not see any
> directly related change in the IMAPx code since your version, thus it
> can be there are (still) some corner cases, where this misbehaves.

Okay, I will try to investigate the trouble later (not before the
weekend) whether it is reproducible with the latest version.


_______________________________________________
evolution-list mailing list
[hidden email]
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-list
Reply | Threaded
Open this post in threaded view
|

Re: IMAP IDLE and TCP resetting

Patrick O'Callaghan
On Tue, 2020-04-21 at 21:00 +0300, Nicholas Guriev via evolution-list
wrote:
> That is not the case. Evolution is in online mode. At least nothing in
> GUI indicates opposite.

Have you tried clicking on the connection icon to put Evolution offline
and then on again?

poc

_______________________________________________
evolution-list mailing list
[hidden email]
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-list
Reply | Threaded
Open this post in threaded view
|

Re: IMAP IDLE and TCP resetting

Gnome Evolution - General mailing list
In reply to this post by Gnome Evolution - General mailing list
I can confirm that the issue is present in Evolution 3.37.1 compiled from
source. After abnormal disconnect, Evolution still thinks that there is an
idling connection and no longer tries emit IDLE command. When I turn Evolution
into offline mode and then online, imapx_disconnect_sync and imapx_connect_sync
functions are called. The idle_mailboxes hash in camel-imapx-conn-manager.c is
therefore cleaned twice, and the mailer may use IDLE again.

If I do imapx_conn_manager_clear_mailboxes_hashes by hand via gdb, the idling is
continuing. But alas, I do not know proper place in code where this function
should be called.

To force disconnecting, I use ss(8) utility. This results in the same state, if
a server would reset IMAP connection.

    sudo ss -K dst 77.88.21.125 dport = 993 -n


_______________________________________________
evolution-list mailing list
[hidden email]
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-list
Reply | Threaded
Open this post in threaded view
|

Re: IMAP IDLE and TCP resetting

Patrick O'Callaghan
On Mon, 2020-05-04 at 18:09 +0300, Nicholas Guriev via evolution-list
wrote:
> I can confirm that the issue is present in Evolution 3.37.1 compiled from
> source.

[Please follow the usual quoting conventions when replying]

Note that 3.37 is an unreleased test version. Try to repeat your
experiment with the currently supported version (3.36).

poc

_______________________________________________
evolution-list mailing list
[hidden email]
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-list
Reply | Threaded
Open this post in threaded view
|

Re: IMAP IDLE and TCP resetting

Gnome Evolution - General mailing list
On Tue, 2020-05-05 at 09:37 +0100, Patrick O'Callaghan wrote:
> Note that 3.37 is an unreleased test version. Try to repeat your
> experiment with the currently supported version (3.36).

        Hi,
I'd say, in this particular case, it won't help. The code is basically
the same in both versions. That Nicholas tried the development version
is a good thing, from my point of view, trying the latest code
available, which I appreciate.

Nicholas, it would be helpful to have filled a bug against IMAPx in
evolution-data-server [1], to not have this lost, though I do not want
to force that. I'll try to cook a fix and commit it to the sources in
the following days (unless I forget).
        Thanks and bye,
        Milan

[1] https://gitlab.gnome.org/GNOME/evolution-data-server/issues/new

_______________________________________________
evolution-list mailing list
[hidden email]
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-list
Reply | Threaded
Open this post in threaded view
|

Re: IMAP IDLE and TCP resetting

Gnome Evolution - General mailing list
On Tue, 2020-05-05 at 11:17 +0200, Milan Crha wrote:
> Nicholas, it would be helpful to have filled a bug against IMAPx in
> evolution-data-server [1], to not have this lost, though I do not want
> to force that. I'll try to cook a fix and commit it to the sources in
> the following days (unless I forget).

Okay, a new issue against evolution-data-server has been opened. Thank you so
much for the help.

https://gitlab.gnome.org/GNOME/evolution-data-server/-/issues/211

_______________________________________________
evolution-list mailing list
[hidden email]
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-list