No internet connection
  1. Home
  2. Ideas

Threading for email notifications

By Michael Lynch @michael
    2022-06-21 19:55:34.115Z

    I'd love it if TalkYard would make it possible for email clients to create email threads that matched the forum threads.

    So if I had email notifications enabled, and there were 5 posts in topic A, and 3 posts in topic B, and 6 posts in Topic C, I'd have 14 total emails, but my email client could group them into three email threads (one thread for each topic).

    So in my email client, the threads would look something like this:

    To do this, TY would have to use subject lines that matched the forum thread and set the In-Reply-To and References email headers to tell the email client which emails are associated with which forum threads.

    When I'm reading TY notification emails, I'd like to follow the thread, but the current implementation makes each notification totally independent, so it creates a lot of clutter. I often see the last message in a thread where the user says their issue is resolved, but then I have to go through all the emails to find all the notifications that were associated with that particular forum thread.

    It looks like part of the motivation for the current implementation was to address needs of users like @chrscheuer, but I'd love to find a way to manage noise from email notifications while also enabling email clients to organize them well.

    • 9 replies
    1. C
      Christian Scheuer @chrscheuer
        2022-06-21 20:07:12.153Z

        I'd love this as well :)

        1. CChristian Scheuer @chrscheuer
            2022-06-21 20:08:28.541Z

            My issue was with the email threads being based on "notification type" which made no sense. Having an option to send emails with subjects matching each Talkyard page would be a great option.
            Ideally each user could decide which format of the two they prefer.

          • In reply tomichael:

            This would be nice for me too. And then, with one email thread per forum topic, it'd be simpler to implement Reply-via-email too (later if anyone wants this).

            Thanks for the mockup, and link to the header docs — I didn't know about In-Reply-To and References. (I've been surprised by how my email client (Gmail) groups emails seemingly by looking at [server.addr] only but I suppose these headers help it to do the right thing)

            1. Michael Lynch @michael
                2022-08-23 22:02:28.212Z

                Just wanted to check in to see if there's been any forward progress or timeline for this issue.

                Our posting volume has increased, and threaded email conversations would make it a lot easier to manage these conversations.

                1. Not yet started. Will have been fixed in the middle of September I think. I just posted a mini roadmap: Mini roadmap, fall 2022

                  (Sorry for the noisy threads)

              • In reply tomichael:
                KajMagnus @KajMagnus2022-09-06 18:53:18.150Z2022-09-06 19:01:08.992Z

                Threading for emails — here's what's on my mind, implementation wise:

                For each [new page notification] being sent, Ty generates a SMTP Message-ID, and includes in the email. Later, when notifications are sent about comments on that page, then, for each person being notified, [LKP:] Ty looks up all previous notifications s/he got, about that page, gets their Message-IDs, sorts by time, and includes in the References SMTP header.

                The lookup [LKP:] above could be done slightly differently: Ty could look up only all ancestor comments, and include only those in the References: ... header. Instead of all comments on the same page. — That is, sub threads on the same page. I wonder what mail clients do then: different "threads", but all of them starting with the same Message-ID (for the Orig Post).

                And Ty also sets In-Reply-To: ... to the Message-ID about the parent comment (which would also ... always? be last in the References: ... list), or the whole page's Message-ID if it's the Orig Post.

                And the subject line would, I suppose, have to include the original title of the page, even if, later on, it got renamed, because apparently Gmail groups by email subject line instead (see: https://stackoverflow.com/a/23482109). — So, for each person getting notified, maybe Ty would need to remember the original title of that email thread ?

                ***

                I wonder if there are security implications from generating these Message-IDs per post, or [per person getting notified]. If it matters if everyone who gets notified about a post, gets the same Message-ID. Or if it's somehow better, if everyone see unique Message-IDs. ... ?

                Any thoughts?

                One thing could be that if both person P and person Q got notified about comment C, then, Q could send a handcrafted email to P, with a Message-ID specifying comment C (if P and Q see the same ID). In this way, Q could try to inject Q's own email in the email thread as seen by P. Then, maybe maybe an email client shows Q's message as part of the email thread, and P mistakenly thinks that it got sent by the Ty server but it's from Q.

                ( @chrscheuer I'll do the may-@mention-me thing first, just nice to start thinking about this threading stuff now directly :- ))

                1. Michael Lynch @michael
                    2022-09-06 19:16:36.955Z

                    Ty looks up all previous notifications s/he got, about that page, gets their Message-IDs, sorts by time, and includes in the References SMTP header.

                    I don't think you'd have to keep track of which messages each recipient has received. It should be sufficient for Ty to store a unique Message-ID for each message in a thread, and then whenever there's a reply, you walk up the tree of responses and add their Message-ID to the References header until you get to the root post of the thread.

                    If a mail client has never seen some of the message IDs referenced in a References header, that should be fine. I think mail clients will just ignore the unrecognized ones.

                    I wonder if there are security implications from generating these Message-IDs per post, or per person. If it matters if everyone who gets notified about a post, gets the same Message-ID. Or if it's somehow better, if everyone see unique Message-IDs. ... ?

                    That doesn't strike me as a security liability, but you could try sending messages from Google Groups / Mailchimp / Topicbox and see if they generate a unique Message-ID per recipient. If they don't, it's likely pretty low risk.

                    One thing could be that if both person P and person Q got notified about comment C, then, Q could send a handcrafted email to P, with a Message-ID specifying comment C. In this way, Q could try to inject Q's own email in P's email thread. Then, maybe maybe an email client shows Q's message as part of the email thread, and P mistakenly thinks that it got sent by the Ty server.

                    Yeah, but there's not much advantage in doing that. Q can already send the same email to P by posting a real comment to Ty, which will generate a message to victim P. That attack also requires Q to know P's email address.

                    And the subject line would, I suppose, have to include the original title of the page, even if, later on, it got renamed, because apparently Gmail groups by email subject line instead (see: https://stackoverflow.com/a/23482109). — So, for each person getting notified, maybe Ty would need to remember the original title of that email thread ?

                    This is unnecessary (for my use case, at least). If I change the subject of the thread on Ty, I'm fine with that breaking the threading in my mail client.

                    1. Thanks! Yes, reading about Message-ID it seems the servers usually send the same ID to everyone. At the same time, someone wrote: "the message id may be the only information your get back from spam notifications or bounces" https://stackoverflow.com/a/831560 so I guess that could be a good thing. Anyway implementation wise, it barely matters.

                      Google Groups / Mailchimp / Topicbox and see if they generate a unique Message-ID per recipient

                      Sounds like a good idea.

                      Q can already send the same email to P by posting a real comment to Ty, which will generate a message to victim P.

                      Yes, at the same time, in the future the Ty server might have some dangerous-link detection and block such messages. Or Q might send a message with angry words to P, which the server staff cannot see, but P might think they read and approved Q's angry message. But this seems a bit far fetched, at least today.

                      Edit: Hmm, using per-email unique Message-IDs, to simplify [looking into bounces], seems to be a thing, here's Postmarkapp:

                      https://postmarkapp.com/blog/using-messageid-to-retrieve-bounces

                      users can use the messageID field on the Bounces API to retrieve a specific bounce [...]
                      The messageID is the string returned for each successfully sent email through the API

                      1. Michael Lynch @michael
                          2022-09-06 20:42:20.118Z

                          Edit: Hmm, using per-email unique Message-IDs, to simplify [looking into bounces], seems to be a thing, here's Postmarkapp:

                          https://postmarkapp.com/blog/using-messageid-to-retrieve-bounces

                          The messageID they're talking about is a Postmark-specific field that they add to the JSON payload for email notifications. It's independent from the Message-ID header in the actual message.

                    2. Progress
                      with doing this idea
                    3. @KajMagnus marked this topic as Planned 2022-06-22 12:40:15.914Z.