1

I'm experiencing significant problems with the time it's taking to deliver signup emails. The signup process requires the user to verify their email address before they can use the site (it's financial services). However, I am finding that the signup emails are taking a long time to deliver. I'd not been measuring them, but last week, one took 53mins.

So, I've written some code to send emails to test accounts and then login to the imap servers to check when they arrive. The results were surprising.

Out of about 60 mails send over the last 30 hours (1 each hour to two destination addresses), 9 took more than 10 seconds, 6 took more than a minute (i.e. 10%).

+----+-------------+----------------+---------------------+---------------------+------+
| id | sender      | receiver       | sent                | received            | time |
+----+-------------+----------------+---------------------+---------------------+------+
| 31 | <redacted>  | btinternet.com | 2015-06-28 16:15:01 | 2015-06-28 16:16:36 |   95 |
| 33 | <redacted>  | gmail.com      | 2015-06-28 17:15:01 | 2015-06-28 17:21:35 |  393 |
| 49 | <redacted>  | btinternet.com | 2015-06-29 01:15:01 | 2015-06-29 01:19:09 |  248 |
| 60 | <redacted>  | gmail.com      | 2015-06-29 07:15:02 | 2015-06-29 07:18:02 |  180 |
| 67 | <redacted>  | btinternet.com | 2015-06-29 11:15:01 | 2015-06-29 11:15:14 |   12 |
| 69 | <redacted>  | btinternet.com | 2015-06-29 13:15:01 | 2015-06-29 13:15:15 |   13 |
| 70 | <redacted>  | gmail.com      | 2015-06-29 13:15:01 | 2015-06-29 13:15:25 |   23 |
| 76 | <redacted>  | gmail.com      | 2015-06-29 17:15:02 | 2015-06-29 17:16:55 |  113 |
+----+-------------+----------------+---------------------+---------------------+------+

Is this normal? What tactics do website developers use to deal with this sort of problem? Do I need to completely re-architect the user sign on process?

UPDATE

So here is the header from two of the mails (entry 49 and 33 in the table) and also the 53minute one. Running them through https://toolbox.googleapps.com/apps/messageheader/analyzeheader puts the blame at the handover point from the remailer/smtp provider to gmail/btinternet - i.e. difficult to say who is responsible. I've also removed entry 78 as that looks to have been a server issue at my end - I should possibly update the code to download the messages and check the headers too... but really I just want this to all to just work without delays.

Entry 49 header:

From noreply@mycompany.com Mon Jun 29 01:15:02 2015
X-Apparently-To: someone@btinternet.com; Mon, 29 Jun 2015 01:19:04 +0000
Return-Path: <bounce-md_30227788.55909c16.v1-23f454369b374fe2aef0d80ce6364ab9@mandrillapp.com>
Received-SPF: neutral (65.20.0.12 is neither permitted nor denied by domain of mandrillapp.com)
X-YMailISG: J0ShZ4sWLDvICB6llYYfaaallq.RgZj2clqpD9Ae_l0rDuLz
 zL98Iz9m7BzvCHQtdaJw0SGa3pMO84r5ksFrUp2ZNU_raBEZo_BSmVPF2W.i
 lVjOm75fUYT4cNnopu90X0fb.HjrWJchvnp9_LKSHbrrc.PupJ6fZg6VNP3S
 zBahV4K7HN8FTOEw4E2RFXSL_IhGaLLm8LrYFKV1gdwE0c8KPNKJyivdUDXL
 C6gCALRoTwClwmUtmV6hXc1.bOXZbxF29lF_09IyWlbcBZ7JQlETizZbon6E
 l4nOcnQKP_YzMoPcVxV1pFo5YhGP2ZhCeKii.9TiWTYcYt5i_NRf6x6sO5Dr
 MNt88B6Nte20gI2FH.O6YJE6uIuc3CX6bpvM6Kpp32_reub6DT77M94f6jWR
 6Y1LUFcNRS0kASy33.IO8fsSDtTUpftLi1a.rP37S05BNafWxelBGQrV2xhP
 FaBYrIdqGrSLsr5v.GsKnAc1zDXFGgjJtq68I0OIMEqIk.BIhKtm3jKlHVnm
 RP9K.w_CxSa0mHrW0oLd_04X1lk2UtDgPgwSnEWPW8x_8Enbd0TALqCbDqbz
 db.RSyd3Z.SBOV99._SDqamE8ZsaMTsUAqnOgDhYwhPjhWZWB2onY9dQFtbV
 XZYzKfdhbt4N8Ba7K9L40sYjTQEMVdFS114C8_eMDuoU7nCqXAUzGdtuozza
 11ksVuD7udtarAUTnf4IhbRBNVdxN.SCv2T1mSnqDwVZ_oYjGfAMj7UuDyT9
 CYrtiAeZ5N0pV90Brt7o8mQLX6maPaC65tMXMMIZ5OwhgZovwQZ6HenSojrZ
 fm1VNlSybha0uJAr_dWT8ZMm7P2mSXF_4VTOEQkL8bdfmkBVyFj77.iEWifl
 dGg.bp8HZ6SNky0ygarmaOYqu6dBGylZ3RcTaRQt1k9up_Wb5xBq3MAjy.TR
 O3vup044WppkIYvn3zMetqZRarCkgIm2ASygRoWrJC_yYLaHUYO8_1OHAL6c
 hs8RyXJtfUyVmJkgyXQ3pQBoqxKBodjnsVSg_9eZYLOIz7HTcE.ARIrI28Cn
 GUC63U9KIxv8HG5PZ9Eb6bHBImtip3osk_pCTp_3YOHk54oZsGOZ.DKY3RDW
 Xo0xDd0Q1vaDJjIN6fEMoGYm.ERP5SaqZhyX_DqwWge6iTf5oXihcZk4dlB4
 AfLWT8ws4AMFNN85GG2ej7FOzm6arJEFNgKmEloW
X-Originating-IP: [65.20.0.12]
Authentication-Results: mta1052.bt.mail.ir2.yahoo.com  from=mycompany.com; domainkeys=fail (bad sig);  from=mandrillapp.com; dkim=pass (ok)
Received: from 127.0.0.1  (EHLO rgin01.bt.int.cpcloud.co.uk) (65.20.0.12)
  by mta1052.bt.mail.ir2.yahoo.com with SMTP; Mon, 29 Jun 2015 01:19:04 +0000
X-OWM-Source-IP: 198.2.135.23(US)
X-OWM-Env-Sender: bounce-md_30227788.55909c16.v1-23f454369b374fe2aef0d80ce6364ab9@mandrillapp.com
X-CTCH-RefID: str=0001.0A0B0202.55909D05.0065:SCCMA14590,ss=1,re=-4.000,recu=0.000,reip=0.000,cl=1,cld=1,fgs=0
X-CTCH-VOD: Unknown
X-CTCH-Spam: Unknown
Received: from mail135-23.atl141.mandrillapp.com (198.2.135.23) by rgin01.bt.int.cpcloud.co.uk (8.6.122.06)
        id 558929EC0F1014AB for someone@btinternet.com; Mon, 29 Jun 2015 02:19:03 +0100
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=mandrill; d=mycompany.com;
 h=From:Subject:To:Message-Id:Date:MIME-Version:Content-Type:Content-Transfer-Encoding; i=noreply@mycompany.com;
 bh=zQ8Pw0yC68z7YFU75ZLqAeJgrbQ=;
 b=jQIwrQ8Igg9mFKpDMTmFwJT1qZrC4Upn8jRNHQIy6dBmKeE1K8KsX8RnOQvpR3iABLRKpbBacjFZ
   KrqCiGhEm/pUBDiA2kmG+Kr2pHE6ok5DuZ8WSj32Tgow+Q0fLegw0QHTmt4IunQ46azhIcZNAGs4
   LvEb4XZca4TycMOL3bg=
DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=mandrill; d=mycompany.com;
 b=CBtnU6eKV2hNgHZWFFHYslDJzGFNttqrOq9Wd9QvlnaLcomkL1Q86iWEufDzw/TFpKZ7cLlDYU0M
   JttFxyIApsaDP9SvJG3rwLIRsEzUsmF2WAV9dvyKgGwTgIQuKewj+vRzFWhCgRK/BswVipZrBzWt
   X7IStV90RGjgLr3bebU=;
Received: from pmta03.mandrill.prod.atl01.rsglab.com (127.0.0.1) by mail135-23.atl141.mandrillapp.com id hi2ecq1sau8o for <someone@btinternet.com>; Mon, 29 Jun 2015 01:15:02 +0000 (envelope-from <bounce-md_30227788.55909c16.v1-23f454369b374fe2aef0d80ce6364ab9@mandrillapp.com>)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandrillapp.com; 
 i=@mandrillapp.com; q=dns/txt; s=mandrill; t=1435540502; h=From : 
 Subject : To : Message-Id : Date : MIME-Version : Content-Type : 
 Content-Transfer-Encoding : From : Subject : Date : X-Mandrill-User : 
 List-Unsubscribe; bh=vaAklnLD4Wop1a0xzgxeP374IDp2vRnwUHbtCbGK/qk=; 
 b=Qp/eUBN9eRAGCVf8nsNTqQ8YTRenIrBBOFBa/guh1ntQNQx/JwsVKD3pTqR17kSzcCkl9n
 8YISxk0TkElvqgrJHTswSzT09Wztz3iGfCMrEIjPl6MlSIdkPHm9rhFMkB6s1sq6m8CFAw4c
 HhWaBT+1gyE0Ymx+JQcqFHBsYyEGg=
From: <noreply@mycompany.com>
Subject: Test Email Delivery Time 48
Received: from [2.24.158.23] by mandrillapp.com id 23f454369b374fe2aef0d80ce6364ab9; Mon, 29 Jun 2015 01:15:02 +0000
To: <someone@btinternet.com>
Message-Id: <E1Z9Nf3-0008Rm-Pr@sully>
X-Report-Abuse: Please forward a copy of this message, including all headers, to abuse@mandrill.com
X-Report-Abuse: You can also report abuse here: http://mandrillapp.com/contact/abuse?id=30227788.23f454369b374fe2aef0d80ce6364ab9
X-Mandrill-User: md_30227788
Date: Mon, 29 Jun 2015 01:15:02 +0000
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Length: 10

And the entry 33 header:

Delivered-To: someone@gmail.com
Received: by 10.27.85.232 with SMTP id l101csp1085170wli;
        Sun, 28 Jun 2015 10:21:30 -0700 (PDT)
X-Received: by 10.170.150.7 with SMTP id r7mr14269483ykc.48.1435512089846;
        Sun, 28 Jun 2015 10:21:29 -0700 (PDT)
Return-Path: <bounce-md_30227788.55902b96.v1-110a60e49eb840e6bc3f42a6c6a34d2d@mandrillapp.com>
Received: from mail135-23.atl141.mandrillapp.com (mail135-23.atl141.mandrillapp.com. [198.2.135.23])
        by mx.google.com with ESMTPS id g63si17330189ywd.182.2015.06.28.10.21.29
        for <someone@gmail.com>
        (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Sun, 28 Jun 2015 10:21:29 -0700 (PDT)
Received-SPF: pass (google.com: domain of bounce-md_30227788.55902b96.v1-110a60e49eb840e6bc3f42a6c6a34d2d@mandrillapp.com designates 198.2.135.23 as permitted sender) client-ip=198.2.135.23;
Authentication-Results: mx.google.com;
       spf=pass (google.com: domain of bounce-md_30227788.55902b96.v1-110a60e49eb840e6bc3f42a6c6a34d2d@mandrillapp.com designates 198.2.135.23 as permitted sender) smtp.mail=bounce-md_30227788.55902b96.v1-110a60e49eb840e6bc3f42a6c6a34d2d@mandrillapp.com;
       dkim=pass header.i=@mycompany.com
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=mandrill; d=mycompany.com;
 h=From:Subject:To:Message-Id:Date:MIME-Version:Content-Type:Content-Transfer-Encoding; i=noreply@mycompany.com;
 bh=zQ8Pw0yC68z7YFU75ZLqAeJgrbQ=;
 b=F0ICKYMvut01JvIgbHMU+yy4KxmymXwGuzIsR987h7iioNigtJmZm8HDW7FHEYsowWSizRD+1gR+
   cBsK9wPgrpaA1AtX03OZDDpXX2PI2AXR57YQMofzev4In3TdWptagwJh4+n9vWDkryh2SK8Mr7Pq
   M+U2TbuTonN+td6NRc0=
DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=mandrill; d=mycompany.com;
 b=GkcT5CE/f77U0tvpJZkO4SdzBh+RmQOZOhrg8OsT8/Lu1zab7NcaOioV4/R5WM571x4PgnJiExUn
   DcPsx632we4ntNZA+0CpcDfHOVmWw2mbMyrJEN7ZWGKD0dHhVSm/6FBmsHud0z5MXXwKMGBGI3x0
   JaTCIcCnWtq4N2uhezI=;
Received: from pmta03.mandrill.prod.atl01.rsglab.com (127.0.0.1) by mail135-23.atl141.mandrillapp.com id hi0mhi1sau8n for <someone@gmail.com>; Sun, 28 Jun 2015 17:15:02 +0000 (envelope-from <bounce-md_30227788.55902b96.v1-110a60e49eb840e6bc3f42a6c6a34d2d@mandrillapp.com>)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandrillapp.com; 
 i=@mandrillapp.com; q=dns/txt; s=mandrill; t=1435511702; h=From : 
 Subject : To : Message-Id : Date : MIME-Version : Content-Type : 
 Content-Transfer-Encoding : From : Subject : Date : X-Mandrill-User : 
 List-Unsubscribe; bh=vaAklnLD4Wop1a0xzgxeP374IDp2vRnwUHbtCbGK/qk=; 
 b=U6C4/OZHtnnOpBb4epphFoosDvWdXo/bLtq4QRB3td7cnAUkwM0nSuwXYqKuQGNVyJwsym
 39qghN90QoTFajJ7F9juNHCJ6NUWC124KJ/yxDzq1z0Hv97RiMEC2SKl/YreLyT1r078zTW2
 iJRW9TlCUQ05Jsag3HWn0H471Tx6w=
From: <noreply@mycompany.com>
Subject: Test Email Delivery Time 32
Return-Path: <bounce-md_30227788.55902b96.v1-110a60e49eb840e6bc3f42a6c6a34d2d@mandrillapp.com>
Received: from [2.24.158.21] by mandrillapp.com id 110a60e49eb840e6bc3f42a6c6a34d2d; Sun, 28 Jun 2015 17:15:02 +0000
To: <someone@gmail.com>
Message-Id: <E1Z9GAX-0005np-9t@sully>
X-Report-Abuse: Please forward a copy of this message, including all headers, to abuse@mandrill.com
X-Report-Abuse: You can also report abuse here: http://mandrillapp.com/contact/abuse?id=30227788.110a60e49eb840e6bc3f42a6c6a34d2d
X-Mandrill-User: md_30227788
Date: Sun, 28 Jun 2015 17:15:02 +0000
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Body text

And the 53 minute one...

Delivered-To: someone@gmail.com
Received: by 10.27.85.232 with SMTP id l101csp604598wli;
        Thu, 25 Jun 2015 07:56:08 -0700 (PDT)
X-Received: by 10.13.241.132 with SMTP id a126mr58052631ywf.72.1435244168411;
        Thu, 25 Jun 2015 07:56:08 -0700 (PDT)
Return-Path: <bounce-md_30227788.558c0a35.v1-8e4d31d6475d49d6b0ed117d59b3028c@mandrillapp.com>
Received: from mail133-4.atl131.mandrillapp.com (mail133-4.atl131.mandrillapp.com. [198.2.133.4])
        by mx.google.com with ESMTPS id 17si11578070ykz.152.2015.06.25.07.56.06
        for <someone@gmail.com>
        (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Thu, 25 Jun 2015 07:56:08 -0700 (PDT)
Received-SPF: pass (google.com: domain of bounce-md_30227788.558c0a35.v1-8e4d31d6475d49d6b0ed117d59b3028c@mandrillapp.com designates 198.2.133.4 as permitted sender) client-ip=198.2.133.4;
Authentication-Results: mx.google.com;
       spf=pass (google.com: domain of bounce-md_30227788.558c0a35.v1-8e4d31d6475d49d6b0ed117d59b3028c@mandrillapp.com designates 198.2.133.4 as permitted sender) smtp.mail=bounce-md_30227788.558c0a35.v1-8e4d31d6475d49d6b0ed117d59b3028c@mandrillapp.com;
       dkim=pass header.i=@mycompany.com
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=mandrill; d=mycompany.com;
 h=From:Subject:To:Message-Id:Date:MIME-Version:Content-Type:Content-Transfer-Encoding; i=welcome@mycompany.com;
 bh=rGv/9CsUhFfFFLLNRNlvI02wtoY=;
 b=UtNcwgANOAQdiV/OU/fJM4n23s35oifYqg/4WxGQwpnNxtYJVNSfshykgT4ASDfJEpFxbfSW3l9P
   bBfgXBDGemNFUee4QB8eSijEW92AUZhEE4WZJfITJZh4oSd+xl8mgz+3h47BWWJ+Y1NcRxOd1zpl
   EAte1O2MsjlNqiuBZKc=
DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=mandrill; d=mycompany.com;
 b=mevW4aYMx/+I8x3xoHpJhgkDYnxD5wWRPg0Q9vXVJgicY+KhxCdDUz8BH+CTHh2s9+e1MzhI/t1q
   ZuWg5REc+T73j0dKnDdTo8CfZjSwCSRJcpFU6UM6L34ZD0MKCuX0hf1ncviXAL4kR5YvlyQD4Tal
   +OhEiJ54Wyw8ewUej9g=;
Received: from pmta02.mandrill.prod.atl01.rsglab.com (127.0.0.1) by mail133-4.atl131.mandrillapp.com id hhgb8e1sar8p for <someone@gmail.com>; Thu, 25 Jun 2015 14:03:34 +0000 (envelope-from <bounce-md_30227788.558c0a35.v1-8e4d31d6475d49d6b0ed117d59b3028c@mandrillapp.com>)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandrillapp.com; 
 i=@mandrillapp.com; q=dns/txt; s=mandrill; t=1435241014; h=From : 
 Subject : To : Message-Id : Date : MIME-Version : Content-Type : 
 Content-Transfer-Encoding : From : Subject : Date : X-Mandrill-User : 
 List-Unsubscribe; bh=sAD9YepWMF6uFlTityzN4EPGSnQxaDjQf+K2eZzmKYM=; 
 b=JzpORb+ZgeBYvaBwHK0iQkmKOhNQyMcgflMGqvWoFi+QMa4AucJUaizvXbd1gSpnjTFlCo
 UK8emzHL++vwDLQeqxTjzMiuipfYOTt+O6le2UqYm73zFWPVPvD1NMcMXcVSNlVAcnvCS6kv
 kG+4hZvbHuofoYlAeIpBUau+AJ9g0=
From: <welcome@mycompany.com>
Subject: Welcome
Return-Path: <bounce-md_30227788.558c0a35.v1-8e4d31d6475d49d6b0ed117d59b3028c@mandrillapp.com>
Received: from [178.62.119.45] by mandrillapp.com id 8e4d31d6475d49d6b0ed117d59b3028c; Thu, 25 Jun 2015 14:03:33 +0000
X-Php-Originating-Script: 33:misc.php
To: <someone@gmail.com>
Message-Id: <E1Z87kb-00056E-Ea@cc-cc>
X-Report-Abuse: Please forward a copy of this message, including all headers, to abuse@mandrill.com
X-Report-Abuse: You can also report abuse here: http://mandrillapp.com/contact/abuse?id=30227788.8e4d31d6475d49d6b0ed117d59b3028c
X-Mandrill-User: md_30227788
Date: Thu, 25 Jun 2015 14:03:33 +0000
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
| improve this question | | | | |
  • 1
    You can use Mandrill - mandrill.com. They have bunch of libraries that you can use based on your app language - mandrillapp.com/docs/integrations.html. – Zulhilmi Zainudin Jun 29 '15 at 21:02
  • Yup, that's what lots of people have said, and I did, and why I am now asking this question. – jmc Jun 29 '15 at 21:12
  • To be clear, I'm using their SMTP relaying - are you suggesting the delays will go away if I use their API's? But in terms of the actual provider, I'm using them, that's where the numbers above came from. – jmc Jun 29 '15 at 21:16
  • 1
    There is a lot that can go into this, and 53 minutes is not that unusual, however, it is not normal either. Do they provide any more granular information? For example, sometimes inbound or outbound queues can get a bit backed up. It could be an anti-spam/virus mechanism. Does your provider give you better clues?? Also, it could be the target networks. Is there a solid pattern to this? – closetnoc Jun 29 '15 at 21:46
  • 2
    Whilst email can often be "quite quick", it was never intended to be an "instant" method of communication. Sites that require email validation often state to "allow upto 15 mins" for the email to arrive (although admittedly this might be down to their own infrastructure also). However, most sites don't necessarily require email validation before the user can do anything. Perhaps allow "limited" access for an initial period of time before validation? – MrWhite Jun 29 '15 at 23:52

Browse other questions tagged or ask your own question.