There was some twittering yesterday about the possibility of Amazon’s Simple Email Service being added to Magento, so that outbound emails could enjoy the high level of deliverability that comes from ESP’s (Email service providers) like Mailchimp or Campaign Monitor.
Update July 2013: I have since released a premium extension for sending email using Amazon SES called MageSend, if you’re having toruble sending email with Magento, please check it out, it was created to solve many common Magento email issues.
I decided to investigate what it would take to add the functionality to my Magento Email extension, SMTP Pro, and the good news is, not too much. I’ll run through what’s changed in SMTP Pro and how the changes were made in this article. If you don’t actually care, just jump straight to the downloads to get the new version, or get it on Magento Connect.
We use Google Apps accounts as our store email accounts and I have no complaints about the service at all. Deliverability is excellent, reliability has never been an issue (how often is gmail down, after all?). However, it’s comforting to know that if something changed, we could now swap to Amazon’s equally reliable, scalable, pay-per-use option quickly and easily.
Implementing the changes
I was able to integrate Magento to SES in a very short amount of time thanks mostly to the work of Christopher Valles, he made a Zend transport for SES, which I was able to plug in to my extension quite simply. Christopher, if I’m ever back in Spain, I’ll buy you a beer.
Asides from the SES API, the extension just needed a few UI tweaks, and to better handle the slightly different paradigm that comes with a non-SMTP email transport (mostly in self testing).
So it hasn’t changed too drastically – ready to give it a go? Read on for set up instructions.
Setting up Simple Email Service
Getting the SES up and running is really easy. Here’s what you need to do.
1) Signup Signup for SES (and if you haven’t already, AWS)
2) Download Tools Get the SES command-line tools. They’re Perl scripts, so I’m going to assume you have a Perl environment installed to run them.
3) Verify Email Verify your test email address(s) using this command:
ses-verify-email-address.pl -v youremail@gmail.com |
Note: You can set an environment variable to save having to specify the location of your AWS credentials all the time.
export AWS_CREDENTIALS_FILE=aws-credentials |
You should get an email like this, follow the link to verify:
4) Test it Run a quick test
ses-send-email.pl -k aws-credentials -s "This is a test" -f you@yourdomain.com you@yourdomain.com < msgbody.txt |
Assuming that worked, you should have the SES side of things all set up. Well done!
Setting up the Extension
To use SES with my SMTP Pro extension first make sure you know your access key and secret key.
Input these into the section in System-Configuration-System-SES settings.
Make sure you choose SES (experimental) as the extension option in the General Settings section too.
You’ll need to make sure the contact form email address is verified with SES (System-Configuration-Contacts), and also the contact form sender address (System-Configuration-Store Email Addresses). In my testing I just made all of these the same address for simplicity.
If you get this part wrong, it’ll break the email sending and self-test. It is important for SES during development as all emails sent to & received from need to be verified. Once you’re in SES production mode that won’t be required.
A symptom of getting it wrong is an error like this in var/log/exception.log:
<ErrorResponse xmlns="http://ses.amazonaws.com/doc/2010-12-01/"> <Error> <Type>Sender</Type> <Code>MessageRejected</Code> <Message>Email address is not verified.</Message> </Error> <RequestId>0da2c23a-52a0-11e0-8fd7-f3adea62ee04</RequestId> </ErrorResponse> |
Once you have it all set up you should be able to run a self test, if should give you output like this:
You should also receive the two test emails like these:
If you see these emails with the amazonses.com mailed-by header, then everything worked, yah! – now post a comment to tell me it worked so I can release without the (experimental) clause!
Downloads
As usual the SMTP Pro 1.4.3 release is available directly from my site, and also on Magento Connect.
Please let me know any feedback about the new service. Particularly if you would like to use it in production. Also, thinking I should rename the SMTP Pro extension to something more like Magento Email Pro now that it’s not just SMTP – anyone care to weigh in on that?
Pretty fantastic. I will give this a try after running a couple of tests and let you know the results. I’ve recenetly switched over to a new server only to find out my new IP has been flagged by some ISPs in a passed life.
Hi, Yash – great reason to switch to SES very interested to hear how it goes. Please report back with results!
Failed to connect to SMTP server. Reason: php_network_getaddresses: getaddrinfo failed: Name or service not known(0)???
@Tiago I need more information – what settings are you using? What server are you trying to connect to?
Hi Asyley,
I have some problem with smtp pro. i have insert all the info require and when i press on run self test.
it show
ASchroder.com SMTP Pro Self-test results
Testing outbound connectivity to Server:
Using Google Apps/Gmail configuration options
Connection to Host SMTP server successful.
Sending test email to your contact form address contact@flingson.com:
Unable to send test email. Exception message was: Unable to connect via TLS…
Please check and double check your username and password.
Contact Form test email did not use SMTPPro to send.
Testing failed, please review the reported problems and if you need further help visit ASchroder.com to contact me.
i have double check the pass and user and is not wrong.
i have also asked my server admin to enable TLS on my server. but it seems that is still not working. any idea?
im using magento 1.5.0.1 and smtppro 1.4.3
@Nicholas can you please try running these two commands from your server:
telnet smtp.gmail.com 465
and
telnet smtp.gmail.com 587
Both should connect OK – but that PHP message hints that one or both will not.
Can it handle throttling?
Amazon SES allows me to send a limited amount of email per second, as well a top of email per day. So, I would like to configure the module in a way that if I send 10,000 emails in a few seconds it should be able to catch them, store them and send them 1 to 2 per second and spaced in several days, sending 2000 per day in order to comply with the Amazon quota.
Is that possible?
Thanks
Diego
Hi Diego, It does not handle throttling like you describe. That would require a server-side store of emails, and quite a bit more complicated throttling logic. It would be better I think to just not send 10000 all at once!
hi ashley, is me again.
i have asked my server administrator to run the command and he told me that i cant telnet smtp.gmail.com for sure as it will get blocked.
Hi, Nicholas – It’s hard to tell if you server admin is saying the ports (465, 587) are definitely blocked at his end (which means you can’t use gmail to send), or if he thinks he can’t telnet to smtp.gmail.com on those (open) ports and won’t try. Either way, if he’s not forthcoming with help on this, I suggest you get a decent magetno host and move on.
I am using this through Amazon SES and it is sending emails from my store just fine. The self test failed as not verified (even though I did verify the send email with Amazon) but I still got the test email. Then I switched my SES service to production and everything is working. Customers are getting emails and the header shows it is coming from Amazon SES.
hi ashley.
i was thingking to change to other host too but the one you suggest, their server is kinda slow for me as im located at south east asia area. Do you have any good server in asia for suggestion?
Hi Ashley,great job ,I was looking up this for several days,your extension must be what I am looking for.Do you have any suggest or solution for the newsletter subscribe ?We all know the Magento Community Version don’t have the user name blank to let them submit with their Email account.Also do you have any suggest and solution on the autoresponder Email managment or if it is possible just use Magento’s basic email management function as the autoresponder.Thanks man !
@Merlin See Mail Chimp – they get my vote.
@Ashley sorry,I didn’t tell you what I really mean. What I am doing is trying to setup my very own autoresponder service for my Magento Store.I have Advanced Newsletter and Follow up Email all from Aheadworks. The last thing I am looking for a email sending solution,I think Amazon SES will do the job especially with your new version SMTP Pro. Do you think this is a plan?I am not technqie like you guys, from my understanding,the autoresponder have 3 parts ,the newsletter subscribe ,the email list and email sequence management and the email sending service(SMTP).Am I heading to the right direction?
Been looking for this for a while, will try with SES, SendGrid and Gmail and let you know how it goes on Magento Enterprise.
@Ashley
This is one of the best extensions I’ve come across, should be part of native Magento. However I have 2 requests:
1. Increase support for sendgrid.com – this is an amazing service, for those that don’t know, its like Mailchimp but for transactional emails, outstanding webhook technology and fantastic API. However, when using it as SMTP with this module, Magento emails come through uncategorised, and it’s difficult to break down statistics by the type of email – this is easily done by adding the following code to the email header:
X-SMTPAPI: {“category”: “Email Type”}
where Email Type is Forgot Password, Welcome Email etc… this allows me to see for e.g. if New Order emails have a higher open rate than Shipment emails, then i might charge a premium for banner ads in my New Order emails.
As you know we cannot modify email headers via Magento’s Email Template editor so it would be great if the module did this automatically.
2. I have not tried SES yet… I’m so caught up in the endless possibilities of now having APi access to opens/clicks/reports on Magento transactional emails… however, it would be nice if it supported Mailchimp’s SES integration, so that we can see all the stats of the SES email campaigns in Mailchimp.
I believe this is done by Magento sending the emails via the Mailchimp API instead of via SES, so long as the Mailchimp account is integrated with SES… more on that here: http://blog.mailchimp.com/mailchimp-launches-transactional-email-service-on-top-of-amazon-ses/
Both sound like good feature requests to me. I shall investigate the API’s and if possible add this to the next release, thanks for letting me know about them.
Thanks for this great extension. Well written and works out of the box even with Google Apps domain aliases.
thanks Ashley, SES works just perfect! Just testing in the admin area did not work for SES, but with SES on production the emails are send and received!
there is one little bug: If you have setup an e-mail address to get a copy of all order/shipping/etc-confirmation mails via BCC, it does not get delivered to the BCC address. If you setup magento to send the copy of the confirmation mails via separate e-mail, SES is working fine again…
Cheers!
Thanks for the heads up Markus!
Hi Ashley,
is it possible to get this extension to work with the contact form of magento as i am having no luck at the minute?
Ian, have you checked for extension clashes with extension made for that? If only one type of email is not sending, it might be due to an extension clash.
Ashley,
Just got setup with SES on a development box today. Works great! The one feature request I would have, if possible, is to include the option to add DKIM authentication. Right now emails are showing up with the “via” message in the header.
More info here: http://docs.amazonwebservices.com/ses/latest/DeveloperGuide/index.html?DKIM.html
Thanks again!
Thanks for the heads up, consider it on the roadmap.
Thanks Marcus and Ashley ofcourse! that did the trick, wasn’t receiving anything on BCC
I have the same issue as @Scott mentioned above. The self test reports a failure:
ASchroder.com SMTP Pro Self-test results
Testing outbound connectivity to Server:
Connection to Amazon SES server not tested (…yet)
Sending test email to your contact form address colemanc@*****:
Unable to send test email. Exception message was: Sender MessageRejected Email address is not verified. d3b5f152-2059-11e1-81d6-0dcf36f85d9f …
Please check and double check your username and password.
Contact Form test email used SMTPPro to send correctly.
Testing failed, please review the reported problems and if you need further help visit ASchroder.com to contact me.
(I substituted the asterisks for my domain to prevent SPAM)
I’ve verified that both the sending email address and recipient are validated in Amazon SES and I >>> do <<< successfully receive the email in my Inbox from the correct sender with correct subject, message body, etc.
My command line tests all work without issue as well.
Any ideas?
I have installed your extension and it is working great with SES. Thank you!
There is one bug that I think might be present. When an order confirmation is e-mailed, we used to get a Bcc of that email (as set up at Configuration > Sales Emails > Send Order Email Copy To). Now the customer is getting sent a confirmation, but the Bcc is not being sent to us.
Any ideas?
I’m using CE 1.6.1 and getting the same results as Chris.
I got it working. I ended up adding a signing certificate. This is done at AWS in IAM > Users > /your username/ > Security Credentials. Although the self test still produced the same error as @Chris above, I received the test e-mail this time. It appears to be working in my CE 1.6.1 install.
I noticed that your email address is hard-coded as the ‘from’ address in line 91 of the IndexController.php.
This causes the SES test email to fail because the ‘from’ address needs to be an authorized sender. When I manually change the line to be ‘from’ one of my authorized senders, everything works.
I’m not a Magento expert, but I assume pulling in the ‘from’ email address from the website settings would fix some of the issues people are describing above.
Thanks for setting up this module – it’s very helpful.
Hi,
Thanks for your useful module but have you added the attachment support to it as well or not. Any plans for it?
Thanks
@Kevin to fix that you need to go to Configuration > Sales > Sales Emails and change the drop down menu Send Order Email Copy Method from Bcc to Separate Email as Marcus suggested earlier – it does then work.
I was getting an error that the test mail wasn’t sending using SMTP PRO. I followed the instructions on this blog (another post) on how to remove a conflict line from the Ebizmarts Mailchimp extension and that fixed the problem. It’s important to note that you need to address this clash even if you’ve deactivated the Mailchimp extension (at least that’s our experience).
Looking forward to the Bcc error being fixed in future versions and hopefully the introduction of DKIM authentication.
We are now running SES on our live site.
Thanks Ashley.
Amazon ses limit 10,000 emails / day , 5 emails/second
Normally , it will work or not in the newsletter of magento ?
public function scheduledSend($schedule)
{
$countOfQueue = 3;
$countOfSubscritions = 20;
Thanks.
Made some mods to this excellent plugin to get DKIM working.
In the zip files are three files which you need to put in the lib folder
/app/code/community/Aschroder/SMTPPro/lib
The file AmazonSES.php is already there and will have to be replaced with the version in the zip file …
Link to the zip archive below
http://www.filedropper.com/aschroedersesdkim
In the file dkim-settings.php there are the instructions as well as the settings. Hopefully that will be all you need to get the DKIM thing working
The most politically correct way to do it is to ofcourse
1. Put all DKIM settings in Magento Backend (for the php retarded)
2. Override this code using the usual Magento gas of config files and directory structure.
I will however wait for someone else to produce a more politically correct version of this mod.
Ok, first of all thanks for such an amazing extension.
For those who are getting MessageRejected error, go to IndexController.php and on Line 91 type your verified email id.
If you guys have production account, it’ll anyway work without any problem.
Now for all those who are getting BCC error and aren’t receiving mails on BCC id, just change send method from BCC to separate mails. It will work
Thanks
divyanshbajaj@gmail.com
i would like to get you to install the extension and setup ses perfectly on our site to get it up and running. pls contact me when possible.
Hi, It works with 1.7.0.2?
Thx.
ASchroder.com SMTP Pro Self-test results
Testing outbound connectivity to Server:
Connection to Amazon SES server not tested (…yet)
Sending test email to your contact form address dadubinin1@gmail.com:
Unable to send test email. Exception message was: Sender RequestExpired Request timestamp: Tue, 21 May 2013 01:06:19 +0000 expired. It must be within 300 secs/ of server time. b46353da-c1a1-11e2-a61e-1b797f39bb9d …
Please check and double check your username and password.
Contact Form test email did not use SMTPPro to send.
Testing failed, please review the reported problems and if you need further help visit ASchroder.com to contact me.
Please help me
the problem was the time on the server
thank you!