Phine Solutions web work notes

Ok this one was really a thinker. I spent at least 5 hours to get this to work and finally I was able to use Sendmail to relay through my Gmail account.

A little background:

I have a Linux VPS with CentOS installed. The only email MTA is the default Sendmail. Everything else is pretty much the standard CentOS 4 installation. I don’t intend to use this box as a mail server or any other type of email processor. What I was trying to do is to add some basic capability to send out email from the box using my existing email accounts hosted in Gmail. And I didn’t want to install any additional software such as Postfix for this.

That being said, let me continue to take you down the path that I have gone through, without the stumbling blocks.

My approach was basically: problem -> Google for solutions -> trouble shoot -> Google again. So I found a lot of useful content on the web during the process.

1. Check sendmail

Since gmail uses TLS, you will need to make sure your sendmail is compiled with TLS (for encryption) and SASL (for authentication). This is the command to use to check it:

/usr/sbin/sendmail -d0.1 -bv root

In my case, sendmail does have the necessary compilation flags so I was good. If yours doesn’t, you’ll need to re-compile sendmail and update the binary that is used to start the sendmail service, which is not covered here.

2. Upgrading Cyrus SASL

If your SASL installation doesn’t have the “plain” and “login” lib you will have authentication problem with Gmail. You can see why when you get to the sendmail configuration in the later steps. The common error in the /var/log/maillog is this:

AUTH=client, available mechanisms do not fulfill requirements

It was a vague error and I was so frustrated with it at one point I was ready to give up. However, this article about setting up Postfix with Gmail casted some light and helped me figured out the cause.

The problem is that SASL doesn’t have all the necessary plugins. The “login” and “plain” are the plugins necessary to talk to Gmail smtp. So I had to upgrade SASL to fix the problem. Here is what I did:

$ wget http://ftp.andrew.cmu.edu/pub/cyrus-mail/cyrus-sasl-2.1.21.tar.gz
$ tar -xzf cyrus-sasl-2.1.21.tar.gz
$ cd cyrus-sasl-2.1.21
$ ./configure
$ make
$ make install

$ mv /usr/lib/sasl2 /usr/lib/sasl2.orig
$ ln -s /usr/local/lib/sasl2 /usr/lib/sasl2

Since I just switched out the old sasl2 lib without recompiling sendmail, I was concerned sendmail would poop during runtime. Luckily that didn’t happen. Dynamic lib rocks!

3. Generate SSL certificate

I made a directory called certs under /etc/mail. Here are the commands that I used to generate the SSL certificates.

openssl req -new -x509 -keyout cakey.pem -out cacert.pem -days 3650
openssl req -nodes -new -x509 -keyout sendmail.pem -out sendmail.pem -days 3650

Notice I made the certificates good for almost 10 years. I didn’t needed the cacert.pem to be exact.

I also copied /usr/share/ssl/ca-bundle.crt to /ect/mail/certs and included it in the sendmail configuration file. Other wise you’ll see some error like this:

unable to get local issuer certificate

The reason is that the ca bundle file has the Gmail certificate issuer. Although I read it somewhere that email still goes out with this error. Nonetheless, we don’t need to see this if we can fix it.

4. Configure sendmail

With the preparations above we are ready to configure sendmail. I found this tutorial very useful in terms of getting the correct sendmail configurations.

In summary, I have the /etc/mail/auth/client-info looks like this:

AuthInfo:smtp.gmail.com “U:root” “I:username@gmail.com” “P:password” “M:PLAIN”
AuthInfo:smtp.gmail.com:587 “U:root” “I:username@gmail.com” “P:password” “M:PLAIN”

If you use Gmail hosted email with your own domain name, you will have username@hostname.tld in there.

Make sure run:

$ makemap -r hash client-info.db < client-info

and chmod 600 on client info files.

Essential lines in my sendmail.mc

FEATURE(`authinfo’,`hash /etc/mail/auth/client-info.db’)dnl
define(`SMART_HOST’,`smtp.gmail.com’)dnl
define(`RELAY_MAILER_ARGS’, `TCP $h 587′)
define(`ESMTP_MAILER_ARGS’, `TCP $h 587′)

define(`CERT_DIR’, `/etc/mail/certs’)
define(`confCACERT_PATH’, `CERT_DIR’)
define(`confCACERT’, `CERT_DIR/ca-bundle.crt’)
define(`confCRL’, `CERT_DIR/ca-bundle.crt’)
define(`confSERVER_CERT’, `CERT_DIR/sendmail.pem’)
define(`confSERVER_KEY’, `CERT_DIR/sendmail.pem’)
define(`confCLIENT_CERT’, `CERT_DIR/sendmail.pem’)
define(`confCLIENT_KEY’, `CERT_DIR/sendmail.pem’)

define(`confAUTH_MECHANISMS’, `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN’)
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN’)

The certificate files are generated/copied from the previous step. I’m no sendmail expert so the configuration lines may not be perfect. But it works. Let me know if you have better settings.

One tip I found very useful is to use the debugging feature. You can set a high log level in the sendmail.mc to see at which step sendmail choked and for what reason.

Also, make sure run “make” or m4 every time you touch the sendmail.mc.

m4 sendmail.mc > sendmail.cf

So that’s pretty much it. I restart the sendmail service and out goes my email.

Depending on the type of the technology you use to build your web sites, there are different ways to put your code out there. Since a lot of sites today are developed in PHP, updating probably means upload a bunch of scripts to the server.

One way to do this is uploading the whole directory from the site root and switch the Apache site directory using symbolic link. But if you only have a handful of files updated it is really not necessary to upload everything every time. And often times there are user uploaded content and log files generated by the web server in the file system, which you don’t want to lose during the process.

Another way that I prefer is using the “sync” methodology. Basically I use a tool to compare the differences between my local development drive and the remote directory, and let the tool handle the remote copying and deleting.

“rsync” it is a great tool from *nix family which can sync two locations using ssh protocol. Although it is command line based you can always write a simple script to automate it. But if you are developing from a Windows PC it might be a bit difficult. I used to run rsync under Cygwin on XP and it does very well backing up files from the remote server. However because of the different ways of handling file permission under Cygwin and Windows I had a lot of problem to commit to rsync as a deployment tool.

For a long time I also used a tool called “Site Publisher” from helexis.com. It is a small FTP based tool which you can use to set up different site profiles and sync the code from your local drive to remote directory. It has worked very well for me but there are a couple of issues that made me to look for new ones:

• No sftp support
• Since I have firewall installed I have to use “Active” mode for FTP. But Site Publisher seems to have some problem with this. It would hang during a transfer session, which is not acceptable for a production release.

Recently I found “InstantSync(TM)” from sitedesigner.com and have been very happy about it. It support sftp so I can probably shutdown the FTP server on my host. It also supports multiple site profiles and the file transmission has been rock solid. Although it cost $99 I think it is definitely worth the money.

During the search process I also evaulated TurboFTP and SynchronEX which didn’t fit my needs. In my opinion, TurboFTP is primarily a FTP tool so there are a lot of features that are great for FTP but not necessarily in my case. I just want something simple to use and does this one thing good. SynchronEX looks promising but its user interface to set up site profile is hard for me to comprehend.

Disclaimer: The tools reviews mentioned above are solely based on personal experience and I have no affiliation with the companies mentioned.