Phine Solutions web work notes

Being a half-baked JavaScript developer for a long time it wasn’t easy for me to grasp the concept like “selectors”, “traversing” and “utilities” in JQuery, and every time I started to write something using JavaScript it was always a series of events of Googling for sample code, checking the API, and Googling again for the right syntax if it doesn’t.

Recently I found this book “JQuery In Action” from Amazon and I was quite pleased after reading it. The information in the book is well organized and illustrated. And the authors explain the concept before digging into syntax and examples, so you not only know how to do code in JQuery, but most importantly, why to do it in this way.

My favorite subjects in this book are “understanding browser event models” (chapter 4) and “JavaScript that you need to know but might now”, the later is actually an appendix. They fill the gaps in my JavaScript knowledge and brought some new but important perspective on how it really works.

The lab code is great and it will probably help me quite a bit for my future projects.

So if you want to get serious about JavaScript programming, get this book. It’s definitely worth the read.

I had a couple of files on my Macbook which I accidentally deleted without knowing it until a couple of days ago. They are quite important files and without them I might have to spend hours to recover the loss. So I gave the Time Machine a try. It was simply amazing. I went into the Time Machine and typed in the key words that I vaguely remembered and hoped for the best. The Time Machine started to search all the saved points and in seconds it found those files out and I was able to restored them.

I keep my files on my machines regularly backed up, but I honestly can’t think of anything else that can do this good job on keeping different backup points of the system and make them easy to retrieve. I guess this is especially useful for me since I love doing cleaning up on file that I don’t recognize .

So I ran into this issue. It appears under certain set up, IE will reject a session cookie, or randomly drops them. I use PHP session to store user authentication information and some of my IE users were not very happy about not being able to log in or getting kicked out of a login session randomly.

The biggest challenge is to duplicate the issue. IE’s security and privacy settings can both affect the way it handles the cookie, and even the same settings on different PCs can have the different result. It is absolutely frustrating to have users shouting at you “This doesn’t work!” while everything works perfectly fine on my own machine.

After a lot of research I ended up completely abandon the session approach and use a database back end to handle the user authentication sessions. A regular cookie is still necessary to store a token (ex. a user id) to identify the user, but instead of using the SESSION variable, I use a database table to store the user token and the last access time. This will certainly complicate the setup – a cron job has to be installed to perform the database clean up; and the increasing the database queries – basically for each access from a “tokened” user I’ll need to update the last access time – will also add some tolls to the performance. However the payoff is also obvious. I don’t have to worry about the different IEs out there, who may behaves differently and sometimes in unexpected mysterious ways thanks to Microsoft, to reject my delicious cookies.

Just for the sake of discussion, there may be a number of reasons that cause IE to reject session cookie:

1. To many cookies from a site, or IE runs out of temporary space. There is a max number of cookies that a site can send. I think 20 is the limit. A lot of good discussions can be found on this thread.

2. IE’s P3P implementation might also have something with it. This is a good article about IE’s Eye of Sauron. There are some suggestions to bypass this by adding a P3P header, which is as simple as a line of characters to your pages. I don’t like the idea mostly because this is not the right way to fix a problem which shouldn’t exist in the first place.

3. There might be a slightly chance that your server time stamp is out of sync. For example when you set a regular cookie use something like “time() + 28400″ in your PHP code to specify the expiration time, wrong time stamp might put your cookie in the history already. This issue is a lot easier to spot since none of your sookies work, not just the session cookies.

It’s interesting. Firefox has problem loading a page with this line in the html header:

<script src=”http://maps.google.com/maps?file=api&amp;v=2&amp;key=google_api_key” type=”text/javascript”></script>

At first I was suspecting Google server was down but it turned out the page would load just fine if I disable (suspend) Firebug extension (1.3.3).

However if I put the line toward the end of the page as inline Javascript everything seems fine even with Firebug on.

I have a Buffalo LinkStation on my network which I use as a back up for all my house hold PCs. Recently I was doing some house cleaning on the drive and saw a “trashbox” folder in each configured shared drive, and it stores all the previously deleted items from shared drive. The “recycle bin” feature can be disabled by going to the LinkStation configuration page, and uncheck the feature in each shared drive which is quite simple. But deleting the actual “trashbox” turned out to be a painful process in Vista.

Like the way I perform file deletion everyday, I right clicked on the folder and select “delete” from Vista. A usual popup window started and showed the progress. However it seems Vista can’t make up its mind in terms of how many files are there to delete and how long it’s going to take. The information changed back and forth and the deletion process went on and on. At some point it became painfully slow and I started to suspect there was something wrong with my network, or the shared drive.

Looking at the number of minutes left on the progress window really frustrated me so I decided to use my MacBook to give it a try.

It’s pretty simple to connect to a shared drive from a Mac. Since AppleTalk protocol was enabled in the drive Mac recognized the IP and connected to it immediately. Deleting the files was also a lot smoother. Mac didn’t change its mind on how many files it needed to delete and the process went just like it promised.

I believe the LinkStation is actually Linux based and used Samba for file sharing protocol. Maybe because of the nature of the OS, Mac handles this kind of device better. But Vista was just disappointing. WIth the flux of the rich web applications today, a reliable and simple OS which has great attentions to details will be the ultimate winner.

This morning I wanted to search some Ringling Brothers tickets and I hit the big G. For some reason almost all the top search results have this warning “This site may harm your computer”. And a warning page would display if I click through it.

This is weird. I know those sites are very legit. So I tried some other keywords it seemed all the organic search results on the top are “labeled”. Using another data center showed the same result.

This became very annoying very quickly. This is probably a Google glitch. With people increasingly relying on search engine to look up information maintaining the service quality becomes more and more important.

Update:

Here is why.

I have been ignoring the sitemap for a long time. It is nice and easy for the Wordpress powered site since there is plugins for that but for other sites I have to write a script to do it. Finally I decided to add it.

Google has a nice sitemap generation tool here. It is written in Python but don’t be scared away if you don’t know Python. I don’t. And it took me about 2 hours of work to set it up (including other scripts that I had to do). Here is what I did:

1. Wrote a PHP script to generate multiple URL files. They are basically text files which have URLs in them. It is one of the ways to feed the Python script to generate the file sitemap.xml. There are other ways too, but this one is the most suitable choice for my situation. The “feed”, among others, can be configured through a config.xml file and the Python script will pick it up. So there is no need to mess with the Python code at all, at least at this point.

2. First I ran my own URL generation script, and then the Py fella. The script can be run from the command line and at the end it will automagically ping Google to notify the updated sitemap.xml. Oh, did I mention you have to have Python installed on your server? I really don’t see why it’s not there if it is one of the Unix cusines but it doesn’t hurt to verify it.

3. I manually submitted the sitemap.xml to Yahoo and Live search. If you haven’t signed up your site there you’ll need to login (assuming you have an account) and add your site.

4. After some staring at my Unix console I started thinking, “since the Python script can notify Google, why not the other two?” So I dig a little bit into the Python script. Although I didn’t know the language it didn’t take me long to find out the location of “NOTIFICATION_SITES”. It is an array that has the notification list of the sites. This is what you can change it to to make it ping Yahoo and MSN/Live as well (while they are still two companies).

NOTIFICATION_SITES = [
('http', 'www.google.com', 'webmasters/sitemaps/ping', {}, '', 'sitemap'),
('http', 'webmaster.live.com', 'ping.aspx', {}, '', 'siteMap'),
('http', 'search.yahooapis.com', 'SiteExplorerService/V1/ping', {}, '', 'sitemap')
]

5. Combine these command lines into a shell script and stick it into the cron.

So there you have it. Now I am just waiting to see the floor of traffic that will be brought in by using a sitemap.

This past weekend my PC was infected by malware and basically it keeps opening the Firefox windows with different websites. After spending hours on how to remove it I gave up. Frankly I think a lot of those sypwares or malwares are created by people who sells the spyware removal software but it is another subject.

So I  upgraded to Vista hoping for a more secured OS.

I have read a lot of negative reviews about Vista and I was always hesitant to upgrade but so far the experience has been quite positive.

The PC I am running Vista on has Intel Dual core processor and 3G ram on it so I didn’t feel a slight slowdown while running applications on Vista. As a matter of fact, I feel everything is a bit faster than XP.

The graphic interface is quite nice and there are certainly a lot of improvements. It seems to me Microsoft also borrowed some ideas from Max OS X, which is not necessarily a bad thing.

A lot of people are complaining about the Vista’s User Account Control (UAC) security prompt. Basically you get a lot of popups when you try to install something or run something that needs to open a port or changes system files. Yes it is annoying but I am not too much bothered by it, as long as it really helps to protect my system (fingers crossed ).

The areas that I think can be improved:

Why so many versions? “Home”, “Business”, “Premium”, “Ultimate”…they are just confusing. The pricing ladder can probably help Microsoft targeting different users and make more money but too many version also create more work on marketing and support.

The UAC is there for a reason but it can be designed better. I know it is not Unix so “sudo” command is not an option but maybe something cleverer so less users will choose to disable it after installation?

Sometimes Vista feels clunky to use. Possiblly because it is designed to incorporate the new features and the old ones so there are many ways to get to the same destination. Simplicity is the ultimate sophistication and Mac OS shows. Maybe WIndows 7 addresses this issue better?

I have only used it for a week so I can’t say for sure Vista’s improved security really delivers. Hopefully I didn’t waste my time and money to upgrade.

Update:

One big hassle that I ran into using Vista is the “safely remove the USB drive” feature. I have a Seagate portable hard drive and I used to use the “safely remove hardware” icon on the toolbar in XP to make sure the data doesn’t get corrupted when unplugging it from USB port. But the feature doesn’t work well in Vista. The icon is still available but Vista always tells me “it is in use and close any program or window that might be using the device…”. I tried some suggestions such as disabling the caching but it didn’t work. Finally I found this Drive Tool page that really saved my time. Thank you Uwe Sieber!

Basically this is what I am trying to do here:

SELECT * FROM Table1 WHERE Table1.col1 = ‘bob’ AND Table1.col3 IN (SELECT Table2.col2 FROM Table2 WHERE Table2.col1 = ‘alice’)

Both Table1 and Table2 have the same column with the name “col1″. And the value of Table1.col3 should match values coming from Table2.col2.

However the query always returns empty set, even there are definitely data matches via the IN where clause. As a matter of fact, if I use the values directly from the subquery the whole query would work.

At a first glance, this looks like a correlated subquery issue but it’s not. Even Table1 and Table2 share the same column name, the subquery is not referring to any Table1 column in the where clause, so it shouldn’t be an issue explained here.

Anyone has some insight on this?

Ok this one was really a thinker. I spent at least 5 hours to get this to work and finally I was able to use Sendmail to relay through my Gmail account.

A little background:

I have a Linux VPS with CentOS installed. The only email MTA is the default Sendmail. Everything else is pretty much the standard CentOS 4 installation. I don’t intend to use this box as a mail server or any other type of email processor. What I was trying to do is to add some basic capability to send out email from the box using my existing email accounts hosted in Gmail. And I didn’t want to install any additional software such as Postfix for this.

That being said, let me continue to take you down the path that I have gone through, without the stumbling blocks.

My approach was basically: problem -> Google for solutions -> trouble shoot -> Google again. So I found a lot of useful content on the web during the process.

1. Check sendmail

Since gmail uses TLS, you will need to make sure your sendmail is compiled with TLS (for encryption) and SASL (for authentication). This is the command to use to check it:

/usr/sbin/sendmail -d0.1 -bv root

In my case, sendmail does have the necessary compilation flags so I was good. If yours doesn’t, you’ll need to re-compile sendmail and update the binary that is used to start the sendmail service, which is not covered here.

2. Upgrading Cyrus SASL

If your SASL installation doesn’t have the “plain” and “login” lib you will have authentication problem with Gmail. You can see why when you get to the sendmail configuration in the later steps. The common error in the /var/log/maillog is this:

AUTH=client, available mechanisms do not fulfill requirements

It was a vague error and I was so frustrated with it at one point I was ready to give up. However, this article about setting up Postfix with Gmail casted some light and helped me figured out the cause.

The problem is that SASL doesn’t have all the necessary plugins. The “login” and “plain” are the plugins necessary to talk to Gmail smtp. So I had to upgrade SASL to fix the problem. Here is what I did:

$ wget http://ftp.andrew.cmu.edu/pub/cyrus-mail/cyrus-sasl-2.1.21.tar.gz
$ tar -xzf cyrus-sasl-2.1.21.tar.gz
$ cd cyrus-sasl-2.1.21
$ ./configure
$ make
$ make install

$ mv /usr/lib/sasl2 /usr/lib/sasl2.orig
$ ln -s /usr/local/lib/sasl2 /usr/lib/sasl2

Since I just switched out the old sasl2 lib without recompiling sendmail, I was concerned sendmail would poop during runtime. Luckily that didn’t happen. Dynamic lib rocks!

3. Generate SSL certificate

I made a directory called certs under /etc/mail. Here are the commands that I used to generate the SSL certificates.

openssl req -new -x509 -keyout cakey.pem -out cacert.pem -days 3650
openssl req -nodes -new -x509 -keyout sendmail.pem -out sendmail.pem -days 3650

Notice I made the certificates good for almost 10 years. I didn’t needed the cacert.pem to be exact.

I also copied /usr/share/ssl/ca-bundle.crt to /ect/mail/certs and included it in the sendmail configuration file. Other wise you’ll see some error like this:

unable to get local issuer certificate

The reason is that the ca bundle file has the Gmail certificate issuer. Although I read it somewhere that email still goes out with this error. Nonetheless, we don’t need to see this if we can fix it.

4. Configure sendmail

With the preparations above we are ready to configure sendmail. I found this tutorial very useful in terms of getting the correct sendmail configurations.

In summary, I have the /etc/mail/auth/client-info looks like this:

AuthInfo:smtp.gmail.com “U:root” “I:username@gmail.com” “P:password” “M:PLAIN”
AuthInfo:smtp.gmail.com:587 “U:root” “I:username@gmail.com” “P:password” “M:PLAIN”

If you use Gmail hosted email with your own domain name, you will have username@hostname.tld in there.

Make sure run:

$ makemap -r hash client-info.db < client-info

and chmod 600 on client info files.

Essential lines in my sendmail.mc

FEATURE(`authinfo’,`hash /etc/mail/auth/client-info.db’)dnl
define(`SMART_HOST’,`smtp.gmail.com’)dnl
define(`RELAY_MAILER_ARGS’, `TCP $h 587′)
define(`ESMTP_MAILER_ARGS’, `TCP $h 587′)

define(`CERT_DIR’, `/etc/mail/certs’)
define(`confCACERT_PATH’, `CERT_DIR’)
define(`confCACERT’, `CERT_DIR/ca-bundle.crt’)
define(`confCRL’, `CERT_DIR/ca-bundle.crt’)
define(`confSERVER_CERT’, `CERT_DIR/sendmail.pem’)
define(`confSERVER_KEY’, `CERT_DIR/sendmail.pem’)
define(`confCLIENT_CERT’, `CERT_DIR/sendmail.pem’)
define(`confCLIENT_KEY’, `CERT_DIR/sendmail.pem’)

define(`confAUTH_MECHANISMS’, `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN’)
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN’)

The certificate files are generated/copied from the previous step. I’m no sendmail expert so the configuration lines may not be perfect. But it works. Let me know if you have better settings.

One tip I found very useful is to use the debugging feature. You can set a high log level in the sendmail.mc to see at which step sendmail choked and for what reason.

Also, make sure run “make” or m4 every time you touch the sendmail.mc.

m4 sendmail.mc > sendmail.cf

So that’s pretty much it. I restart the sendmail service and out goes my email.

UPDATE:

Recently I installed Sendmail on a brand new VPS and had hard time get the authentication working. It turned out saslauthd was not running. So a note for new system is to make sure saslauthd has to be up and running (better use chkconfig to make sure it starts up at run level 3) in order to get Sendmail authentication working. This may help resolve some issues in the comments.