Phine Solutions web work notes

This past weekend my PC was infected by malware and basically it keeps opening the Firefox windows with different websites. After spending hours on how to remove it I gave up. Frankly I think a lot of those sypwares or malwares are created by people who sells the spyware removal software but it is another subject.

So I  upgraded to Vista hoping for a more secured OS.

I have read a lot of negative reviews about Vista and I was always hesitant to upgrade but so far the experience has been quite positive.

The PC I am running Vista on has Intel Dual core processor and 3G ram on it so I didn’t feel a slight slowdown while running applications on Vista. As a matter of fact, I feel everything is a bit faster than XP.

The graphic interface is quite nice and there are certainly a lot of improvements. It seems to me Microsoft also borrowed some ideas from Max OS X, which is not necessarily a bad thing.

A lot of people are complaining about the Vista’s User Account Control (UAC) security prompt. Basically you get a lot of popups when you try to install something or run something that needs to open a port or changes system files. Yes it is annoying but I am not too much bothered by it, as long as it really helps to protect my system (fingers crossed ).

The areas that I think can be improved:

Why so many versions? “Home”, “Business”, “Premium”, “Ultimate”…they are just confusing. The pricing ladder can probably help Microsoft targeting different users and make more money but too many version also create more work on marketing and support.

The UAC is there for a reason but it can be designed better. I know it is not Unix so “sudo” command is not an option but maybe something cleverer so less users will choose to disable it after installation?

Sometimes Vista feels clunky to use. Possiblly because it is designed to incorporate the new features and the old ones so there are many ways to get to the same destination. Simplicity is the ultimate sophistication and Mac OS shows. Maybe WIndows 7 addresses this issue better?

I have only used it for a week so I can’t say for sure Vista’s improved security really delivers. Hopefully I didn’t waste my time and money to upgrade.

Basically this is what I am trying to do here:

SELECT * FROM Table1 WHERE Table1.col1 = ‘bob’ AND Table1.col3 IN (SELECT Table2.col2 FROM Table2 WHERE Table2.col1 = ‘alice’)

Both Table1 and Table2 have the same column with the name “col1″. And the value of Table1.col3 should match values coming from Table2.col2.

However the query always returns empty set, even there are definitely data matches via the IN where clause. As a matter of fact, if I use the values directly from the subquery the whole query would work.

At a first glance, this looks like a correlated subquery issue but it’s not. Even Table1 and Table2 share the same column name, the subquery is not referring to any Table1 column in the where clause, so it shouldn’t be an issue explained here.

Anyone has some insight on this?

Ok this one was really a thinker. I spent at least 5 hours to get this to work and finally I was able to use Sendmail to relay through my Gmail account.

A little background:

I have a Linux VPS with CentOS installed. The only email MTA is the default Sendmail. Everything else is pretty much the standard CentOS 4 installation. I don’t intend to use this box as a mail server or any other type of email processor. What I was trying to do is to add some basic capability to send out email from the box using my existing email accounts hosted in Gmail. And I didn’t want to install any additional software such as Postfix for this.

That being said, let me continue to take you down the path that I have gone through, without the stumbling blocks.

My approach was basically: problem -> Google for solutions -> trouble shoot -> Google again. So I found a lot of useful content on the web during the process.

1. Check sendmail

Since gmail uses TLS, you will need to make sure your sendmail is compiled with TLS (for encryption) and SASL (for authentication). This is the command to use to check it:

/usr/sbin/sendmail -d0.1 -bv root

In my case, sendmail does have the necessary compilation flags so I was good. If yours doesn’t, you’ll need to re-compile sendmail and update the binary that is used to start the sendmail service, which is not covered here.

2. Upgrading Cyrus SASL

If your SASL installation doesn’t have the “plain” and “login” lib you will have authentication problem with Gmail. You can see why when you get to the sendmail configuration in the later steps. The common error in the /var/log/maillog is this:

AUTH=client, available mechanisms do not fulfill requirements

It was a vague error and I was so frustrated with it at one point I was ready to give up. However, this article about setting up Postfix with Gmail casted some light and helped me figured out the cause.

The problem is that SASL doesn’t have all the necessary plugins. The “login” and “plain” are the plugins necessary to talk to Gmail smtp. So I had to upgrade SASL to fix the problem. Here is what I did:

$ wget http://ftp.andrew.cmu.edu/pub/cyrus-mail/cyrus-sasl-2.1.21.tar.gz
$ tar -xzf cyrus-sasl-2.1.21.tar.gz
$ cd cyrus-sasl-2.1.21
$ ./configure
$ make
$ make install

$ mv /usr/lib/sasl2 /usr/lib/sasl2.orig
$ ln -s /usr/local/lib/sasl2 /usr/lib/sasl2

Since I just switched out the old sasl2 lib without recompiling sendmail, I was concerned sendmail would poop during runtime. Luckily that didn’t happen. Dynamic lib rocks!

3. Generate SSL certificate

I made a directory called certs under /etc/mail. Here are the commands that I used to generate the SSL certificates.

openssl req -new -x509 -keyout cakey.pem -out cacert.pem -days 3650
openssl req -nodes -new -x509 -keyout sendmail.pem -out sendmail.pem -days 3650

Notice I made the certificates good for almost 10 years. I didn’t needed the cacert.pem to be exact.

I also copied /usr/share/ssl/ca-bundle.crt to /ect/mail/certs and included it in the sendmail configuration file. Other wise you’ll see some error like this:

unable to get local issuer certificate

The reason is that the ca bundle file has the Gmail certificate issuer. Although I read it somewhere that email still goes out with this error. Nonetheless, we don’t need to see this if we can fix it.

4. Configure sendmail

With the preparations above we are ready to configure sendmail. I found this tutorial very useful in terms of getting the correct sendmail configurations.

In summary, I have the /etc/mail/auth/client-info looks like this:

AuthInfo:smtp.gmail.com “U:root” “I:username@gmail.com” “P:password” “M:PLAIN”
AuthInfo:smtp.gmail.com:587 “U:root” “I:username@gmail.com” “P:password” “M:PLAIN”

If you use Gmail hosted email with your own domain name, you will have username@hostname.tld in there.

Make sure run:

$ makemap -r hash client-info.db < client-info

and chmod 600 on client info files.

Essential lines in my sendmail.mc

FEATURE(`authinfo’,`hash /etc/mail/auth/client-info.db’)dnl
define(`SMART_HOST’,`smtp.gmail.com’)dnl
define(`RELAY_MAILER_ARGS’, `TCP $h 587′)
define(`ESMTP_MAILER_ARGS’, `TCP $h 587′)

define(`CERT_DIR’, `/etc/mail/certs’)
define(`confCACERT_PATH’, `CERT_DIR’)
define(`confCACERT’, `CERT_DIR/ca-bundle.crt’)
define(`confCRL’, `CERT_DIR/ca-bundle.crt’)
define(`confSERVER_CERT’, `CERT_DIR/sendmail.pem’)
define(`confSERVER_KEY’, `CERT_DIR/sendmail.pem’)
define(`confCLIENT_CERT’, `CERT_DIR/sendmail.pem’)
define(`confCLIENT_KEY’, `CERT_DIR/sendmail.pem’)

define(`confAUTH_MECHANISMS’, `EXTERNAL GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN’)
TRUST_AUTH_MECH(`EXTERNAL DIGEST-MD5 CRAM-MD5 LOGIN PLAIN’)

The certificate files are generated/copied from the previous step. I’m no sendmail expert so the configuration lines may not be perfect. But it works. Let me know if you have better settings.

One tip I found very useful is to use the debugging feature. You can set a high log level in the sendmail.mc to see at which step sendmail choked and for what reason.

Also, make sure run “make” or m4 every time you touch the sendmail.mc.

m4 sendmail.mc > sendmail.cf

So that’s pretty much it. I restart the sendmail service and out goes my email.

Start from yesterday I noticed some changes with my iGoogle page.

iGoogle is my personalized home page that has all my gadgets on one page for quick access, for example, RSS reader, Gmail, Calendar, etc. One noticeable changes is that the “Home” tab is moved to the left of the screen, with a gadget menu drop down. From the new “Home menu”, I’ll be able to open the reader or Gmail on the same iGoogle page without having to deal with a new browser window (or tab).

It takes me a little getting used to at the beginning. The new menu on the left takes up some space and the home view is definitely squeezed in this design. Plus, some of the functionality changes the behavior so I had to adjust. After a little while I started to feel the convenience.

I guess this is another step toward a browser desktop from Google. The Home menu definitely reminds me of the Windows start menu or the dashboard on Mac OS. With more and more gadgets (application) released to the wild, iGoogle can become very useful if you like Google’s product.

As a long time PC/Windows user I finally got my first Apple computer: a shiny white Macbook.

Macbook is not a complicated machine. It doesn’t overwhelm you with bells and whistles. But everything is there to work. The hardware design is truly the best.

The Mac OS X is a great OS. I didn’t have much learning curve to get familiar with it except to learn all the keyboard shortcuts since I want to stick with the touchpad without a mouse. I found out that you can put two fingers on the touchpad and scroll down a page, which is an awesome feature. One great thing about Mac OS X is that it is based on the Free BSD so I can always use a terminal to do the work. It is essentially a Unix machine with a nice GUI.

Mac app did crash once. My iTunes crashed and a message box popped up so I can end the program. However it didn’t bring down the whole machine and everything else was business as usual. There is another time that iTunes appeared to be in a runaway mode and I had to kill it manually. Seems that Apple has some work to do on the iTunes. But overall it synced with my iPod without any problems. The mac sleep mode also works flawlessly, which I always have problems with my Windows machine.

So here is my short review. I remember some famous people once said “simplicity is the ultimate sophistication”, and a Mac reminds me a lot of that.

Google released Chrome yesterday, a new web browser, currently available for Windows XP only.

It is very simple to install and It is very fast based on my experience. I haven’t tried many sites though. So there are probably sites that don’t play well with Chrome. Chrome will not have a bookmark if you don’t import one from Firefox or IE. And also, I can’t use a drop down to see the sites I used to visit from the address bar (or so called omnibox in Chrome), which I think is a big inconvenience. However, upon opening a new tab, a bunch of sites that I have visited will be there to provide some kind shortcuts to the sites that I used to visit.

It is probably still early to say how dominant (or how irrelevant) Chrome will be in the future but it will certainly change the way people access the Internet. I am already using Chrome for my GMail, Google Reader and some other Google apps with Firefox opened for other web browsing and development work. When there are more and more browser based apps available, Chrome may be the perfect choice to run those heavy JavaScript sites.

I think besides the fresh look and some usability innovations of this new browser, the key is its open source root. Looks like the desktop app itself is on a good start, now the next thing is to extend it into other territories like mobile and smart devices, which is quite an open field to get into.

In the Desktop environment I think it is still too early to tell how long it will take to be adopted by the mainstream, considering 20-25% of the web users are still using IE6 today. Google might have to bundle Chrome with some really great browser apps to get it replace IE on people’s Windows desktop. In the meantime Microsoft will keep improving IE so it is still a long way to go for Chrome. But I think Chrome is a real threat to the browsers like Opera today and if when the development of 3rd party add-ons starts to catch on for Chrome, it will most definitely cut into Firefox’s market share.

I have a Dell Inspiron 1000 laptop which was mainly used by my wife for checking emails and some casual web surfing. I installed Xubuntu on it hoping to get the best performance out of the limited horsepower that this old laptop has. It worked OK, until I changed my wireless network security from WEP to WPA.

I found out Xubuntu doesn’t have the support for it. To better phrase it, to get WPA working in Xubuntu, I have to install some 3rd party software like wpasupplicant and make some configuration changes to get it working. Later versions of ubuntu might “just works” as described in this doc, but there seems to be issues around it.

After following the steps in one of the forum thread and failed to make it work I quit. I have to say I didn’t try very hard at it but I didn’t think spending hours on this laptop just to make wireless working made much sense, with work, business and my 2 year old I am not that kind of geek who spend days to get dialup modem working on my Linux machine any more. So I ended up putting XP (Gasp back on this machine. As a matter of fact, when XP runs with least graphic candy the performance is on par with Xubuntu running in graphic mode.

I am a person who is pretty familiar with Linux, as a matter of fact I use it every day. I like it. I would recommend it. But do I think it is ready for a average home PC user? No. It is just not simple enough to get some basic things to work. It could be a great OS to be installed in a more uniformed environment, like a lab in a university or an office environment.

Last week Amazon A3 was down for 4 hours and made a lot of webmasters unhappy. It further proves that it is quite risky to design your site solely relying on A3 to provide the essential functionalities, at least for now.

One way to reduce the risk is to have a copy of the files, for example images, saved in your server and design a flag in your code to pull the file from your own server if A3 downtime was detected. The flag can be controlled by some parameter in a configuration file so it can be easily switched.

One might argu this defeats the purpose of using AWS storage since the load copies take up the space. But I believe the storage cost will worth it in the event of the A3 hiccups. By using A3 service when it’s up and running, you will still save the bandwidth when serving those files, which is a lot more expensive than the storage cost.

Good documentation can save a lot of time to look for the information that you have once obtained. I had been looking for some documentation system that can help me to write down notes and finally decided to use a wiki.

Using a wiki can provide some great benefits:

• It is web based can I can access anywhere.
• My documents can be organized by sites and category so I have a central place to look for things that I have done but not necessarily memorized.
• Easy collaboration with others.

There are several free wiki softwares and I chose DocuWiki after some research and test run. Since I only need a simple interface to create structured content DocuWiki was a good choice for me. This is a nice comparison on some popular wiki software that I also used to help me make my decision.

There are a lot of talks around Yahoo’s move recently. Y’s decisions to reject the Microsoft deal and partner with Google on search advertising market don’t sit well with a lot of people. Some think Yahoo is making stupid decisions.

I beg to differ.

Being bought by Microsoft will make a lot of shareholders happy today. But if bought by Microsoft, 3 years from now, will Yahoo still be relevant at all? Choosing to remain independent will give Yahoo a chance. But what about the shareholders? Well shareholders don’t RUN the company. They can make their influence by choosing the board, or cast their vote by selling the stock. I don’t own YHOO so this might sound insensitive. But company shareholders having disagreement with management team is not new and opinions from shareholders who want to sell the stock and cash in should not be used to judge the company’s direction.

Will Yahoo+Google deal hurt Yahoo in the long run? It might hurt Yahoo’s advertising network. But Google is already so dominant in search market and Yahoo may never be able to catch on with its technology today. The key is for Yahoo to innovate and find new ways to establish a stronger brand, not necessary all in the search market though.

Some also have concerns that Yahoo is helping to create a monopolistic Google which will turn evil in no time. Google has 60% search market share because so far its search technology is the best. And it is not so dominant in other areas and Yahoo’s deal will not make it be. I think the key is to believe that technology is alway moving forward, today’s big players will be replaced if they stay still, and there will always be newer and better things coming out. Technology is simply not a business that can be established upon monopoly.

Having said that, the challenges ahead of Yahoo are obvious and I wish them the best of the luck.