Phine Solutions web work notes

An unlikely place to look for an Amazon S3 issue

Filed under: PHP development by 1.618 — August 28, 2011 1:59 pm

This has driven me nuts for a few hours so I have to share just in case someone else has similar issue.

Basically I have a PHP script to access Amazon S3 using its SDK for PHP lib, and it runs from a server that I recently built (CentOS). The problem is, the script won’t work. Not that it threw out an error, it just wouldn’t return the result. For example if I was trying to get a list of files with same prefix, it returned an empty array. And it worked every where else and I knew for sure the stuff it’s looking for is there.

I looked everywhere and couldn’t figure out why. Until finally, I noticed that the system time is off by a few hours. It was a whole new story to fix that and this article should explain all you need to know. But in the end, after the system time is corrected, the script works again.

So if you have a script which consumes a web service, make sure the host’s system time is set correctly. It might save you a few hours of a Sunday afternoon.

A PHP class to read Microsoft Access Database

Filed under: PHP development by 1.618 — February 6, 2011 10:35 am

A friend of mine recently asked me to help out building a data warehouse based on some Access database files. Here is a PHP class that I created to read database record from Access.

class DataMdb {
	private $conn;

	function __construct($mdbFile) {
		// Set up the connection
		if (!$this->conn = new COM("ADODB.Connection")) {
			exit("Unable to create an ADODB connection");
		}
		$strConn = "DRIVER={Microsoft Access Driver (*.mdb)}; DBQ=" . realpath($mdbFile);
		try {
			$this->conn->open($strConn);
		} catch (Exception $e) {
			exit("Caught exception: " . $e->getMessage());
		}
	}

	function getAll($sql) {
		$result = array();
		try {
			$rs = $this->conn->execute($sql);

			if (!$rs->EOF) {
				$rs->MoveFirst();
				$fieldCnt = $rs->Fields->Count();

				while (!$rs->EOF) {
					$row = array();
					for ($i=0; $i<$fieldCnt; $i++) {
						$row[$rs->Fields($i)->name] = $rs->Fields($i)->value;
					}
					$result[] = $row;
					$rs->MoveNext();
				}
			}
			$rs->close();

		} catch (Exception $e) {
			exit("Caught exception: " . $e->getTraceAsString());
		}

		return $result;
	}

	function disconnect() {
		$this->conn->close();
	}
}

A PHP boolean variable note

Filed under: PHP development by 1.618 — June 22, 2008 10:40 am

I am by no means a PHP know-all so this might have been documented.

If I have a boolean variable, say

boolean x = true;

and if I use it in a if statement like:

if ( x == ‘abc’)

The result will always return “true”.

I believe the PHP version I am using is 5.2.

PHP image filter

Filed under: PHP development by 1.618 — September 23, 2007 8:13 pm

I am looking for some PHP nudity image filter library and I found this one on phpclasses.org: PHP Image Nudity Filter.

The class was written by Bakr Alsharif. According to the web site, the class “analyses the colors used in different sections of an image to determine whether those colors match the human skin color tones.”
I downloaded the code and it does use this interesting approach to analyze the image color pixel by pixel and see if they match the skin tone. Every match will add to a score and the higher the final score, the more likely the image contains nudity. For example, if an image scored above 30, it is likely a nude picture.

The class library is easy to understand and very simple to use. However I did discover one possbile bug. The upper bound for the “for” loop on the pixels may need to be changed to “<” instead of “<=”. Otherwise it will generate tons of “array out of bound” PHP warnings.

I test run the program using some pictures and it was quite interesting.

I used some baby photos, and it recognized them. I used some pictures with mountains and trees and it worked great.

But here is the issue. I used it to scan a bunch of real estate pictures and wow, one of them has a score of 100! I checked the photo and it is a kitchen photo with mostly beige cabinet and counter tops. The tone of the image is also light so that’s why the filter totally misjudged it. Beside that particular one, there are quite a few scored over 50 or 60. And they all have the similar characters.

I wonder if it would be help if we set a “ceiling score”. For example, if an image scores over 80 it is not likely a nude picture since you are not likely to have nudity filling up the whole screen. So I did another test and this time only single out the ones which scored between 30 and 80. Still, there are still too many “good” picutures being filtered out.

Although the test wasn’t quite successful I still think this PHP class is a heading to the right direction. Maybe some more sophisticated algorithm can improve the accuracy. For example, in stead of checking for the color for each pixel, we can figure out an “average tone” of an image and then analyze each pixel color to make the decision. Standard deviation, maybe?

Installing APC

Filed under: PHP development,server setup by 1.618 — September 20, 2007 9:43 pm

APC stands for “Alternative PHP Cache”. It’s one of the 3 PHP accelerators out there (the other 2 are Zend and eAccelerator).

The installation package can be found here: http://pecl.php.net/package/APC

Follow the installtion guide in the package, the apc.so is installed under:

/usr/local/lib/php/extensions/no-debug-non-zts-20060613/

This path will probably varies in different systems.

Now modify the php.ini:

1. extension_dir needs to be modified to the path above.

2. add the extension=apc.so to activate it.

3. tweak a few settings:

apc.shm_size=30 <– 30m is the default value. If up it to something like 128m, I would think the Linux system shared memory setting will also need to be increased more than that. It is 32 by default, which can be found in this file: /proc/sys/kernel/shmmax

I leave the shm_size to 30 for now.

apc.ttl=7200
apc.user_ttl=7200

4. restart the httpd service. Copy apc.php to the webserver. Check phpinfo(). Watch the Apache error_log.

To increase the APC shm_size, the kernel’s max shared memory size will also need to be increased since it is set to a very low value by default.

Add kernel.shmmax=134217728 to /etc/sysctl.conf, and run sysctl -p to make the setting take effect. This will increase the max shared memory size to 128MB.

enable curl in PHP in Windows

Filed under: PHP development by 1.618 — June 27, 2007 8:48 pm

php_curl.dll CURL, Client URL library functions Requires:Â Â libeay32.dll, ssleay32.dll

meaning they need to copied to to system32

imagecreateresampled vs. imagecreateresized

Filed under: PHP development by 1.618 — December 11, 2006 4:56 pm

If you are using image manipulation functions in PHP to resize a image, a common mistake is to use the imagecreateresized() function. Actually, most of the time, imagecreateresampled() function should be used.

According to the API document, imagecreateresampled()

copies a rectangular portion of one image to another image, smoothly interpolating pixel values so that, in particular, reducing the size of an image still retains a great deal of clarity.

This brings the question that if imagecreateresampled() produce better image, why do we still need imagecreateresized() at all? My take on this is that the latter, although not creating as pretty image as the former one, executes much faster since less smoothing (anti-aliasing) is done during the process.

building a Smarty driven PHP site – 1

Filed under: PHP development by 1.618 — December 10, 2006 10:58 am

The Model, View and Controller (MVC) design has been used widely to develop complex web sites. The key of this design is to separate the data, process and presentation so they can be developed separately and easier to maintain. 

A web server was originally just a file server over the network. It became fancier with more functionalities added and people begin to use it for wider range of things. As a result, web sites become complex applications to run on the web server to provide all kinds of information in the form of the web pages.

To create the web pages, especially the dynamic ones, the web application has to go through the following steps in most cases:

  1. Get the request from the user and decide how to process it.
  2. Once decided how, it needs to know where and then retrieve the information that the user requested, in most cases today, a database stores this information.
  3. After getting the information, the it will format it and present it in a certain way, mostly, in a HTML format.

With all these tasks, it becomes a challenge to write the application because you simply don't want to stuff all these codes into one program.

In the old CGI world, a program (in Perl or C) was written to display a web page, and it would include everything from getting the data from a DB and creating HTML page by calling the function like: print "<html>"… 

Imagine if a designer wants to make some change of a line of HTML code, he has to do the code change, probably through a developer, to make this happen. Even with the help of some tools or scripts for automatic code generation, we have to come up with some new ways so the development process can be easier and the code can be more robust and flexible.

This is the time when the MVC design comes into play. We separate the code into different groups based on what is does:

Controller handles the user's request. It decides which part of the code should be used to process the user's request. With the controller, you can clearly define the flow of the request handling.

Models defines the info retrieval and processing part. For example, if a user points to the browser to a page for sports news, the Model layer will try to search the database to get the them based on the user's request. For example, if the user wants to see NFL news, the program should use the correct query to find the news that are NFL related.

Views basically finishes the work by formatting the information and present it to the user. In a web application, the view program creates the HTML pages and they are eventually sent to the web browser.

The benefit of the MVC design is that it greatly separate the code based on what it does. With this kind of separation, it becomes easier to manage different parts of the application, maintain the small pieces without breaking the others. It also becomes possible that developers and designers can work on different parts of the application without messing up the other's work.

To implement this type of design, some type of framework needs to be in place to connect the different pieces. A developer can certainly write his own, but there have been several well-written open source project out there for us to use.

In PHP world, the Smarty template engine is a great tool to implement the MVC design.

PHP output compression

Filed under: PHP development by 1.618 — October 31, 2006 9:20 pm

In php.ini file

zlib.output_compression=On
zlib.output_compression_level=5

If you use ob_start() (output buffering)Â in php script:

use ob_start(‘ob_gzhandler‘);

Using both will not get the output compressed twice ;) , there will be an error reporting confliction instead.

PHP caching

Filed under: PHP development by 1.618 — October 18, 2006 8:33 pm

I started to implement the PHP caching for one of my site. Since I use PHP/Smarty, there are a few options.

Smarty offers a page level caching.

You can cache the entire page with Smarty’s caching capability. It is very easy to use. Just specify the $smarty->caching attribute and the whole page will be cached. The only downside I see is its inflexibility to cache the data in a more granular level. With a complicated database driven backend, we need to deal with different type of information we pulled from the database and decide whether to cache or how long to cache.

The Cache_Lite from PEAR is a great PHP lib to do this. It can take a String and cache it to a file, or talk a more complicated object and serialize it then cache it. Or you can use the serialize() /unserialize() method directly, like the way its done in the following code.

A “wrapper” class can be created to suite the need for caching different type of data:

require_once PEAR_DIR . “Lite/Cache/Lite.php”;
class CacheUtil {
private $cLite;
private $group;
function __construct($type = 0) {
// Set a few options
$options = array(
‘cacheDir’ => CACHE_DIR,
‘lifeTime’ => 3600
);
switch ($type) {
case TYPE_0:
$options['cacheDir'] = CACHE_DIR . “type0/”;
$options['lifeTime'] = 3600*24; // cache 1 day
$this->group = “type0″;
break;
case TYPE_1:
$options['cacheDir'] = CACHE_DIR . “type1/”;
$options['lifeTime'] = 60;
$this->group = “type1″;
break;
}
// Create a Cache_Lite object
$this->cLite = new Cache_Lite($options);
}
/*
* Takes an object needs to be cached, serialize it and store in the CacheLite
*/
public function set($cacheContent, $id) {
if (USE_CACHE == 0) {
return false;
}
$this->cLite->save(serialize($cacheContent), $id, $this->group);
}
public function get(&$cacheContent, $id) {
if (USE_CACHE == 0) {
return false;
}
if ($data = $this->cLite->get($id, $this->group)) {
$cacheContent = unserialize($data);
return true;
} else {
return false;
}
}
public function remove($id) {
if ($this->cLite->get($id, $this->group) !== false) {
$this->cLite->remove($id, $this->group);
}
}
public function clean() {
$this->cLite->clean($this->group);
}
public function nuke() {
$this->cLite->clean();
}
}
 Â

While using caching can definitely help speeding up the data retrieval if there is database involved, it also adds the complication to the code, for example: whenever there is any type of insert/delete/update, the cache will also need to be flushed if affected; caching is not free, it takes up the file system space too, therefore, a backend console for monitoring the caching usage is also needed to be considered.

Next Page »

©phinesolutions.com