I used pure relational database approach (postgresql to be exact) to store information collected via flickr initially, but it wouldn’t really scale (or I just simply suck at designing/maintaining database tables) and my queries got killed numerously by the web host. Besides that, as the table grow larger, the amount of resource consumed to serve a query also grow, hence I needed to find another way to store the data. At first I thought of using some popular noSQL solutions, but my supervisor told me to turn the data into RDF format instead. So I went on with Redland and use postgresql (again) as storage.

However, for some reason, postgresql doesn’t seem to work efficiently and a simple query can take tens of minutes to run when it holds more than 100k RDF statements. For some reason, the hash storage doesn’t work on my Ubuntu development VM, so I switched the storage engine to MySQL after @dajobe’s suggestion.

However, as PHP binding apparently not that popular, so there aren’t much information / tutorial posted. I actually wanted to write a collection of scripts to collect data from flickr for my research project, so I began with finding an existing script for that task. However, I didn’t find good ones in PHP, so I ported this from Perl to PHP and use Redland. Then after knowing how it really works, I rewrote everything again from scratch (will put them up to bitbucket when I have time to clean up). I even wrapped the library with class methods, which I hope to release later (I only knew about another OO wrapper in PHP for Redland after almost done writing my scripts).

To use Redland’s PHP functions without OO wrapper, we always start with a statement to build a world. I am not very sure what this means (yes, I didn’t really read the documentation that thoroughly), but it seems that almost all constructor functions depend on it to create new object (yes, Redland has this OO feel although all the function calls are in procedural style). So, in PHP, the statement would look like:

$world = librdf_new_world();

Then you would want to decide where to store your RDF statements. For this piece of note, I will just use non-persistent memory store. So the function call to build a storage object is

$storage = librdf_new_storage($world, 'memory', $name, $options);

where $name stores the name of the storage object, and $options often carries the DSN, but in our case it is NULL. Now that we have storage defined, then we proceed with building a model to actually store RDF statements into the storage (it would be easy to think model as a database library, and storage as an abstraction layer).

$model = librdf_new_model($world, $storage, NULL);

Statements are consists of nodes, so let’s create some. To create a URI node, it is just as easy as

$foo = librdf_new_node_from_uri_string($world, 'urn:foo');
$bar = librdf_new_node_from_uri_string($world, 'urn:bar');

Creating a literal node would be just as simple as

$baz = librdf_new_node_from_literal($world, 'baz', NULL, FALSE);

Putting them into a statement

$statement = librdf_new_statement_from_nodes($world, $foo, $bar, $baz);

which is the equivalent to this

<urn:foo> <urn:bar> 'baz' .

To run a query to the model, just simply send a SPARQL statement as follows

$query = librdf_new_query(
    $world,
    'sparql',
    NULL,
<< ?object .
        }
SPARQL
);

Running the query and get result

$result = librdf_model_query_execute($model, $query);
var_dump(librdf_query_results_to_string2($result, 'json', 'application/json', NULL, NULL));

Enjoy!

$result = array_map(
'some_callback',
array_fill(0, count($some_array), 'some_string'),
array_fill(0, count($some_array), 'some_other_string'),
$some_array
)

It doesn’t look good IMO, as it makes the code looks complicated. Hence, after seeing how the code may vary in all different scenarios, I created some functions to clean up the array_map call as seen above. Code snippet after the jump

function array_map_mix($callback, $param) {
$main = array();

return $result;
}

array_create

Before stepping into the main array_map_mix function, we first look at the array_create function. It is basically a variant of array_fill function, but without the initial index argument.

The important argument here is the $input argument, depending on what is passed into the function, the end result can be different. If anything but array is passed in, then it will behave exactly like the array_fill call as follows:

$foo = array_create(5, 'bar');
// is equivalent to
$foo = array_fill(0, 5, 'bar');

However, if an array is passed in, and the array element count is equivalent to the $count argument, then array_create will return $input without any modification.

array('bar', 'baz') == array_create(2, array('bar', 'baz'));

If the array passed in has one and only one element, and the element happens to be another array, then the child array is passed into the array_fill function, as follows

$foo = array_create(3, array(array('bar', 'baz')));
// is equivalent to
$foo = array_fill(0, 3, array('bar', 'baz'))

If the array passed in doesn’t fulfill the above two conditions, then the end result is similar like passing in a non-array argument.

$foo = array_create(3, array('bar'));
// is equivalent to
$foo = array_fill(0, 3, array('bar'));

The weird behavior caused by the array argument is actually intended, as this function is initially created for the following function, which is the array_map_mix function.

array_map_mix

The first 2 mandatory arguments for array_map_mix are the $callback as well as $param. The $callback is the callback function, and $param is the parameter. The parameter can be either a number, or an array.

When a number is passed in as the parameter, then the code is throwing all the following arguments into array_create function as described above, with the number is sent in as $count. This is an example showing the equivalent array_map function call.

$foo = array_map_mix(
'some_callback',
2,
array('foo', 'bar'),
'baz'
);
// is equivalent to
$foo = array_map(
'some_callback',
array('foo', 'bar'),
array_fill(0, 2, 'baz')
);

What if the number passed in does not equal to the count of array elements?

$foo = array_map_mix(
'some_callback',
2,
array('foo', 'bar', 'baz')
);
// is equivalent to
$foo = array_map(
'some_callback',
array_fill(0, 2, array('foo', 'bar', 'baz'))
);

What if I have 2 array, $foo and $bar, even both has the same amount of array elements, but I want the callback to receive $bar as a whole, instead of just an element?

// given
count($foo) == count($bar);
// wants this
$foo = array_map(
'some_callback',
$foo,
array_fill(0, count($foo), $bar)
);
// equivalent array_map_mix call
$foo = array_map_mix(
'some_callback',
count($foo),
$foo,
array($bar)
);

Hopefully this explains why array_create is coded like above. As mentioned above, $param can be an array as well. This is helpful when the count of array to be passed to array_map is dynamic and a fixed number is not always available.

// suppose function foo() returns an array of variable size
$foo = array_map_mix(
'some_callback',
foo(),
'baz'
);
// the equivalent code
$bar = foo();
$foo = array_map(
'some_callback',
$bar,
array_fill(0, count($bar), 'baz')
);

Nevertheless, array_map_mix can also be used as a direct array_map alternative, although this is not advisible as the function call should take much longer time to complete because of the call_user_func_array call within array_map_mix.

array_map('some_callback', array('foo', 'bar')) == array_map_mix('some_callback', array('foo', 'bar'));

The total number of arguments required is 2 when $param is an array (means using array_map_mix as array_map) and 3 when $param is a non-array. An exception will be thrown if expected argument is missing. Also, if anything but an array or a number is passed into array_map_mix as $param, then another exception is thrown.

Hopefully these two short code snippet helps for people who uses array_map as much as I do.

Base Action Controller

When I used other frameworks like Kohana or CodeIgniter, without extra modules installed, I would have to define view scripts specifically. However, in Zend_Controller, a view is likely to be defined properly and is waiting for data to be populated in. However, I still don’t get any output after a few attempts so I went to compare my dispatch method in the derived action controller with Zend_Controller_Action. The following code shows the updated dispatch method:


public function dispatch($actionName) {
$this->_helper->notifyPreDispatch();
$this->preDispatch();
$parameters = array();
foreach($this->_parametersMeta($actionName) as $paramMeta) {
$parameters = array_merge(
$parameters,
$this->_parameter($paramMeta, $this->_getAllParams())
);
}
call_user_func_array(array(&$this, $actionName), $parameters);
$this->postDispatch();
$this->_helper->notifyPostDispatch();
}


Apparently there are some ‘hooks’ that needs to be called throughout the dispatching process. Besides that, I want my view scripts to be loaded from another folder instead of the default ones, so I added a init method in the base action controller, as follows:


public function init() {
$this->view->addScriptPath(sprintf('%s/View', APP_PATH));
}


The script path should be defined during bootstrap phase, and I will post up a better solution once I find the exact place to configure it. Now that we have our base action controller fixed, let’s move to the actual action controller.

Action Controller

I am not sure whether action stack is HMVC, however, I like the way it works (although it is evil). I will probably stick to action stack for the time being while finding out how partial view can help in widgetizing my page. Besides having a view defined automatically, to put them into a layout I made a call in my bootstrap script, as follows:


Zend_Layout::startMvc(array(
'layoutPath' => APP_PATH . '/View/_layout'
));


Then I did the action stack thingy as follows:


class Controller_Index
extends Coolsilon_Controller_Base {
public function index() {
$this->_helper->actionStack('menu', 'index');
}
public function menu() {
$this->_helper->viewRenderer->setResponseSegment('menu');
}
}


Then, in my View/_layout/layout.phtml,


< ?php echo $this->layout()->menu; ?>
< ?php echo $this->layout()->content; ?>


View/index/index.phtml

This is the main content.

View/index/menu.phtml

This is the menu.

Then by visiting the project site, then I get the following output


This is the menu.
This is the main content.


Phew, I can’t believe I spent so much time on this easy stuff, XD. Now I am one step closer in writting the actual website, now what kind of website should I make?

Suppose the file structure of my experimental project as follows

• private_files
  • Model
  • View
  • Controller
    • Index.php
  • Libraries
• public_html

I am expecting if I need to load a controller named Controller_Index, then it should be able to find the controller class file in private_files/Controller/Index.php. To do so, as hinted by Tim Lytie at stackoverflow, I wrote a dispatcher class that subclassed from Zend_Dispatcher_Standard as follows


class Coolsilon_Controller_Dispatcher
extends Zend_Controller_Dispatcher_Standard {
public function __construct() {
parent::__construct();
}
 
public function formatControllerName($unformatted) {
return sprintf(
'Controller_%s', ucfirst($this->_formatName($unformatted))
);
}
 
public function formatActionName($unformatted) {
$formatted = $this->_formatName($unformatted, true);
return strtolower(substr($formatted, 0, 1)) . substr($formatted, 1);
}
}


Then, in order to make the parameters passed in as method parameters, after reading through the code as posted here, I refactored it to (I <3 short methods).


abstract class Coolsilon_Controller_Base
extends Zend_Controller_Action {
 
public function dispatch($actionName) {
$parameters = array();
 
foreach($this->_parametersMeta($actionName) as $paramMeta) {
$parameters = array_merge(
$parameters,
$this->_parameter($paramMeta, $this->_getAllParams())
);
}
 
call_user_func_array(array(&$this, $actionName), $parameters);
}
 
private function _actionReference($className, $actionName) {
return new ReflectionMethod(
$className, $actionName
);
}
 
private function _classReference() {
return new ReflectionObject($this);
}
 
private function _constructParameter($paramMeta, $parameters) {
return array_key_exists($paramMeta->getName(), $parameters) ?
array($paramMeta->getName() => $parameters[$paramMeta->getName()]) :
array($paramMeta->getName() => $paramMeta->getDefaultValue());
}
 
private function _parameter($paramMeta, $parameters) {
return $this->_parameterIsValid($paramMeta, $parameters) ?
$this->_constructParameter($paramMeta, $parameters) :
$this->_throwParameterNotFoundException($paramMeta, $parameters);
}
 
private function _parameterIsValid($paramMeta, $parameters) {
return $paramMeta->isOptional() === FALSE
&& empty($parameters[$paramMeta->getName()]) === FALSE;
}
 
private function _parametersMeta($actionName) {
return $this->_actionReference(
$this->_classReference()->getName(),
$actionName
)
->getParameters();
}
 
private function _throwParameterNotFoundException($paramMeta, $parameters) {
throw new Exception("Parameter: {$paramMeta->getName()} Cannot be empty");
}
}


It may affect the performance but the code is much easier to read as I am the only one who is wokring on the project.

So for the current stage, I have a set of relations as follows.

Resources (resource_id, name, description) a nested set table
Roles (role_id, name, description) a nested set table

Resources table stores resources available in the system (duh~), and Roles table stores roles available in the system… Both of them are nested set table as they store hierarchical data (handled by doctrine). The documentation provided by Zend states that:

• a resource is an object to which access is controlled.
• a role is an object that may request access to a Resource.

Put simply, roles request access to resources. For example, if a parking attendant requests access to a car, then the parking attendant is the requesting role, and the car is the resource, since access to the car may not be granted to everyone.

So in my project, to further simplifying the concept, a resource simply means a table which access is managed.

Then I have a states table, which is one of the key tables in work flow component.

States (state_id, name, description, is_start, is_end, is_delete)

Each record has a state, therefore each of the record references to this table to find out what the state is. There are three boolean fields added, which is `is_start`, `is_end` and `is_delete`, to further describe the status. As there is a is_delete field, the site will have the unwanted record soft deleted at first.

Then we have a composite table to link up resource and states to be used later

States_Resources (state_resource_id, state_id *, resource_id *)

Followed by the actions table, which provide a set of verbs that can be used as action name. The primary concern of having this table is to achieve consistency. Otherwise I would have URL in the form of user/profile/edit, post/update, feed/edit.

Actions (action_id, name, description)

Followed by the transition table, which stores possible transition for each of the resource at every state. Speaking in terms of Zend_ACL, each transition is like a privilege which is explained by Zend as follows:

Zend_Acl also supports privileges on resources (e.g., “create”, “read”, “update”, “delete”), so the developer can assign rules that affect all privileges or specific privileges on one or more resources.

Transitions (transition_id, action_id *, from_resource_state_id *, to_resource_state_id *)

Then I have access_rules table which stores the actual ACL

Access_Rules (access_rules_id, role_id, transition_id)

To check whether a transition is doable, frequently I would have to check the condition through some callback functions, therefore I need to store them into another table.

Transitions_Callbacks (transition_callback_id, transition_id *, callback_class_id *)

Callback_Classes (callback_class_id, name)

Lastly, the history table (there is nothing much to track at the moment)

Transition_History (transition_history_id, transition_id, user_id, ip_address, timestamp)

Controller

Basically there is no much difference in the controller, besides the naming convention that one has to follow. However, in Zend Framework, every public action method in the controller will be configured to a view template by default. For example, if you have a controller as follows:


<?php
class IndexController extends Zend_Controller {
public function __construct() {
parent::__construct();
}
 
public function indexAction() {
// some code
}
}


Even if you choose only to perform one print hello world statement, you will still get an error message indicating that the view is not found, as follows:

Uncaught exception ‘Zend_View_Exception’ with message ‘script ‘index/index.phtml’ not found

Just like what I mentioned above, in Kohana, they don’t even care whether you are using view, I can simply just output from the controller whenever I want.

View

From the first glance, Kohana and Zend Framework is doing similar things with the View. In Kohana, you can have a view defined with the following code in the controller (or any other place):


// in the controller
// instantiate the view object
$view = new View('path_to_the_filename_without_extension');
// another form
$view = new View;
$view->set_filename('path_to_the_filename_without_extension');
// yet another form
$view = View::factory('path_to_the_filename_without_extension');
 
// inject in some values to the view to be manipulated
$view->set('variable_name', 'some value');
// this is also valid
$view->variable_name = 'some value';
 
// display the view
$view->render(TRUE);


Most of the time you won’t even need to further subclass the supplied View class. To display content to the browse, you will need a view file (more like a template file). The view file will be something like follows:


<html>
<head>
</head>
<body>
<p><?php echo $variable_name; ?></p>
</body>
</html>


In Zend Framework, the code is almost similar, but I think they have more to offer if you want flexibility.

Model

In Kohana, you have 2 choices, either to use ORM as model or just a model that doesn’t really do anything much by default. I use a variant of IgnitedRecord as model in work, and I just started to question whether ActiveRecord pattern is suitable as a model in MVC few weeks ago. In Zend Framework you get no model at all as you are expected to create one for yourself.

Conclusion

Nothing much to conclude, but through the comparison between 4 different implementations to MVC, I kinda learn a lot. As I have just only started learning MVC these days, I think I will post further notes on MVC in Zend Framework soon.

Somehow the Technical Director prefers CI over Symphony (as my colleague, also a new staff member like me, was working with Symphony at the same time as I was developing my first project) which I don’t have any idea why. Then he made a decision to choose CI as the new framework to be used for future projects, well, at least for the particular project which we are still working on. The reason for the switch from internal framework to another framework like CI is that their internal framework was written for PHP4 and they want to switch to PHP5.

Their internal framework was very well done, at least from what my Technical Director showed. They have their own database abstraction layer, a CRUD application generator, Object-Relational Mapping etc. that really helps building an application fast. By comparing to their framework, I find CI although quite well done as well but it is no way near as good as the internal framework. Then I went looking around to find some plugins for the CI to implement some of the mentioned features above. Then I found Rabbit-Forms for CRUD application generator and form generation, and Ignited Record for proper ActiveRecord support.

However, I found Rabbit-Forms very restrictive and decided to write a simple form generator and CRUD application generator. Although the final product may not be good enough, but I certainly enjoyed and learned a lot through actually developing it.

These few days I am picking up Drupal, a content management system (CMS). Drupal is a very interesting system where one can almost build any website as long as it is community driven with it. However, there are a lot of things to learn to make sure you are not screwing your installation. I am extremely amazed by the way they implement the plugin system. Although it is a procedural system, but as my Technical Director says there are many object oriented concepts being implemented. Besides that, with plugins like CCK one can even do CRUD application with Drupal (although debugging will be extremely painful IMHO). There are a lot of fun stuff in Drupal for example the form generator, the hooking system, access control and of course their nearly perfect documentation (somehow i prefer it than YUI’s documentation).

Besides that, I am also doing some personal reading on Test Driven Development (TDD) and is currently focusing on unit testing PHP scripts with simpletest. Many developers who practices TDD claimed that TDD helps increasing the productivity. However after reading many documents and I am still don’t have the idea how to set up tests. Guess I would have to find out some examples elsewhere.

I only realize that my software design really sucks after getting involved in a real life development project and is currently doing a lot of readings to improve my design skill. So the main impact made to my personal project is that the project is currently on hold indefinitely until I feel that I am ready and want to practice what I have learnt these few weeks.

Developing the scheduler was not too difficult as I do not have to deal with browser experience inconsistencies very much and only concentrate on the business logic. However, when I started on my second project that extends a form to provide various widgets to guide users in filling the form correctly using YUI I nearly killed myself due to frustration. I am developing my scripts in a Ubuntu Gutsy Gibbon 7.10 desktop and use firefox 3 beta 4 to test them. I finally managed to produce a workable prototype in a week that runs almost perfectly under firefox.

Nightmare comes when the same script is deployed to a windows machine with the major 4 browsers installed (firefox, safari, opera and internet explorer). All browsers except IE7 runs my script and the error message offered by IE7 has no meaning. The first problem encountered was with the select box creation through DOM. As suggested by a tutorial in w3school, I used the following code to create the selection box.

var y=document.createElement('option');
y.text='Kiwi'
var x=document.getElementById("mySelect");
try {
x.add(y,null); // standards compliant
}
catch(ex) {
x.add(y); // IE only
}

Internet Explorer was not giving any helpful error message telling me which part is not right and I would have to use the following code to achieve the same thing.

x.options.add(new Option(...))

The reference of the constructor parameters to instantiate an Option object can be found here.

There are a couple of other problems too but fortunately I managed to fixed it in time. Internet Explorer is very strict with DOM but the problem with it is that the error message it spits is often far from being helpful. How the hell can one find out what can possibly be the cause of a “random runtime error”?

Recently I started to learn how to code using VIM through CREAM and vimtutor. I am now in the progress of remembering the key commands and probably uninstall cream by the end of the week to fully switch to VIM. I used to code in a mini-emacs editor provided by swi-prolog when I was doing my prolog subject but I didn’t quite like it because I don’t have a very flexible fingers to press the modifier keys all the time.

I did not quite like ASP.NET when I took the course in Internet Programming. Probably the unit merely touches the surface of the power of ASP.NET but I still prefer PHP because every thing is not encapsulated. However, ASP.NET is great in the sense that it helps desktop application programmers to do web-development because the programmer can still double click on a control and then add in codes to model the behavior. Anyway, since I work in Gnome Environment most of the time, unless I install mono develop, I would not have a chance to do ASP.NET unless somehow I find myself a job for that.

This is also the reason why my new personal project will be written in PHP5 instead of ASP.NET. I have recently applied a free subversion hosting at unfuddle as my current web host does not offer subversion. The reason I start learning sub-version is because in this post, the author says,

The n00b way of updating files on your web host would be to FTP in and drag and drop / delete stuff. Doing it like this will quickly get you into a mess with a large site with many files – there’s the potential to forget to upload certain files, accidentally overwriting things you shouldn’t etc. Some FTP apps have a “sync” button that tries to solve this, but I don’t trust them; you’re still overwriting files that you can never get back. That’s not a good policy to have with the valuable data on your production server.

I quite agree with that as it happened when I was deploying my final year project online last year. Uploading the files online was one of the problems, and another one was to keep mine and Regina’s work in sync. We didn’t use revision control at the time because we thought we didn’t have enough time on learning it and it seems that we wasted more time in making sure we have the identical copy all the time.

However, I am still not being able to use subversion fully because my current web-host doesn’t allow me to use SSH. I will have to find a way later to deploy my personal project online.

April FOOOOL~

Anyway, yesterday was a wonderful day as you an find jokes everywhere. Recently a lot of people seems to love Mr. Rick Astley and his “Never Gonna Give You Up” music video clip very much. I was told that youtube was linking all the featured videos to the music video clip yesterday (Rick Rolling). Digg on the other hand shows random mathematical symbols(some claimed each symbol represents a certain number of diggs) after a user diggs on the article.

My fellow bloggers were having fun as well as one of them claimed that their blog was being sold (in Chinese language) and there are also some saying that they have stopped blogging. As mentioned above, I helped mybloggercon to set up a joke (in Chinese language) claiming that we are filming a movie. Then I further rick rolled the visitors without them knowing that the song actually means. Besides that, I have recently just finished posting my short story and made an announcement that the story will be published. Some friends like Regina actually took it seriously, anyway, I will make a hardcopy for her as she wanted earlier before she flies back to Sandakan.

Hopefully I can get employed by the end of the month.

While my static pages and site theme is still under construction, I went to CodeIgniter.com to see how to work with it and played the tutorial screencasts. The reason behind in considering CodeIgniter (CI) to be used in my next project is because I don’t feel like re-inventing the wheel. However to port my current project to use CI may cause some problems as there are differences in how we implement MVC structure.

Model-View-Controller Architecture

Quoted from Java BluePrints :

Model

The model represents enterprise data and the business rules that govern access to and updates of this data. Often the model serves as a software approximation to a real-world process, so simple real-world modeling techniques apply when defining the model.

View

The view renders the contents of a model. It accesses enterprise data through the model and specifies how that data should be presented. It is the view’s responsibility to maintain consistency in its presentation when the model changes. This can be achieved by using a push model, where the view registers itself with the model for change notifications, or a pull model, where the view is responsible for calling the model when it needs to retrieve the most current data.

Controller

The controller translates interactions with the view into actions to be performed by the model. In a stand-alone GUI client, user interactions could be button clicks or menu selections, whereas in a Web application, they appear as GET and POST HTTP requests. The actions performed by the model include activating business processes or changing the state of the model. Based on the user interactions and the outcome of the model actions, the controller responds by selecting an appropriate view.

Note: You may have to view the following page with the source view as I am still looking for a good syntax highlighting plugin for my wordpress installation.