Machine Learning

• Regression with 2-means clustering Annoy (non-scientific)

  So apparently Annoy is now splitting points by using the centroids of 2 means clustering. It is claimed that it provides better results for ANN search, however, how does this impact regression? Purely out of curiosity, I plugged a new point splitting function and generated a new set of points.

  (more…)

• Regression with Annoy (cont’d)

  After a year and half, a lot of things changed, and annoy also changed the splitting strategy too. However, I always wanted to do a proper follow up to the original post, where I compared boosting to Annoy. I still remember the reason I started that (flawed) experiment was because I found boosting easy.

  (more…)

• Quick experiment, boosting vs annoy

  While following through the Statistical Learning course, I came across this part on doing regression with boosting. Then reading through the material, and going through it makes me wonder, the same method may be adapted to Erik Bernhardsson‘s annoy algorithm.

  (more…)

• Further Hack on the Multi-Dimensional Approximate Neighbour Search

  Traversing a tree structure often involves writing a recursive function. However, Python isn’t the best language for this purpose. Therefore I started flattening the tree into a key-value dictonary structure. Logically it is still a tree, but it is physically stored as a dictionary. Therefore it is now easier to write a simple loop to traverse it.

  (more…)

• Approximate Neighbour Search in Multiple Dimensions

  In the previous post, I re-implemented Annoy in 2D with some linear algebra maths. Then I spent some time going through some tutorial on vectors, and expanded the script to handle data in 3D and more. So instead of finding gradient, the perpendicular line in the middle of two points, I construct a plane, and find the distance between it and points to construct the tree.

  (more…)

• Re-implementing Approximate Nearest Neighbour Search

  Recently I switched my search code to Annoy because the input dataset is huge (7.5mil records with 20k dictionary count). It wasn’t without issues though, however I would probably talk about it next time. In order to figure out what each parameters meant, I spent some time watching through the talk given by the author @fulhack.

  (more…)

• Random notes on Pandas and Scikit-learn

  So I first heard about Panda probably a year ago when I was in my previous job. It looked nice, but I didn’t really get the chance to use it. So practically it is a library that makes data looks like a mix of relational database table and excel sheet. It is easy to do query with it, and provides a way to process it fast if you know how to do it properly (no, I don’t, so I cheated).

  (more…)

Random Posts

• Mixing Non-array and Array in array_map()

  array_map function is a function that I use the most in my php scripts recently. However, there are times where I want to pass some non-array into it, therefore often times I have code like the snippet shown below:

  $result = array_map(
'some_callback',
array_fill(0, count($some_array), 'some_string'),
array_fill(0, count($some_array), 'some_other_string'),
$some_array
)

  It doesn’t look good IMO, as it makes the code looks complicated. Hence, after seeing how the code may vary in all different scenarios, I created some functions to clean up the array_map call as seen above. Code snippet after the jump

  (more…)

• Evaluating logical statements

  Often times one would have to write code to evaluate logical statements. For example, given statement p and q, what is p implies q? As there’s no operator for implication in PHP, one would have to rewrite the statement that consists only in NOT (!), AND (&&) and OR (||) operators. When there are a huge load of these statements, code can be difficult to read.

  (more…)

• Visualization of my profile, in (not quite) RDF way

  After a miserable trip back to academic world, I finally re-gained the courage to get back to job-market. For the time spent in university, I spent quite some time reading about Semantic Web and RDF. Then I thought, I should have published more in this format in future. However, that didn’t really happen, mostly because I am too lazy.

  (more…)

• Blending ACL and FSM-based Workflow

  After reading through the documentation, I find that the role based ACL and work flow can be more tightly integrated. Therefore I made all the transaction into many FSMs and my work flow component now consists of one work flow library and one work flow management model. As I am going a more normalized design (I use denormalized design in work as it deals with a lot of documents, however for a small project like mine, a denormalized design should do well).

  (more…)

• Adding a new settings applet for my daemon

  This is basically a small incremental update to my script published here. For some reason, the previous version of the script didn’t really work, so this release should fix the problem. Besides fixing the problem where the daemon did not actually launched at start up, I have added a settings applet for this script as well.

  (more…)