Machine Learning

• Regression with 2-means clustering Annoy (non-scientific)

  So apparently Annoy is now splitting points by using the centroids of 2 means clustering. It is claimed that it provides better results for ANN search, however, how does this impact regression? Purely out of curiosity, I plugged a new point splitting function and generated a new set of points.

  (more…)

• Regression with Annoy (cont’d)

  After a year and half, a lot of things changed, and annoy also changed the splitting strategy too. However, I always wanted to do a proper follow up to the original post, where I compared boosting to Annoy. I still remember the reason I started that (flawed) experiment was because I found boosting easy.

  (more…)

• Quick experiment, boosting vs annoy

  While following through the Statistical Learning course, I came across this part on doing regression with boosting. Then reading through the material, and going through it makes me wonder, the same method may be adapted to Erik Bernhardsson‘s annoy algorithm.

  (more…)

• Further Hack on the Multi-Dimensional Approximate Neighbour Search

  Traversing a tree structure often involves writing a recursive function. However, Python isn’t the best language for this purpose. Therefore I started flattening the tree into a key-value dictonary structure. Logically it is still a tree, but it is physically stored as a dictionary. Therefore it is now easier to write a simple loop to traverse it.

  (more…)

• Approximate Neighbour Search in Multiple Dimensions

  In the previous post, I re-implemented Annoy in 2D with some linear algebra maths. Then I spent some time going through some tutorial on vectors, and expanded the script to handle data in 3D and more. So instead of finding gradient, the perpendicular line in the middle of two points, I construct a plane, and find the distance between it and points to construct the tree.

  (more…)

• Re-implementing Approximate Nearest Neighbour Search

  Recently I switched my search code to Annoy because the input dataset is huge (7.5mil records with 20k dictionary count). It wasn’t without issues though, however I would probably talk about it next time. In order to figure out what each parameters meant, I spent some time watching through the talk given by the author @fulhack.

  (more…)

• Random notes on Pandas and Scikit-learn

  So I first heard about Panda probably a year ago when I was in my previous job. It looked nice, but I didn’t really get the chance to use it. So practically it is a library that makes data looks like a mix of relational database table and excel sheet. It is easy to do query with it, and provides a way to process it fast if you know how to do it properly (no, I don’t, so I cheated).

  (more…)

Random Posts

• MVC : Me vs. CodeIgniter

  Background

  While my static pages and site theme is still under construction, I went to CodeIgniter.com to see how to work with it and played the tutorial screencasts. The reason behind in considering CodeIgniter (CI) to be used in my next project is because I don’t feel like re-inventing the wheel. However to port my current project to use CI may cause some problems as there are differences in how we implement MVC structure.

  (more…)

• Maintaining State with YUI Event

  When one start writting Javascript in patterns like the module pattern, then sooner or later he would want to maintain the state when an event handler is called. The reason I am still using YUI to handle my event handling code is because I like how state can be maintained.

  (more…)

• Approximate Neighbour Search in Multiple Dimensions

  In the previous post, I re-implemented Annoy in 2D with some linear algebra maths. Then I spent some time going through some tutorial on vectors, and expanded the script to handle data in 3D and more. So instead of finding gradient, the perpendicular line in the middle of two points, I construct a plane, and find the distance between it and points to construct the tree.

  (more…)

• Packaging a Daemon Script for Meego 1.2 Harmattan

  Another half a day spent on figuring out how to package my daemon properly, fortunately with help from friends over at #harmattan IRC channel as well as cckwes, I finally get the deb package generated properly. So just a quick reminder on what my daemon does, it is just a quick hack that toggles the ‘allow background connections’ on and off depending which kind of data network a user is connected to. Apparently I am not the only one who are looking for this, as a feature request was filed long long time ago.

  (more…)

• Reinventing wheels with cheats using Rustlang

  I finally put in some time and effort learning myself a bit of Rust. Though I am still struggling with ownership and lifetimes (which is essentially everything about the language, to be honest), I find it more interesting compared to Golang, which is relatively boring, though being functional (no pun intended). While learning the language, the one thing I came across often is the Option enum, then I remembered that I read something about Monad.

  (more…)