Notes on codes, projects and everything
Information Retrieval
-
Further Hack on the Multi-Dimensional Approximate Neighbour Search
Traversing a tree structure often involves writing a recursive function. However, Python isn’t the best language for this purpose. Therefore I started flattening the tree into a key-value dictonary structure. Logically it is still a tree, but it is physically stored as a dictionary. Therefore it is now easier to write a simple loop to traverse it.
-
Approximate Neighbour Search in Multiple Dimensions
In the previous post, I re-implemented Annoy in 2D with some linear algebra maths. Then I spent some time going through some tutorial on vectors, and expanded the script to handle data in 3D and more. So instead of finding gradient, the perpendicular line in the middle of two points, I construct a plane, and find the distance between it and points to construct the tree.
-
Re-implementing Approximate Nearest Neighbour Search
Recently I switched my search code to Annoy because the input dataset is huge (7.5mil records with 20k dictionary count). It wasn’t without issues though, however I would probably talk about it next time. In order to figure out what each parameters meant, I spent some time watching through the talk given by the author @fulhack.
-
Information Retrieving with ….. a lot of libraries
Implementing a Information Retrieval system is a fun thing to do. However, doing it efficiently is not (at least to me). So my first few attempts didn’t really end well (mostly uses just Go/golang with some bash tricks here and there, with or without a database). Then I jumped back to Python, which I am more familiar with and was very surprised with all the options available. So I started with Pandas and Scikit-learn combo.
Random Posts
-
Hello World from Zend MVC
This post continued from this post. Finally I have found some time to start developing my pet project using Zend Framework. After getting the controller to work the way I am more familiar (comparing to Kohana which I used at work) with, the next step is to get it to output some data.
-
More Interviewing Questions
Just survived a job interview, so I should probably celebrate this despite the outcome. Well, considering I was off the job market for a couple of years, I probably has all the reason to be nervous. Anyway, like most
geekyserious job interview, there are a test given by the company to the attendees. -
Building librdf-storage-virtuoso Package
I wanted to try using virtuoso as the storage engine for Redland but unfortunately there is no librdf-storage-virtuoso package for Ubuntu. After getting some help from @dajobe, I attempted to build the packages myself. Although it takes quite some time to build packages, but not too difficult it seems.
-
MVC: Kohana vs. Zend Framework
After comparing my own implementation of MVC with CodeIgniter’s, now I’m comparing Kohana’s and Zend’s. I have just shifted from CodeIgniter to Kohana recently in work and is currently learning on how to use Zend Framework to build my web-app. As everybody knows, Zend Framework is more like a collection of library classes than a framework a la Ruby on Rails, using MVC in Zend Framework would require one to begin from bootstrapping stage. However, in Kohana, just like other frameworks, bootstrapping is done by the framework itself so the developer will get an installation that almost just works (after a little bit of configuration).
-
Re-organizing my MacVim configuration
It is very difficult to like the way vim handle plugins by default, so I was really thrilled to find out about pathogen when a geek I followed tweeted about it. It took me some time to actually re-organize my current configuration to this new format. Then I thought why not reorganize my .vimrc as well, as my current version looks a bit cryptic after a while.
