Notes on codes, projects and everything
2016 January
-
Further Hack on the Multi-Dimensional Approximate Neighbour Search
Traversing a tree structure often involves writing a recursive function. However, Python isn’t the best language for this purpose. Therefore I started flattening the tree into a key-value dictonary structure. Logically it is still a tree, but it is physically stored as a dictionary. Therefore it is now easier to write a simple loop to traverse it.
-
Approximate Neighbour Search in Multiple Dimensions
In the previous post, I re-implemented Annoy in 2D with some linear algebra maths. Then I spent some time going through some tutorial on vectors, and expanded the script to handle data in 3D and more. So instead of finding gradient, the perpendicular line in the middle of two points, I construct a plane, and find the distance between it and points to construct the tree.
-
Re-implementing Approximate Nearest Neighbour Search
Recently I switched my search code to Annoy because the input dataset is huge (7.5mil records with 20k dictionary count). It wasn’t without issues though, however I would probably talk about it next time. In order to figure out what each parameters meant, I spent some time watching through the talk given by the author @fulhack.
-
Information Retrieving with ….. a lot of libraries
Implementing a Information Retrieval system is a fun thing to do. However, doing it efficiently is not (at least to me). So my first few attempts didn’t really end well (mostly uses just Go/golang with some bash tricks here and there, with or without a database). Then I jumped back to Python, which I am more familiar with and was very surprised with all the options available. So I started with Pandas and Scikit-learn combo.
Random Posts
-
Working Environment Setup, for 2020
In recent years, I start to make my development environment decouple from the tools delivered by the package manager used by the operating system. The tools (compiler, interpreters, libraries etc) are usually best left unmodified so other system packages that rely on them keeps working as intended. Also another reason for the setup is I wanted to follow the latest release as much as possible, which cannot be done unless I enroll myself to a rolling release distro.
(more…) -
Evaluating logical statements
Often times one would have to write code to evaluate logical statements. For example, given statement p and q, what is p implies q? As there’s no operator for implication in PHP, one would have to rewrite the statement that consists only in NOT (
!), AND (&&) and OR (||) operators. When there are a huge load of these statements, code can be difficult to read. -
Adding complexity to the bot: threads and processes
Being new to asyncio, after publishing the previous post on running multiple applications in one event loop, I also cross posted it to the discussion board for feedback. So apparently instead of cramming everything to the same event loop, it would be better if each application run on a separate thread. That makes sense, considering all the code that was written for that.
(more…) -
Recommender System Classification
Had a discussion with my secondary supervisor and it turned out pretty bad because I wasn’t fully prepared and he was rushing to somewhere else for a meeting. So I am jotting down a brief summary (read: highly based on personal/subjective feelings/opinions) of my readings here to help organize things before the followup meeting that is taking place next week.
-
Revisiting Fizz Buzz in PHP
Back then when I was attending a job interview, I was asked to write a Fizz Buzz program to prove that my coding ability. There was only a pen and a piece of paper, so basically means there’s no way I can refer to the documentation for the API syntax. Fortunately I somehow managed to remember and not screw up.
