Python

• Protobuf and dask

  While JSON is a fine data-interchange format, however it does have some limitations. It is well-known for its simplicity, that even a non-programmer can easily compose a JSON file (but humanity will surprise you IRL). Therefore, it is found almost everywhere, from numerous web APIs, to geospatial data (GeoJSON), and even semantic web (RDF/JSON).

  (more…)

• The Little Pythoner, Part 3

  Previously, I started practising recursions by implementing a type check on lat (list of atoms), and ismember (whether an atom is a member of a given lat). Then in the third chapter, named “Cons the Magnificent”, more list manipulation methods are being introduced.

  (more…)

• The Little Pythoner, Part 2

  In the last part, I implemented a couple of primitive functions so that they can be applied in the following chapters. The second chapter of the book, is titled “Do it again, and again, and again…”. The title already hints that readers will deal with repetitions throughout the chapter.

  (more…)

• The Little Pythoner, Part I

  I saw this article from alistapart, which is about Javascript’s prototypal object orientation. So the article mentioned Douglas Crawford, and I was immediately reminded about my struggle in understanding the language itself. Back then I used to also refer to his site for a lot of notes in Javascript. So I went back to have a quick read, and found this article that discusses the similarity between Javascript and Lisp.

  (more…)

• Oh NO! My print output is chopped!

  So my cheat with dask worked fine and dandy, until I started inspecting the output (which was to be used as an input for another script). While the script seemed to work fine, however when I started to parse each line I was hit with some funny syntax errors. After some quick inspection I found some of the lines was not printed completely.

  (more…)

• Processing JSON with dask.bag

  Often times, I am dealing with JSONL files, though panda’s DataFrame is great (and blaze to certain extend), however it is offering too much for the job. Most of the received data is in the form of structured text and I do all sorts of work with them. For example checking for consistency, doing replace based on values of other columns, stripping whitespace etc.

  (more…)

• Estimating the value of pi with a line of Python

  I came across a video on Youtube on Pi day. Coincidently it was about estimating the value of Pi produced by Matt Parker aka standupmaths. While I am not quite interested in knowing the best way to estimate Pi, I am quite interested in the algorithm he showed in the video however. Specifically, I am interested to find out how easy it is to implement in Python.

  (more…)

• Drink All You Can

  Sometimes I really doubt about the advantage of recycling old stuff to fund for new units beyond goodwill. Sure you get to convince yourself that you are saving the environment by doing so, and it also saves money in the long run. However, I didn’t realize how much it generates it may be after trying to work out an answer for a fictional IQ question.

  (more…)

• Linting a libsvm-formatted data

  While working on a text classification task, I spent quite some time preparing the training set for a given document collection. The project is supposed to be a pure golang implementation, so after some quick searching I found some libraries that are either a wrapper to libsvm, or a re-implementation. So I happily started to prepare my training set in the libsvm format.

  (more…)

Random Posts

• Further Hack on the Multi-Dimensional Approximate Neighbour Search

  Traversing a tree structure often involves writing a recursive function. However, Python isn’t the best language for this purpose. Therefore I started flattening the tree into a key-value dictonary structure. Logically it is still a tree, but it is physically stored as a dictionary. Therefore it is now easier to write a simple loop to traverse it.

  (more…)

• Self-hosted photo album

  My cloud storage is nearly exploding, and I am not in a good position to start subscribing for more storage (Yes, I am still #opentowork). Considering I just moved my domain settings to Cloudflare and started using Cloudflare tunnel, I figure I probably should just back up some of my photos, and host it on my workstation.

  (more…)

• My Collection of Common Javascript Functions – Part 1

  After coded enough Javascript few months back, I found that there are a couple of functions that I kept re-using in different projects. Therefore I took some time to refactor them and re-arrange them into a single file. The common code that I keep reusing even today consists of functions that does prototypical inheritance, scope maintenance, some jquery stuff, google maps api stuff and some general ajax application related code.

  (more…)

• MVC : Me vs. CodeIgniter

  Background

  While my static pages and site theme is still under construction, I went to CodeIgniter.com to see how to work with it and played the tutorial screencasts. The reason behind in considering CodeIgniter (CI) to be used in my next project is because I don’t feel like re-inventing the wheel. However to port my current project to use CI may cause some problems as there are differences in how we implement MVC structure.

  (more…)

• Writing jQuery Plugin 101

  I often struggle to get my Javascript code organized, and have tried numerous ways to do so. I have tried putting relevant code into classes and instantiate as needed, then abuse jQuery’s data() method to store everything (from scalar values to functions and callbacks). Recently, after knowing (briefly) how a jQuery plugin should be written, it does greatly simplify my code.

  (more…)