Notes on codes, projects and everything
Python
-
Protobuf and dask
While JSON is a fine data-interchange format, however it does have some limitations. It is well-known for its simplicity, that even a non-programmer can easily compose a JSON file
(but humanity will surprise you IRL). Therefore, it is found almost everywhere, from numerous web APIs, to geospatial data (GeoJSON), and even semantic web (RDF/JSON). -
The Little Pythoner, Part 3
Previously, I started practising recursions by implementing a type check on lat (list of atoms), and
ismember(whether an atom is a member of a given lat). Then in the third chapter, named “Cons the Magnificent”, more list manipulation methods are being introduced. -
The Little Pythoner, Part 2
In the last part, I implemented a couple of primitive functions so that they can be applied in the following chapters. The second chapter of the book, is titled “Do it again, and again, and again…”. The title already hints that readers will deal with repetitions throughout the chapter.
-
The Little Pythoner, Part I
I saw this article from alistapart, which is about Javascript’s prototypal object orientation. So the article mentioned Douglas Crawford, and I was immediately reminded about my struggle in understanding the language itself. Back then I used to also refer to his site for a lot of notes in Javascript. So I went back to have a quick read, and found this article that discusses the similarity between Javascript and Lisp.
-
Oh NO! My print output is chopped!
So my cheat with dask worked fine and dandy, until I started inspecting the output (which was to be used as an input for another script). While the script seemed to work fine, however when I started to parse each line I was hit with some funny syntax errors. After some quick inspection I found some of the lines was not printed completely.
-
Processing JSON with dask.bag
Often times, I am dealing with JSONL files, though panda’s DataFrame is great (and blaze to certain extend), however it is offering too much for the job. Most of the received data is in the form of structured text and I do all sorts of work with them. For example checking for consistency, doing replace based on values of other columns, stripping whitespace etc.
-
Estimating the value of pi with a line of Python
I came across a video on Youtube on Pi day. Coincidently it was about estimating the value of Pi produced by Matt Parker aka standupmaths. While I am not quite interested in knowing the best way to estimate Pi, I am quite interested in the algorithm he showed in the video however. Specifically, I am interested to find out how easy it is to implement in Python.
-
Drink All You Can
Sometimes I really doubt about the advantage of recycling old stuff to fund for new units beyond goodwill. Sure you get to convince yourself that you are saving the environment by doing so, and it also saves money in the long run. However, I didn’t realize how much it generates it may be after trying to work out an answer for a fictional IQ question.
-
Linting a libsvm-formatted data
While working on a text classification task, I spent quite some time preparing the training set for a given document collection. The project is supposed to be a pure golang implementation, so after some quick searching I found some libraries that are either a wrapper to libsvm, or a re-implementation. So I happily started to prepare my training set in the libsvm format.
Random Posts
-
Linked Data Principle
Semantic Web is not just about putting data on the web, but also making links to allow a person as well as a machine to explore the web of data. Links are made in the web of data connects arbitrary things together as described by RDF as opposed to links in the web of hypertext, where links connects to only web-resources. Linkage of arbitrary things then allow related things to be found while performing search.
-
Sorting Multilingual Data
Been trying my best to stick to the well-known UNIX Philosophy – “Do one thing and do it well”, so I have been breaking down my projects into numerous pieces of small tasks and rely on existing tools whenever possible. One of the existing tool that I use a lot is the GNU sort tool. Generally sort utility is really doing fine and dandy without having to configure anything, at least not until I realize the problem that leads to this post.
-
Overdoing a task part 2: csv-to-json
This is the second part of the golang learning
rantlog. Previouslyon (note (code cslai))I managed to make each line in the CSV into a hash map. So today I am going to make it into JSON Lines. -
Evaluating logical statements
Often times one would have to write code to evaluate logical statements. For example, given statement p and q, what is p implies q? As there’s no operator for implication in PHP, one would have to rewrite the statement that consists only in NOT (
!), AND (&&) and OR (||) operators. When there are a huge load of these statements, code can be difficult to read. -
Webmail Checker Alpha for MeeGo Harmattan
After shifting all my instant messaging accounts to my Nokia N9, I stopped getting email alerts via Adium. Therefore, when I finally remember to check my mailboxes, they are already loaded with exploding amount of mails (mostly junk and newsletter though). I don’t fancy doing my email stuff with my device, and don’t feel like installing a webmail checker to my browser, hence this simple little script is written for my phone.
