2016 March

• Oh NO! My print output is chopped!

  So my cheat with dask worked fine and dandy, until I started inspecting the output (which was to be used as an input for another script). While the script seemed to work fine, however when I started to parse each line I was hit with some funny syntax errors. After some quick inspection I found some of the lines was not printed completely.

  (more…)

• Processing JSON with dask.bag

  Often times, I am dealing with JSONL files, though panda’s DataFrame is great (and blaze to certain extend), however it is offering too much for the job. Most of the received data is in the form of structured text and I do all sorts of work with them. For example checking for consistency, doing replace based on values of other columns, stripping whitespace etc.

  (more…)

• Estimating the value of pi with a line of Python

  I came across a video on Youtube on Pi day. Coincidently it was about estimating the value of Pi produced by Matt Parker aka standupmaths. While I am not quite interested in knowing the best way to estimate Pi, I am quite interested in the algorithm he showed in the video however. Specifically, I am interested to find out how easy it is to implement in Python.

  (more…)

Random Posts

• Blending ACL and FSM-based Workflow

  After reading through the documentation, I find that the role based ACL and work flow can be more tightly integrated. Therefore I made all the transaction into many FSMs and my work flow component now consists of one work flow library and one work flow management model. As I am going a more normalized design (I use denormalized design in work as it deals with a lot of documents, however for a small project like mine, a denormalized design should do well).

  (more…)

• Building a simple ASGI Websocket Application

  Getting comfortable to asyncio takes a bit of practice, so I revisited a practice project I did when I was working for my previous company. Suppose I want to build a very simple websocket application, without use of any web application library/framework. In order to keep it simple, I also opt to just build the frontend with minimal setup (just plain ES6 without webpack/vite).

  (more…)

• Proof of Concept: YQuery

  After the last post, I found that it may be fun to write a wrapper for YUI in order to make it behave like jQuery. Therefore, the code below is clearly mainly for self-amusement and is not intended to be used in production projects. However, through coding this, I found that although the difference in design, but YUI is obviously capable to do what jQuery offers (if not more). I will not continue working on this so whoever interested may just copy and paste the code to further developing it.

  (more…)

• New directory structure

  While the previous file structure works well, I decided to tune some details before deploying the latest WordPress release. Besides that, I also started a new theme development project after my last theme which was developed more than 2 years ago. Thankfully, everything seems to work so far.

  (more…)

• How to solve all puzzles at 4clojure in a week

  I was trying to learn scala and clojure to find one that I may want to use in my postgraduate project. After trying to learn scala for a couple of days, I gave up because I really don’t like the syntax (too OO for my liking). Then I continued with clojure and found myself liking the syntax better.

  (more…)