Notes on codes, projects and everything
2016 March
-
Oh NO! My print output is chopped!
So my cheat with dask worked fine and dandy, until I started inspecting the output (which was to be used as an input for another script). While the script seemed to work fine, however when I started to parse each line I was hit with some funny syntax errors. After some quick inspection I found some of the lines was not printed completely.
-
Processing JSON with dask.bag
Often times, I am dealing with JSONL files, though panda’s DataFrame is great (and blaze to certain extend), however it is offering too much for the job. Most of the received data is in the form of structured text and I do all sorts of work with them. For example checking for consistency, doing replace based on values of other columns, stripping whitespace etc.
-
Estimating the value of pi with a line of Python
I came across a video on Youtube on Pi day. Coincidently it was about estimating the value of Pi produced by Matt Parker aka standupmaths. While I am not quite interested in knowing the best way to estimate Pi, I am quite interested in the algorithm he showed in the video however. Specifically, I am interested to find out how easy it is to implement in Python.
Random Posts
-
MVC: Kohana vs. Zend Framework
After comparing my own implementation of MVC with CodeIgniter’s, now I’m comparing Kohana’s and Zend’s. I have just shifted from CodeIgniter to Kohana recently in work and is currently learning on how to use Zend Framework to build my web-app. As everybody knows, Zend Framework is more like a collection of library classes than a framework a la Ruby on Rails, using MVC in Zend Framework would require one to begin from bootstrapping stage. However, in Kohana, just like other frameworks, bootstrapping is done by the framework itself so the developer will get an installation that almost just works (after a little bit of configuration).
-
Revisiting semantic web (plus wheel building with Poetry)
Back then, when I was still working on my postgraduate degree research, I used RDF, which was the preferred format in the world of Semantic Web to represent data. I eventually dropped the degree, and stopped following the development of the related technology and standards. Until I volunteered to update the import script for popit when I was looking for the next job/project.
(more…) -
Spiraling Number
Back then in college, we were given a lot of programming practices. These questions usually shows a desired output format, and we were required to write a program to print out the exact thing. Usually it involves printing a matrix of numbers, or symbols etc. For these problems, usually a loop structure or two should solve the problem.
-
Random Color Cloud
Just happened to see this post a few months ago, and the author created another cloud that uses almost the same technique to ‘visualize’ a list of countries. The author uses PHP to generate the cloud originally and I thought I may be able to do in javascript. After some quick coding I managed to produce something similar to the first example, source code after the jump.
-
Blending ACL and FSM-based Workflow
After reading through the documentation, I find that the role based ACL and work flow can be more tightly integrated. Therefore I made all the transaction into many FSMs and my work flow component now consists of one work flow library and one work flow management model. As I am going a more normalized design (I use denormalized design in work as it deals with a lot of documents, however for a small project like mine, a denormalized design should do well).
