This is the second part of the golang learning rant log. Previously on (note (code cslai)) I managed to make each line in the CSV into a hash map. So today I am going to make it into JSON Lines.
Before I continue ranting documenting my experience, the final revision of the code is here (not production ready, very n00bie, very amateur-ish).
package main
import (
"encoding/csv"
"encoding/json"
"fmt"
"io"
"log"
"os"
)
func csv_get_reader(path string) func() ([]string, error) {
file, file_error := os.Open(os.Args[1])
if file_error != nil {
log.Fatal(file_error)
}
file_reader := csv.NewReader(file)
return func() ([]string, error) {
return file_reader.Read()
}
}
func record_get_builder(field_names []string, encoder func(map[string]string) string) func(func() ([]string, error), chan string) {
return func(reader func() ([]string, error), ch chan string) {
var read_err error
var record_raw = []string{}
record := map[string]string{}
for ; read_err != io.EOF; record_raw, read_err = reader() {
if len(record_raw) > 0 {
if read_err == nil {
for index, field := range record_raw {
record[field_names[index]] = field
}
ch <- encoder(record)
} else {
log.Fatal(read_err)
}
}
}
}
}
func record_get_encoder() func(map[string]string) string {
return func(record map[string]string) string {
result, _ := json.Marshal(record)
return string(result)
}
}
func producer(ch chan string) {
var builder func(func() ([]string, error), chan string)
reader := csv_get_reader(os.Args[1])
record_header, record_error := reader()
if record_error != nil {
log.Fatal(record_error)
} else {
builder = record_get_builder(record_header, record_get_encoder())
builder(reader, ch)
close(ch)
}
}
func consumer(ch chan string) {
for halt := false; halt != true; {
result, ok := <-ch
switch {
case ok:
fmt.Println(result)
case ok == false:
halt = true
}
}
}
func main() {
ch := make(chan string)
go producer(ch)
consumer(ch)
}
So in addition to the JSON Lines part, I have also added some concurrency code inside. This is most probably the part where the word “Overdoing” in title refers to. Nothing too fancy, very much a proof-of-concept and a practical exercise.
One of the many things thing I don’t like about golang is that it is a statically typed language. Usually this isn’t a big problem, but having the support for closure, and hence having a function declaration in the following form is in fact, ridiculous.
func record_get_builder(field_names []string, encoder func(map[string]string) string) func(func() ([]string, error), chan string)
Of course I could do a
type some_type func(map[string]string) string
However, what is the point of doing that if I am writing anonymous function?
Back to another problem I mentioned yesterday, this time regarding the need to put everything in $GOPATH. So I cheated a little bit and made it work like tools like composer, and wrote the following shell script
#!/usr/bin/env bash
GOPATH=$PWD:${GOPATH}
${HOME}/.local/bin/go $@
I put this script in one of my $PATH, and hopefully this allows me to separate my golang projects from each other. Not sure what other problem this is going to cause, so again this is not recommended for production use.
Lastly for the whole golang learning diary series, Y U NO have easy way to do set operations (intersection, union etc etc.).
