Semergence

When I was testing performance in Erlang using funs, I mentioned I wanted to see what would happen if I made the functions tail recursive. I just took a crack at it, and it looks like making this particular function tail recursive didn’t help performance. (as always, please let me know if my assumptions are incorrect.)

Tail Recursive Line Processing

process_file(Filename) ->
	{ok, File} = file  pen(Filename, read),
	process_lines(File, first, 0).

process_line(eof) -> done;
process_line(Line) ->
	L = string:tokens(string:strip(Line, both, $\n), "	"),
	{Dimensions, Measures} = lists:split(10, L),
	lists:map(fun(X) -> {I,_} = string:to_integer(X), I end, Measures).

process_lines(_, eof, _) -> done;
process_lines(File, PreviousLine, LineNum) ->
	Line = io:get_line(File, ''),
	process_line(Line),
	process_lines(File, Line, LineNum + 1).

Running this code agains a file of 10037355 lines and 1028071833 bytes big, it takes on average 401.40 seconds. This is only slightly better than the previous code attempts that are not tail recursive (and imo easier to read).