16 Feb 2011

disk_log vs file

168 words

I’ve been wondering about the overhead incurred by using Erlang’s disk_log (instead of plain old file) for simple, append-only term storage. It turns out there is essentially none.

Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [kernel-poll:false]

Eshell V5.7.5  (abort with ^G)
1> c(bench).
{ok,bench}
2> bench:bench().
time(bench):           7438.615ms disk_log
time(bench):           7510.093ms file
time(bench):          41961.662ms disk_log_sync
time(bench):          40884.254ms file_sync
ok
3> bench:bench().
time(bench):           7399.243ms disk_log
time(bench):           7485.069ms file
time(bench):          41741.028ms disk_log_sync
time(bench):          42417.001ms file_sync
ok
4> bench:bench().
time(bench):           7404.627ms disk_log
time(bench):           7530.133ms file
time(bench):          41294.755ms disk_log_sync
time(bench):          41174.412ms file_sync
ok
5>

Code available: https://gist.github.com/829306

UPDATE:

Same comparison on a modern commodity SSD:

Erlang R14B03 (erts-5.8.4) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.8.4 (abort with ^G)
1> c(bench).
{ok,bench}
2> bench:bench().
time(bench): 6193.493ms disk_log
time(bench): 6295.853ms file
time(bench): 22405.512ms disk_log_sync
time(bench): 22422.493ms file_sync
ok
3> bench:bench().
time(bench): 6316.854ms disk_log
time(bench): 6337.419ms file
time(bench): 22216.919ms disk_log_sync
time(bench): 22234.268ms file_sync
ok
4> bench:bench().
time(bench): 6270.888ms disk_log
time(bench): 6337.806ms file
time(bench): 22261.496ms disk_log_sync
time(bench): 22290.733ms file_sync
ok
5>

Thanks, Uwe!