UnixJunkie/parany

Parallelize _anything_ //

OCaml

54

156 commits

updated Oct 5, 2023

See the code

README

parany

Parallelize any kind of computation

Generalized map reduce for parallel computers (not distributed computing).

Can process a very large file in parallel on a multicore computer; provided there is a way to cut your file into independent blocks (the "demux" function).

Can process in parallel an infinite stream of elements (if you don't care about their processing order). In practice, if the stream is really infinite but you want to collect results in order, some integer counter will eventually overflow and the behavior is unspecified and unknown after this unfortunate event.

The processing function is called "work". The function gathering the results is called "mux". The number of processors running your computation in parallel is called "nprocs".

multi-processing
ocaml-library
parallel-computing
stream-processing

Contributors

UnixJunkie

147 commits

kit-ty-kate

2 commits

darrenldl

2 commits

mseri

2 commits

Languages

OCaml

95.3%

Shell

3.0%

Makefile

1.7%

UnixJunkie/parany

Parallelize _anything_ //

OCaml

54

156 commits

updated Oct 5, 2023

See the code

README

parany

Parallelize any kind of computation

Generalized map reduce for parallel computers (not distributed computing).

Can process a very large file in parallel on a multicore computer; provided there is a way to cut your file into independent blocks (the "demux" function).

Can process in parallel an infinite stream of elements (if you don't care about their processing order). In practice, if the stream is really infinite but you want to collect results in order, some integer counter will eventually overflow and the behavior is unspecified and unknown after this unfortunate event.

The processing function is called "work". The function gathering the results is called "mux". The number of processors running your computation in parallel is called "nprocs".

multi-processing
ocaml-library
parallel-computing
stream-processing

Contributors

UnixJunkie

147 commits

kit-ty-kate

2 commits

darrenldl

2 commits

mseri

2 commits

Languages

OCaml

95.3%

Shell

3.0%

Makefile

1.7%