Parallel jobs
This MR implements parallel job execution for coordinator. The previous batching and queueing system has been completely replaced and simplified with a first-come-first-serve + priority system on a per-job basis.
This feature adds the MAX_JOBS
environment variable for configuration, with a default value of 5.
Resolves #6 (closed)
REMAINING TO DO:
-
update all relevant documentation -
stress testing and validation on another machine -
implement feedback -
clean up logging