Job scheduling. Using Rust.

Benchmark of several job scheduling algorithms implemented in Rust.

The problem

We have a list of independent tasks that we want to execute as fast as possible. In order to achieve this, we want to run then in parallel. However, those tasks have different execution times that can vary from several orders of magnitude. Fortunately, it is possible to estimate the execution time of each task before running it.

The goal is therefore to schedule the execution of the tasks on all available threads in such a way that minimizes the total running time. This is known as the optimal job scheduling problem and can often be seen as a partition problem, which is known to be NP-complete.

Example

We have 4 tasks tasks to execute. Three of them takes 1 second each to execute and the last one takes 3 seconds. We have 2 processing units of equal power.

If each processing unit starts by executing one of the 1s tasks, they will both finis this job after one second. Then, one of those processing unit will execute the remaining 1s task while the other will execute the 3s task. At the end, one processing unit would have ran 2 seconds and the other one 4 seconds. The total time is therefore 4 seconds.

Instead, we could start having one processing unit to execute the 3s task and the other one execute the three 1s task. At the end, both processing units would have ran 3 seconds. The total time is therefore 3 seconds.

The second job scheduling was clearly better than the first one, resulting in zero idling processing unit and therefore the shortest total execution time.

Goal

Talking about job scheduling algorithms and all the cool stuff you can use to implement them in an efficient way is one thing. Actually implementing them and gathering data on how it actually went in a real situation is another.

This project choses the later and therefore aims to implement several of those algorithms in Rust and benchmark them. Let the battle begin.

How to use

Install Rust and gnuplot.
Run cargo bench.
Use a web browser to look at the results located in ./target/criterion/report/.

Algorithms

single thread: No multi-threading or any optimization of any kind. Used as a reference.
lpt: Manual multi-threading. Tasks are sorted so the shortest ones are executed first. This is the longest-processing-time-first algorithm.
rayon: Multi-threading provided by the rayon crate, but no optimization of any kind. Used as a reference.
rayon sorted: Multi-threading provided by the rayon crate. Tasks are sorted so the shortest ones are executed first.
rayon sorted reverse: Multi-threading provided by the rayon crate. Tasks are sorted so the longest ones are executed first. This should behave as an LPT.
rayon sorted reverse: Similar to rayon sorted reverse, but tasks are sorted from shortest to longest and are then iterated in the reverse order.

3.3 KiB Raw Blame History