I recently finished all of Games104 on Bilibili. One lesson discussed multithreading and the Job System, so I decided to record some of the key points and introduce Unity’s C# Job System for parallel processing.
Multithreaded programming means distributing program tasks across multiple threads so hardware can be used more efficiently and the program can be accelerated.
Why do we need the Job System? C# itself provides APIs such as Thread.Start and TaskFactory.StartNew. How does the Job System’s multithreading differ from calling those APIs directly? Before discussing the differences, we need to understand the advantages and disadvantages of writing multithreaded programs.
Advantages of multithreaded programming
Move computation off the main thread to relieve CPU-bound work. For a game client, the main thread is the battlefield where all project code competes for resources, yet players expect it to finish all calculations within 16.66 ms (60 FPS).
On modern CPUs, single-core clock-speed growth has slowed, while multicore CPUs are the market trend.
Common problems in multithreaded programming
Compared with a single-threaded program, multithreaded programs commonly encounter:
- Race conditions
- Deadlocks
- Context switches
- Compiler optimization and other issues
Race condition
When different threads simultaneously read and write the same data, a race condition occurs. In the following example, suppose a value in Data is initially 2 and threads A and B both add 5 to it at the same time. As observers, we naturally expect the result to be 2 + 5 (A) + 5 (B) = 12. In reality, both threads may read the value as 2, and when they write their results, one overwrites the result calculated by the other. The output becomes 7.
When a program contains a race condition, it becomes unstable and difficult to debug because the unexpected result is often caused by a tiny difference in execution timing.
Deadlock
A common way to prevent race conditions is to use a lock. When a thread wants to read or write some data, it acquires a lock; any other thread that wants to modify that data must wait for the thread holding the lock to unlock it. But what if it never unlocks? What if the thread dies because of an exception? That situation is a deadlock, and the other threads wait forever. The program soon crashes.
Context switch
The number of CPU cores is limited, and the number of threads created by a program can exceed the number of cores that can execute at once. In that case, a CPU core must pause the currently running thread task and execute a higher-priority thread task instead. The CPU core’s act of switching threads is a context switch.
During a context switch, the current thread’s state must be recorded before switching to the new thread so execution can resume later. After switching, the data needed by the new thread may not be in the CPU cache, so the CPU must fetch it from RAM again.
All of these operations are expensive. They can take from 10,000 to 1,000,000 nanoseconds (reading data from L1 cache takes only about 1 nanosecond), so when writing game code we want to avoid context switches as much as possible.
Compiler optimization (out-of-order execution and memory reordering)
Many people do not realize that different CPUs optimize code to different degrees. To maximize execution efficiency and instruction throughput, modern CPUs do not necessarily execute every instruction in source order. They only need to guarantee that the final output is the same.
In the diagram below, suppose func1 (Thread A) and func2 (Thread B) are called at the same time. Will func2’s flag variable be true or false?
Looking only at the code for func1 on the left, we expect that when b is 0, a has already been set to 2, so flag in func2 must be true.
With out-of-order execution (see the image below), however, when b is set to 0, a may not yet have been updated to 2, producing an unexpected result. This is even more likely in multithreaded code.
Anyone who has built a game or app client should be familiar with options such as release build and debug build. Compilers often optimize release builds automatically, strip unnecessary code, and sometimes obfuscate the code. As a result, multithreaded code can have bugs that appear only in release builds.
Is multithreaded programming so difficult that I should just keep writing single-threaded code?
Fiber-based Job System
A fiber-based Job System solves the bugs and inefficiencies caused by the issues above. The following is a simple diagram.
A short summary of the Job System
- Use a one-to-one relationship between worker threads and CPU cores to avoid context switches.
- Put logic in jobs rather than in threads.
- Use a scheduler to distribute jobs among worker threads.
- Use dependencies between jobs to implement complex task combinations.
- To avoid race conditions, a synchronization point follows job execution. After the synchronization point, we can guarantee that no other thread is reading or writing the same data.
C# Job System
Unity’s Job System scheduler generally follows the fiber-based Job System concept above. Unity’s official diagram shows the relationship between the Job Queue, worker threads, and CPU cores (on the right side of the image below).
Safety System
Another feature of Unity’s Job System is that data manipulated inside a job must be a value type. This prevents different threads from modifying data through references and completely avoids race conditions.
Native Container
The disadvantage of using only value types is that data is isolated inside each job. Suppose jobs 1, 2, and 3 all want to perform different calculations on an object’s position; using value types would be inconvenient. To solve this, Unity provides containers such as Native Containers, allowing different jobs to share a block of memory. The Safety System automatically tracks all jobs currently reading or writing that memory and reports an error if a race condition occurs.
- A Native Container is, as its name suggests, a C# wrapper around native memory. It is not tracked by the GC, so it must be disposed of manually.
- A Native Container may be read by multiple jobs in parallel, but only one job may write to it at a time.
JobHandle and dependencies
An important feature of the Job System is the ability to set dependencies between jobs.
For example, imagine a system that makes every AI agent walk toward and attack the player. The first job updates the player’s position, the second updates each agent’s pathfinding path, and the last moves all AI agents. The jobs therefore need an explicit order of dependency.
Unity’s Job System provides the JobHandle structure for setting dependencies. Each call to Job.Schedule() returns a JobHandle, which can be passed as an argument when scheduling a dependent job.
What is the C# Job System used for?
Here are some common uses of Unity’s C# Job System:
Run a task in the background to reduce main-thread load. Common examples include updating in-game AI pathfinding results and rebuilding a navmesh; these tasks are well suited to background execution.
Large amounts of computation. Parallel processing is the strength of multithreading, especially in Unity’s Job System, which provides jobs designed specifically for parallel processing. Such jobs can split a long array calculation into smaller jobs and distribute them across threads. CPU culling, joint IK calculations in skeletal animation, and CPU cloth simulation are common applications.
Data-oriented programming. The special part of Unity’s Job System is that it is fully compatible with the Burst Compiler. Unity’s Burst Compiler can compile selected C# code into SIMD-friendly machine code. Combined with tightly packed data such as a NativeArray, this can make processing large amounts of data 30–100 times faster.
When using Unity’s Job System, blindly adding
[BurstCompile]to every job can multiply its processing speed by N.
Summary
Unity’s C# Job System feels restrictive at first (the value-type limitation, operating on Native Containers, and so on), but after getting used to it, the barrier to writing multithreaded code becomes much lower. It helps us avoid many of the traps of multithreaded programming, and the effortless speedup from Burst is extremely powerful.
I have not used Unity’s newly released ECS 1.0 yet, but I have used the Job System many times to break through performance bottlenecks in existing programs. If you are interested in Unity DOTS, starting with the C# Job System is a good way to become familiar with the concepts.
That is all for this brief introduction. I will explain the practical use of each type of Job in the Job System when I have time.