Further Build Performance Optimizations

Last updated on Nov 05, 2024 This section explains how to further improve Incredibuild’s performance in your environment by fine-tuning your environment settings. Note that each environment and project responds differently to a configuration change, so the best way to go is to try out the different suggestions and see the effect each has on performance. For additional performance-related tips specific to using Incredibuild with MS Visual Studio, see this section.

​

Avoiding Task Execution on the Local Machine

Occasionally, the Agent initiating the build may suffer from a high I/O load as a result of having to manage both file synchronization to and from remote Agents and writing the build’s output to disk. This is especially relevant in the following scenarios:

1. The initiating machine is utilizing more than one CPU for builds.
2. A large number of remote Agents are set to participate in the build.When this is the case, it may be beneficial to prevent the Agent that initiated the build from executing tasks that can also be run by remote Agents – compilation of source files, for example, in order to allow it to dedicate its resources entirely to build management and file synchronization.

The Avoid task execution on local machine whenever possible option can be enabled through the Agent Settings dialog box, Initiator->General page.

​

Restarting Remote Processes on the Local Machine When Possible

When the local machine is idling, this option (which is enabled by default) enables a “competition mode” where the same task also runs on the local initiator machine. This scenario allows speeding up the task execution as it runs in parallel on both machines; to complete the task, the output from the first machine is applied to the build. Note that if the initiating machine has a high I/O load, this option should not be used, as in this case the competition mode is not efficient. The “Restart remote processes on local machine when possible” option can be enabled/disabled through the Agent Settings dialog Initiator->General page.

​

Utilizing Multiple CPUs/Cores

Incredibuild can utilize multiple CPUs and cores when executing a distributed process. This requires the purchase of a Multi-CPU license for each machine with multiple CPUs/cores (in trial licenses, all Agents automatically utilize up to eight available CPUs and cores). Once this is done, distributable tasks are run using each of the available CPUs/cores, depending on each Agent’s current availability. If the machine’s hardware supports Hyper threading, it is possible to use all logical cores in order to improve performance. This can be set under Agents Settings -> Agent -> CPU Utilization.

​

Moving your Build’s Output Folder to a Fast-performing File System Location

In builds involving a large amount of output data, writing the build’s output files to disk can become a bottleneck and slow down the distribution process. This is especially relevant in the following scenarios:

1. The Initiating machine is utilizing more that one CPU for builds.
2. A large number of remote Agents are set to participate in the build.In these cases, it can be very beneficial to set the project’s output folders to a location that will perform as fast as possible. An ideal location for the project’s output file would be a local, high-performance drive that is not running the Windows system.

​

Minimizing Directory Access

In certain environments, a performance gain can be obtained by minimizing the number of accesses to directories performed during builds. This is especially common in the following scenarios:

1. Builds that store input files in network-based file systems.
2. Projects using an input file directory tree, which spreads over an unusually large number of directories.

For Visual Studio builds only, you can use Minimal directory synchronization mode. When this feature is enabled, Incredibuild only performs source-file timestamp checking once, except when it is possible that a non-compilation build step has modified the source file or directory after starting the build (as is the case, for example, in custom build steps that “generate” source files). The minimal directory synchronization mode can be activated using the Agent Settings dialog Initiator->Advanced page.

​

Running the Network Connectivity test

The Network Connectivity Test can locate machines with weak network performance, which is slowing down builds. These machines can then be either upgraded or disabled from participating in remote builds.

​

Disabling Windows File Search Indexing on Intermediate and Output Folders

Making sure that Windows File Search doesn’t index your intermediate and output folders reduces overall disk access and speeds up your builds.

​

Increasing Strong Agents’ Assignment Priority

Incredibuild automatically prefers Agents with stronger hardware over others, when assigning Agents as remote resources in distributed jobs. However, in some cases, it is helpful to fine-tune the Coordinator’s assignment logic, instructing it to prefer certain Agents over others. For example, you might want to increase the assignment priority of Agents that are “dedicated” to function as computing resources in builds and do not run any other jobs. Another example is to increase the assignment priority of Agents running on hardware that Incredibuild underrates for some reason.

​

Disabling Agents on Machines with Weaker Hardware

Machines with slower CPUs and slower Ethernet cards may, in some cases, have a negative impact on overall build speed. If such a machine is being used by a developer who still wishes to run distributed builds, disable that user’s Agent to prevent it from participating in other Agents’ builds; if such a machine is used only as a remote resource, try unsubscribing it and see the effect on build time.

​

Configuring File and Network Scanning Applications

Some “real-time scanning” applications, such as anti-virus and firewall software, can have a negative effect on build performance. Configuring these applications not to scan files with extensions that are regularly used in builds (.obj, .pdb, .idb, .cpp, .c, .h, .sbr, .lib) or not to scan project intermediate folders, may improve performance.

​

Increasing Build Priority

Build priorities are useful when better performance is required for builds initiated by specific Agents. These Agents can then be given a higher priority to ensure that builds they start are assigned more Agents in comparison to others. Build priorities are set from the Coordinator Monitor.

​

Configuring Incredibuild for Operation Alongside CPU-Consuming Background Processes

Some applications are designed to run in the background and consume only as much CPU as is left unused by a PC. One common example is the SETI@home project. As Incredibuild Agents also run in a low process priority, these applications may take away CPU from builds, slowing them down. If any CPU-consuming background application is running on a machine that is also used as an Incredibuild Agent, use the Agent Settings dialog Helper->CPU Availability page to configure Incredibuild to run alongside this application.

​

Increasing the File Cache Size

Incredibuild caches files from remote Agents in a special file cache stored under the installation folder. If there is not enough space to store all of the files, additional synchronization is performed over the network, resulting in extra network load and decreased performance. To see if the file cache is full and needs resizing, see the “File Cache” column in the Coordinator Monitor.

​

Configuring Parallel / Sequential Build for “Batch Build” Feature that Builds Many Configurations at Once

In cases where problems are encountered when building several configurations at the same time, the Batch Build feature, the ‘BatchBuildParallelExecution2010’ registry key (under HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Xoreax\Incredibuild\Builder) can be used. By default, this key is set to 1 (parallel building); setting it to 0 forces each configuration to be built separately (sequential building).