Incredibuild Components and Architecture

Incredibuild consists of two types of machines: Agents and Coordinators.

Agents

Agents are machines that run your workloads. Agents can act as Initiators or Helpers with respect to a particular build. Agents that are requesting a workload are called initiators. Agents that are assisting the initiator with some of their idle CPU resources are called helpers.

  • Initiator Agents Once assigned with a proper solution an Agent can initiate executions with Incredibuild. Tasks spawned as part of these execution may be distributed by Incredibuild across the various idle CPU's on Agents available as part of the Incredibuild pool, resulting in much faster execution. Every Initiator Agent can also function as a Helper Agent.

  • Helper Agents These machines are used to execute distributed tasks. Helper Agents don't need to have anything besides an Incredibuild Agent. For example, if you are distributing Visual Studio compilations using Incredibuild to Helper A, this Helper doesn't need to have Visual Studio installed on it, neither the compiler or other build tools nor the source code you are compiling, only an Incredibuild Agent should be installed on this host machine. Once a job that was assigned to a Helper Agent is completed, the output is synced back to the Initiator machine (such as object files, pdb files, etc).

Coordinator

The Coordinator is responsible for listing and managing all the machines that are part of Incredibuild Environment. Every Incredibuild Environment should include only one Coordinator, and the Coordinator can be installed on any participating machine.

The Coordinator performs the following functions:

  • Subscribes and unsubscribes Agents from participating within the resource pool.

  • Maintains up-to-date Agent information through Status Reports received from Agents.

  • When tasks are waiting to be distributed, the Coordinator provides the Initiator with the necessary Helper agents.

  • The Coordinator is responsible for allocating Incredibuild solutions and core packages to Agents.

The Coordinator communicates with the Helper Agents to identify the availability of their CPU resources. When a request for resources arrives from an Initiator Agent, the Coordinator provides to the Initiator the available resources of the Helper Agents in the Environment. The Coordinator is also responsible for allocating Incredibuild licenses to the Agents.

In addition to the primary Coordinator, it is also possible to install and use a Backup Coordinator. This Backup Coordinator serves as a backup to the primary Coordinator, and it is activated automatically when the primary Coordinator is unavailable.  

Example Workflow

  1. The Initiator calculates how many additional remote cores it requires.

  2. The Initiator sends a request to the Coordinator for necessary number of Helper Agents.

  3. The Coordinator scans the network, checks where resources are available, and assigns them for the Initiator to use.

For example, if the Initiator wants to execute 10 concurrent processes in a distributed manner and requests for 10 Helper cores, it may receive all or some of them from the Coordinator. After completing the execution of these 10 processes, if the Initiator needs to distribute 8 additional processes, as it already has 10 Helper cores assigned to it, it will not request additional remote cores from the Coordinator but will instead release two remote cores back to the resource pool.