Build Cache

Overview

Build Cache saves time and resources by reusing portions of previous builds. As you run builds, we store the outputs in a file cache. When future builds are run, we check the cache to avoid regenerating portions of the build that have not changed.

This can have a major impact, as you can reuse previous build data stored in Build Cache to drastically reduce build times without impacting your bandwidth.

You can use Build Cache locally by creating and reusing content from a single machine, create a shared cache for more than one Initiator (beta feature), or use both local and shared.

To use Build Cache when running projects, add the parameters below to your build commands. This checks to see if there is anything useful in the cache, and stores the build you are running in the cache for future use. If you are using shared cache, you also need to install the Build Cache Service on a dedicated machine.

Using local build cache requires additional storage space on the Initiator machine. We recommend allocating at least twice the size of the project output folder.

Deployment Types

Build cache can be deployed in different ways depending on how your Clients connect to Endpoints. A single Client can function as its own Endpoint (local), multiple Clients can connect to a single Endpoint (remote), and more complex or dynamic deployments are possible as well.

Local Cache refers to using the same Initiator Agents to host the Build Cache Endpoint and Build Cache Client. This means that each agent can only benefit from the cache of builds that were previously run on the same machine. This can be ideal if you are not sharing code with other developers, or if you are working from home with limited bandwidth.

Remote Cache refers to using a single dedicated Agent to host the Build Cache Endpoint. Multiple Initiator Agents contribute and benefit from the same build cache.

You can enable both Local and Remote Caches. If both are enabled, Incredibuild uses the following logic for each task whenever a build is run:

  1. Check the local cache. If there is a cache hit, it is used. If there is a cache miss, the remote cache is checked.

  2. If there is a remote cache hit, use the data and update the local cache.

  3. If there is no remote cache hit, run the task and update the local cache.

Use Cases and Best Practices

Developer Machines - Working from home or in the office

For most machines, we recommend enabling both local and shared cache. This assumes that these machines are persistent (laptops, desktops, or persistent VMs), and that there are other machines (typically CI servers) that use only the shared cache and are writing to that cache. If no machines are writing exclusively using the shared cache, developer machines should only use local cache.

When you enable both local and shared cache on an agent, that agent can only contribute to the local cache.

When a build is run, the local cache is checked first. If there is a hit, the cache is used. Otherwise, Incredibuild checks the shared cache. If there is a hit, we use it and also update the local cache so that future executions will use the local cache and reduce network traffic. This is particularly useful when working from home due to the lower upload bandwidth. If there are no hits, the build is run and the information is added to the local cache.

CI Machines

CI builds can typically be considered a “source of truth”: they usually contain code that is trustworthy, stable, and is shared by other CI builds and developers. For the most part, code that is merged so that it is part of a CI build will change less often. As a result, its compiled artifacts (object files, etc.) are likely to be used by others, improving Build Cache efficiency and faster builds for everyone. When using machines for CI use cases, we recommend only using shared cache. The machines will then both read and write to the shared cache. This ensures that their content will be available even if the CI machine is torn down (as is done in many use cases). This also allows this cache to be shared with your developers working from home or in the office.

Shared Cache Service

For machines that host a shared cache service, use the following recommendations to maximize performance: 

  • Install Incredibuild on an Agent, but do not assign any licenses to the Agent so that it does not assist with or initiate any builds.

  • Do not run other programs on this machine, it should be a dedicated machine.

  • Add an Agent description to clearly identify the Agent as your shared cache.

Installing the Build Cache Service

If you are using shared cache, you need to install the Build Cache Service on a dedicated machine. Use the following format:

sudo bash ./incredibuild_<version>.ubin install -Y -O <Primary_Coordinator_Machine_IP_or_Hostname> -A <Path_to_Incredibuild_Installation_Directory> -Y -Q <size of cache>

For more information, see Installing Components.

Build Cache Parameters

The following parameters can be added to your build command to use Build Cache. Even if Build Cache is enabled, the parameters must be used on every build that you want to use Build Cache.

Parameter Description Notes

--build-cache-local-user

--build-cache-local-shared

--build-cache-local-group="groupname"

Add one of these three commands to use local Build Cache. You can specify whether the cache is accessible and shared across a single user on this machine (-user), everyone on this initiator (-shared), or a specific group of users that includes the current user (-group)

This can be used on it's own, or in addition to a Shared Cache.

Note: These parameters were changed in version 3.19. They used to be --avoid-local-user|group|shared. The original parameters will still work but may be deprecated in future releases.

 

--build-cache-service= <URL>

Use a Shared Cache running on a Build Cache Service defined by the protocol, an IP or hostname, and the port number of the host machine. For exampe: http://build-cache-service:8080.

The protocol can be either http or https, but we recommend using the http protocol to improve performance.

You can define a default Build Cache Service in the Agent Settings or using the CLI. If you have defined a default, you can use this parameter with no URL. If you define a default and use a different URL in the parameter, the value specified in the parameter will be used.

 

--build-cache-basedir=<path>

Incredibuild views files originating from different paths as cache misses, even if the files are identical. This parameter allows you to define which part of the path is NOT shared by other user running similar builds. Incredibuild will then ignore this portion of the path when determining if two files are identical.

For example, if you include usernames as part of your path, (e.g. /home/bill_gates/windows_vista/libs/src/pm-functions.cpp), you might define the basedir as /home/bill_gates/windows_vista/.

This parameter is required.

Note: This parameter was changed in version 3.19. It used to be --avoid-basedir. The original parameter will still work but may be deprecated in future releases.

 

Build Examples

Run local cache only for all users on an initiator

ib_console --build-cache-local-shared --build-cache-basedir=/home/myuser/opencv make -j 40

Run local and shared cache

ib_console --build-cache-local-shared --build-cache-service=http://192.54.11.9:8080 --build-cache-basedir=/home/myuser/opencv make -j 40

Local Cache Settings

You can edit the local Build Cache size in the Agent Settings > General > Local Build Cache Size setting. This can also be configured when installing an Initiator.

You can also run the following commands on an Initiator using local Build Cache to configure other local cache settings: 

  • Return information about the location and size of your local build cache:

    /opt/incredibuild/management/build_avoid_cache.sh <user|shared|group='groupname'> info

  • Set the local build cache size. By default, the size is 100GB, but you can modify it: 

    /opt/incredibuild/management/build_avoid_cache.sh <user|shared|group='groupname'> set_size <size in GB>

  • Clear the local build cache:

    /opt/incredibuild/management/build_avoid_cache.sh <user|shared|group='groupname'> clear

Shared Cache Settings

You can define a default Build Cache Service in the Agent Settings > General > Build Cache Service URL setting. If you use the --build-cache-service parameter with no value, the value defined here will be used. If you specify a different value, it will override the value in the Agent Settings.

You can also modify this setting using the command line interface.

You can also run the following commands on the Build Cache Service machine to configure other settings: 

  • Get the shared build cache size. By default, the size is 100GB: 

    sudo /opt/incredibuild/management/build_cache_service.py get-size-limit

  • Set the shared build cache size. By default, the size is 100GB.

    sudo /opt/incredibuild/management/build_cache_service.py set-size-limit <size>

  • Return the location of the shared build cache on the Build Cache Service machine:

    sudo /opt/incredibuild/management/build_cache_service.py get-path

  • Clear the shared build cache:

    sudo /opt/incredibuild/management/build_cache_service.py clear

  • Return the currently used size of the Build Cache on the Build Cache Service machine:

    sudo /opt/incredibuild/management/build_cache_service.py get-used-size