Build Cache Overview

This feature is only available in Incredibuild's Enterprise and Business Plans and is currently in limited availability. For more details, speak to your Customer Success Representative.

Overview

Build Cache saves time and resources by reusing portions of previous builds. As you run builds, we store the outputs in a file cache. When future builds are run, we check the cache to avoid regenerating portions of the build that have not changed.

This can have a major impact, as you can reuse previous build data stored in Build Cache to drastically reduce build times without impacting your bandwidth.

Build Cache requires additional resources and has specific System Requirements.

License Model

Build cache uses special licenses called Cache Cores. One Cache Core is consumed whenever using the Build Cache saves you one hour of a helper core's execution. Cache Cores are reusable and are refreshed every hour.

If you run out of Cache Cores, data is still written to the cache, but cannot be read until your Cache Cores are refreshed. You can view information about your license usage at the top of the Agent List.

If you are using Incredibuild Cloud as well, see Using Build cache with Incredibuild Cloud to learn about how these licenses interact.

Supported Tools

  • Build Systems: All build systems supported by Incredibuild.

  • Compiler: cl.exe (Visual Studio 2017 or higher), clang-cl.exe (beta)

  • Linker: link.exe (Visual Studio 2017 or higher)

    • Link.exe support in Build Cache is disabled by default. Enable it if your project has many DLL's or executables that rarely change.

  • /Z7 is fully supported

    • /Zi is automatically converted to /Z7 when Build Cache is used

How is Build Cache Different from the Helper Cache

When Helpers are given a task, the input files (e.g. source and header files) they need to process are stored in the Helper Cache. If the same exact file is sent, the Helpers still process it again, but the files do not need to be re-transferred.

In Build Cache, the Initiator recognizes tasks that have already been processed, and they are skipped entirely. Output files (e.g. object files) do not need to be re-transferred and tasks do not need to be rerun.

Benefits and Use Cases

  • Boost developer productivity from anywhere
    Stop running builds from scratch every time you switch branches, even when they've already been built elsewhere. Build Cache automatically stores previous build data and drastically reduces local build times.

  • Build from home without impacting your speed
    Working from home affects build speeds due to limited upstream bandwidth. Build Cache lets you rely more on downstream bandwidth, giving you greater speed and better performance when starting new builds.

  • Use cached data between CI builds
    Use Build Cache to share build artifacts generated by one CI server with another to save time and resources. Bring the incremental build experience to your CI.

  • Reuse CI results for faster development builds
    Instead of having to incorporate every single change made when restarting a build from version control, Build Cache lets developers reuse outputs generated on later CI builds and workflows, even when working on prior versions.

How Does Build Cache Work? 

Builds are often processed in tree-like structures based on the dependencies in your code. The files in Layer 1 are used to create the files in Layer 2, Layer 2 is used to create the files in Layer 3, and so on until the final output is produced.

As builds are processed, Incredibuild saves details of the process for future use. Each part of the process that is generated from other files is saved as a pair of input files and output files in the cache. If you run a build after changing some of the files, Build Cache can reuse the portions that are identical to prevent parts of the process from running.

The build cache is optimized in various ways such as removing duplicate files and automatic cleanups.

Build Cache Clients and Endpoints

Every Initiator can use Build Cache as a Build Cache Client by contributing and/or reading from the cache. A Build Cache Client uses the cache in one or more Build Cache Endpoints.

Local and Remote (Shared) Cache

By default, each Initiator runs Build Cache locally, meaning that both the Build Cache Client and Endpoint are on the Initiator itself. In addition to the local Build Cache, the Initiator can connect to a remote (shared) cache (you can use both simultaneously).

To create a remote build cache, install an Incredibuild Agent on a dedicated machine that will not be used to assist or initiate builds. The remote cache is shared when multiple Initiators use and/or contribute to this cache.

To configure if an Agent is using a local or remote cache, on the Agent machine go to the Agent Settings > Build Cache Client tab.

Use Cases and Best Practices

Developer Machines - Working from home or in the office

For most machines, we recommend enabling both local and remote cache. This assumes that these machines are persistent (laptops, desktops, or persistent VMs), and that there are other machines (typically CI servers) that use only the remote cache and are writing to that cache. If no machines are writing exclusively using the shared cache, developer machines should only use local cache.

When you enable both local and remote cache on an agent, that agent can only contribute to the local cache.

When a build is run, the local cache is checked first. If there is a hit, the cache is used. Otherwise, Incredibuild checks the remote cache. If there is a hit, we use it and also update the local cache so that future executions will use the local cache and reduce network traffic. If there are no hits, the build is run and the information is added to the local cache.

CI Machines

When using machines for CI use cases, we recommend only using remote (shared) cache. The machines will then both read and write to the remote cache. This ensures that their content will be available even if the CI machine is torn down (as is done in many use cases). This also allows this cache to be shared with your developers working from home or in the office.

Remote (Shared) Cache Hosts

For machines that host a remote/shared cache, use the following recommendations to maximize performance: 

  • Install Incredibuild on an Agent, but do not assign any licenses to the Agent so that it does not assist with or initiator any builds.

  • Do not run other programs on this machine, it should be a dedicated machine.

  • Add an Agent description to clearly identify the Agent as your shared cache.

Limitation

Predictive Execution is not supported with Build Cache. It is automatically disabled when Build Cache is enabled.