Working from Home
Incredibuild can leverage the computing power of every machine in your environment. However, this requires fast network connections between your computers, whether they are physical machines, virtual machines, or cloud resources.
Work patterns are changing, and work is often done from home environments with less than optimal network speeds. To address this new situation, we have created some best practices when configuring your environment to support working from home.
Network Requirements
-
Bandwidth: 70MB/s or higher. The higher the better.
-
Latency: Below 200MS.
Types of Home Environments
While you can technically work with Incredibuild from any environment, the speed of your network can be a performance bottleneck. Therefore, we recommend connecting to a set of resources that are on the same network such as in the cloud or in your office. If you have no other choice, you can also install Incredibuild on multiple personal devices at home for free.
You can connect directly to an Incredibuild agent in your office using RDP. If you use the agent in your office as the initiator, you will be using your office network and will therefore not be depending on your home network connection. Make sure you are not relying on helper agents that are outside of the office such as other colleagues working from home.
Virtual Desktop Infrastructure (VDI) is defined as the hosting of desktop environments on a central server. It is a form of desktop virtualization, as the specific desktop images run within virtual machines (VMs) and are delivered to end clients over a network.
Since VDI can be accessed at home, but the resources exist in your office or in the cloud, the dependency on your home network is greatly reduced.
You can use the cloud to deploy helper agents on demand using Incredibuild's cloud solution. This solution will only work if your home network connection is strong. It is recommended in the following cases:
-
You have a strong home network connection, but the helper agent machines may not have good connections (other colleagues at home).
-
You have a strong network connection, but you are required to connect to helper agents via a VPN that does not have a strong network connection.
In these cases, you can use your own machine as the initiator and leverage as many agents as you need in the cloud.
If none of the above solutions are possible, and you have more than one machine at home, you can create a mini-home network. This will allow you to leverage the power of idle cores on your machines to speed up your builds.
-
Install Incredibuild on all of your home machines
-
Install the Coordinator on one of your machines
Best Practices to Optimize Performance when Working from Home
In order to maximize your performance when working from home, we recommend using the following best practices if they apply to your situation.
You can build from home without impacting your speed by using Build Cache. Working from home affects build speeds due to limited upstream bandwidth. Build cache lets you reuse previous build data stored on your local machine to drastically reduce build times.
Note: The following recommendations were tested and verified with Unreal Engine 4.24 and 5.0, and an asymmetrical bandwidth connection of 100 Mbps down, 5 Mbps up.
Using Unreal Engine from home with a limited upload bandwidth may result in a significant performance decrease. There are two main reasons for this decrease in performance:
-
Unreal Engine creates in each build very large PCH files.
-
Links for the target toolset (such as Visual Studio compiler toolset) are running remotely, syncing many OBJ and PDB files to remote machines.
These attempts to sync large PCH, OBJ, and PDB files to remote machines over a very limited upload bandwidth, slow down the task execution dramatically, and can even cause abrupt termination of tasks and disconnections. Therefore, in order to overcome the decrease in performance and to achieve acceleration, these two issues need to be addressed.
Disabling the Usage of PCH Files
To increase performance when building with Unreal Engine from home, it is recommended to disable the usage of PCH files. This is done through the BuildConfiguration.xml file of the Unreal Engine version you are using. The BuildConfiguration.xml file is located at: install_dir\Unreal Engine\UnrealBuildTool\BuildConfiguration.xml
> To disable the PCH files usage:
Add the following XML element to the BuildConfiguration.xml file of your Unreal Engine:
<BuildConfiguration>
<bUsePCHFiles>false</bUsePCHFiles>
</BuildConfiguration>
For example, the usage of PCH files was disabled in the following BuildConfiguration.xml file:
<?xml version="1.0" encoding="utf-8" ?>
<Configuration xmlns="https://www.unrealengine.com/BuildConfiguration">
<BuildConfiguration>
<bUsePCHFiles>false</bUsePCHFiles>
</BuildConfiguration>
</Configuration>
Forcing Links to Run Locally
To increase performance when building with Unreal Engine from home, it is also recommended to force links NOT to run remotely, and to avoid syncing large OBJ and PDB files to remote machines over a limited bandwidth connection.
This is done by setting as False the linker flag - LinkAction.bCanExecuteRemotely. By setting it to False, this flag instructs the Unreal Build System that is integrated with Incredibuild to execute all link tasks on the local machine. This flag is applied separately to each toolset, and it resides in the .cs file of the selected toolset.
By default, for most toolsets the LinkAction.bCanExecuteRemotely flag is already set to False. In these cases, the links will run only on your local machine and you do not need to change their settings. However, there are two toolsets for which you need to change the settings:
-
VCToolChain
The settings file is located at:
Install_dir:\UE4\Engine\Source\Programs\UnrealBuildTool\Platform\Windows\VCToolChain.cs
-
HoloLensToolChain
The settings file is located at:
Install_dir:\UE4\Engine\Source\Programs\UnrealBuildTool\Platform\HoloLens\HoloLensToolChain.cs
> To force links to run locally:
-
Open the .cs file that applies to the toolset you are using.
-
Locate the LinkAction.bCanExecuteRemotely flag, and set its value to False.
For example, the links of VCToolChain were set NOT to run remotely in the following VCToolChain.cs file:
If at all possible, we recommend using a physical wired connection to ensure that your network speed is as fast and reliable as possible.
Even if your home network connect is fast, sometime connecting through a VPN can introduce an additional performance bottleneck. If you have to connect through a VPN, make sure the bandwidth is large enough for your needs.
If possible, avoid distributing tasks that require very large input files from your home machine. For example, try to work without utilizing PCH and PDB files.
This option helps prevent cases in which a job "waits" for a slow remote machine to complete execution of a task. If a machine is reacting slowly, Incredibuild executes the task on both the remote machine and your local machine and uses whichever output is completed first.
Since the likelihood of slow remote machines is greater when working from home, we recommend enabling this option.
Go to Agent Settings > Initiator > General, and check Restart remote processes on local machine when possible.
If it’s possible to obtain the same number of cores with fewer machines, this can help reduce the total required network traffic and therefore improve performance.
In some cases, reducing the total number of helper cores per build may improve performance when working from home.
You can change these settings on an individual initiator agent, or in the coordinator to apply the changes to all agents:
-
On an initiator agent, go to Agent Settings > Initiator > General. Check Limit maximum number of cores utilized in build to and reduce the number.
-
On the coordinator, go to Coordinator Settings > Agent Assignment > Advanced and reduce the number.
-
For virtual machines that function as Helpers, use a high number of CPU cores such as 32 or 64. This is not required for virtual machines that function solely as Initiators.
-
When using Incredibuild’s Cloud solution, verify that your pool size is large enough to include all the Helper VMs that you will want to use for accelerating your tasks, in order to avoid cache loss.In the Settings page of your Incredibuild Cloud solution, verify that the Total Cores Limit equals the Number of Cores times the Number of VMsin the Pool. If the Total Cores Limit is higher, VMs will be used and then terminated, losing their cache and reducing efficiency when working from home. In this example: 160 Total Cores=32 Cores * 5 VMs in Pool