Site Survey Tool - IncrediBuild Documentation

The Site Survey Tool is an external, executable, command-line tool that allows you to learn more about your environment and the effects Incredibuild might have on your build speeds. This can be used to:

Assess how much of an effect Incredibuild’s distribution and caching will have on your builds
Improve Incredibuild performance by locating any issues in your environment
Troubleshoot issues causing performance degradation

The Site Survey Tool can be run on any machine that has an Incredibuild installed and the tool’s exe file. This tool is generally run in coordination with your Customer Success Manager or Support. If your computer is connected to the internet, you can automatically upload the results of the tool to Incredibuild. This is done automatically using secured FTP. If you cannot connect to the internet or don’t want to upload these files automatically, you can manually send them at any time. These options are controlled in the command line or in the .ini file (details below). You download the Site Survey Tool from XXXX. (or it is included when you install Incredibuild??)

Interactive CLI vs. Non-Interactive CLI

For every different site survey tool command, you can either use it in two ways:

Interactive CLI: During the tool execution, you are required to answer a few questions in the command line. By default, the tool is run in Interactive mode unless the file site_survey.ini exists in the location that you use to run the Site Survey tool.
Non-Interactive CLI: The tool execution does not require interaction, you add the information to a configuration file instead. This is ideal for CI environments. Non-interactive mode is used whenever the file site_survey.ini exists in the location that you use to run the Site Survey tool. This file must be custom configured depending on how you want to use the tool. Details are included below for each mode.

System Overview Mode

The System Overview mode collects information about your machine, the software installed on the machine, the software configurations, Incredibuild logs, and system logs. This is commonly used to verify that you meet the system requirements and check if there is a problem with your firewall, ram, etc. It can be run from any machine that has Incredibuild installed (Agent or Coordinator) To use this mode, run the following command: site_survey.exe /system-overview You will then need to answer the following questions or include them in the site_survey.ini file:

Company name
Build name
Whether or not to upload the output files automatically to Incredibuild automatically (requires internet connection)

Example site_survey.ini file:

[system-overview]
company_name=Dunder Mifflin
build_name=scranton_sales_processing
upload = no

The output is stored in the following three files that are generated in the output_files directory, in the location where you ran the Site Survey tool. Files collected:

C:\Program Files (x86)\Incredibuild\ Logs
C:\\ProgramData\\Incredibuild (exclude FileCache)
EvenLog -> Windows Logs -> Application & System

On coordinator machine additional path is collected:

C:\Program Files (x86)\Incredibuild\Temp

Files created:

output.xlsx – including only System Info
system.xml – system information
SystemStatisticsDB.db – SQLite Database including the following tables:

Disk Information (disk type, disk size)
Environment variables
Firewall (Windows Defender FW open ports based on system requirements per machine type
Network (NIC name, mac, speed, type)
Proc (running processes)
Installed programs
Registry values related for IB installation
Services (running services)
Stats (Statistics) – see perf-duration option for details
System (system information) including:

Hostname
Windows version
IB License
IB version
Coordiantor ID
License key
CPU Model
CPU count
CPU Max Frequency
GPU model
Total Memory
Total Swap Memory
Cloud Provider
VM status

Performance Monitoring Mode

Performance Monitoring Mode collects information about your system performance for a specified number of seconds. This is designed to be run during a build. This tool uses very few resources and should not affect your build speeds significantly. It can be run from any machine that has Incredibuild installed (Agent or Coordinator) To use this mode, run the following command: site_survey.exe /system-overview /perf-duration=<test_duration_in_seconds> For example: site_survey.exe /system-overview /perf-duration=300 If you uploaded the files automatically, the output is sent to Incredibuild and nothing else is required. If you did not, then you can send your Customer Success Manager the zip file that is located in CWD that is specified in the command line output. The output is stored in the output_files/SystemStastics.db file and contains the following information:

Timestamp
CPU usage
Available Memory
Free Swap Memory
I/O read per second
I/O write per second
Bytes sent per second
Bytes received per second

Process Monitor Mode

You can use Process Monitor mode to learn which processes are run during a build. It does this by running a build and monitoring your environment while it runs. This allows you to create a process tree which can be helpful when creating a custom profile.xml file. It can be run from any machine that has Incredibuild installed (Agent or Coordinator), but usually it is run on an Initiator from the PWD to monitor your build environment. It must be run with a user that has admin privileges. To use this mode, run the following command: site_survey.exe /process-monitor You will then need to answer the following questions or include them in the site_survey.ini file:

Company name
Build name and/or comments (keep these as short as possible)
Build command
Clean command
Whether or not to upload the output files automatically to Incredibuild automatically (requires internet connection)

Example site_survey.ini file:

[process-monitor]
company_name=Dunder Miflin
build_name=TestBAC-100 Process Tree
command_line="C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\devenv" "C:\QA\VS2022\TestBAC100\TestBAC100.sln" /rebuild  "Debug|x64"
clean_non_ib_command="C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\devenv" "C:\QA\VS2022\TestBAC100\TestBAC100.sln" /clean  "Debug|x64"
upload = no

If you uploaded the files automatically, the output is sent to Incredibuild and nothing else is required. If you did not, then you can send your Customer Success Manager the zip file that is located in CWD that is specified in the command line output. The output is stored in the following three files that are generated in the output_files directory, in the location where you ran the Site Survey tool.

procmon.db – SQLite DB storing information on all started processes during the build
procmon_tree.txt – aggregated process tree showing all relevant processes child-parent relationships
procmon_not_in_profile.txt – looks for processes related to the build, but not included into embedded profile.xml, provided in txt format

Benchmark Modes

Benchmark modes run a build you specify four times in four different ways:

Without Incredibuild at all
With Incredibuild distribution, but without Build Cache
With Incredibuild distribution and Build Cache to populate the cache
With Incredibuild distribution and Build Cache (local only) to use the cache

These four builds should give you a good picture of how much benefit Incredibuild’s distribution and Build Cache can provide. You can see more details about how Build Cache is functioning in your environment by reading this article. There are three benchmark modes depending on the type of build and environment you are using:

Visual Studio - use the benchmark-vs command and run the command in the Visual Studio command line
Unreal Engine - use the benchmark-ue command and run the command from the same location you would run your build
All Others - use the benchmar-wrap command and run the command from the same location you would run your build

It can only be run on an Incredibuild Initiator machine, from the PWD location where you usually build the project or from the Visual Studio command line for the first option: To use this mode, run one of the following commands:

site_survey.exe /benchmark-vs
site_survey.exe /benchmark-ue
site_survey.exe /benchmark-wrap

You will then need to answer the following questions or include them in the site_survey.ini file:

Company name
Build name and/or comments (keep these as short as possible)
Build Information
1. For Visual Studio:
  - Sln full path in double quotes
  - Build Configuration
2. For Unreal Engine and Others
  - The build command
  - The clean command
Build Directory or CWD
Whether or not to upload the output files automatically to Incredibuild automatically (requires internet connection)

Example site_survey.ini file for Visual Studio:

company_name=Dunder Mifflin
build_name=TestBAC 100
sln_path="C:\QA\VS2022\TestBAC100\TestBAC100.sln"
build_config="Debug|x64"
build_dir="C:\QA\VS2022\TestBAC100\"
upload = no

Example site_survey.ini file for Unreal Engine:

company_name=Dunder Mifflin 
build_name=TestBAC-500
command_line="C:\QA\VS2022\Incremental\UE\UE\Engine\Binaries\DotNET\UnrealBuildTool\UnrealBuildTool.exe" UnrealEditor Win64 Development
clean_non_ib_command="C:\QA\VS2022\Incremental\UE\UE\Engine\Binaries\DotNET\UnrealBuildTool\UnrealBuildTool.exe" UnrealEditor Win64 Development -clean
build_dir="C:\QA\VS2022\Incremental\UE\UE\"
upload = no

Example site_survey.ini file for Wrap (everything else):

company_name=Dunder Mifflin
build_name=TestBAC-500
command_line=ninja
clean_non_ib_command=ninja -t clean
build_dir="C:\QA\TestBAC500_ninja\TestBAC\"
upload = no

SystemStatisticsDB.db - SQLite Database (refer to System-overview mode for details)
Non-IB.log – build log in text format
Non-IB.db – build statistics only (BDE output)
For other builds (IB_dist, bc_population and bc_consumption):
- BDE - collected db file per build (build tasks and statistics)
- BDE – BuildHistoryDB.db all builds information
- Ib_mon – build monitor files
output.xlsx
- Builds statistics per build type:
  - Duration
  - Exist code
  - CPU load in percentage
  - Peek RAM usage
  - CPU load over 90% (in minutes)
  - Count of distributed tasks
  - Total count of tasks
  - Top 3 processes (by execution time)
- System Info (information about machine and software)
- Build Cache Info (BC size)
- Build Info (Company and Build name, Build and Clean command and etc)
- Build Folder info (folder info before and after build, show clean issues)
- BC report
- Build Time chart
- Top 3 processes pie chart
Note: output.xlsx including alerts on outputs which doesn’t meet requirements. Refer to full list of alerts in Output.xlsx – Alerts section

Output.xlsx - Alerts

System Info

CPU cores:

If Initiator (with local cache) and < 8 cores
If helper and < 2 cores
If coordinator and < 4 cores

Total Memory:

if total RAM is lower then twice as CPU cores
for initiator RAM is lower than 16GB (local cache)
for helper RAM is lower than 4GB
for coordinator RAM is lower than 4GB

IB License: If no active license - error VM: If machine is VM – warning Windows FW Status:
Check required ports per installation type (initiator/helper/coordinator).

If one/many required ports are closed prints specific port/ports – error
If no active license also show error, since the ports cannot be checked

Restricted APPs (only if problem found!): Check if antivirus applications (the list of supported antiviruses) exists on machine and can cause problems for IB. If antivirus found, print the name of the application – error Note: Currently supported CrowdStrike, Falcon , Sophos, SentinelOne, McAfee Build Folder info If folderSize_after is lower or equal to folderSize_before, Possible clean problem warning Build Cache report

Cacheable tasks - When percentage of Cacheable tasks from total number of tasks is lower then 50%
taskHits – When percent of consumption taskHits is lower than 80% of total cached tasks

​Interactive CLI vs. Non-Interactive CLI

​System Overview Mode

​Performance Monitoring Mode

​Process Monitor Mode

​Benchmark Modes

​Output.xlsx - Alerts

​System Info