> ## Documentation Index > Fetch the complete documentation index at: https://docs.incredibuild.com/llms.txt > Use this file to discover all available pages before exploring further. # Scaling Helpers in AWS EC2 > Set up AWS Auto Scaling Groups to scale Incredibuild Helper Agents automatically. This topic explains how to scale Incredibuild Helper Agents automatically with AWS Auto Scaling Groups, using the AWS CLI or the AWS Management Console. ## Prerequisites 1. Create an Incredibuild grid that consists of the following: * Initiator * Coordinator * Helper (on Windows, SCL or Linux) which will be used to create the Launch Template. 2. Ensure that you know the credentials or have the PKI key of the Helper machine. 3. Ensure that **hostname** of the Helper machine is **NOT** static. 4. Ensure that the Incredibuild grid (especially the Helper) is working prior to creating the Auto Scaling group. Do this by testing the build capability and ensuring the Helper is used during the build. ## Create an Auto Scaling Group using AWS CLI 1. Copy the following values from the Incredibuild grid: * **Helper instance:** `` * **Subnet:** `` * **Security group or list of security groups:** `` 2. Select a VM size for machines to be created by the Auto Scaling group: * **Instance type:** ``, for example, **c5a.2xlarge** 3. Stop the Helper instance: ```bash theme={null} aws ec2 stop-instances \ --instance-ids ``` 4. Create the **AMI** (image) from the Helper instance: ```bash theme={null} aws ec2 create-image \ --instance-id \ --name "Helper-ImageName" \ --tag-specifications "ResourceType=image,Tags=[{Key=Name,Value=Helper-ImageName}]" ``` 5. Copy the AMI ID ``, for example: ```bash theme={null} "ImageId": "ami-0df8cbefb46402855" ``` 6. Check the AMI creation state: ```bash theme={null} aws ec2 describe-images --image-id --query 'Images[0].State' ``` 7. When the returned **State** is **available**, create the Launch Template: ```bash theme={null} aws ec2 create-launch-template --launch-template-name Helper_Template --version-description Helper_Template_v1 \ --launch-template-data '{"NetworkInterfaces":[{"DeviceIndex":0,"SubnetId":"","Groups": [""]}],"ImageId":"","InstanceType":""}' ``` 8. Note the launch template ID ``, for example: ```bash theme={null} "LaunchTemplateId": "lt-02d7a3f031e94d432" ``` 9. Create the Auto Scaling group, changing the **max-size** to the required integer value: ```bash theme={null} aws autoscaling create-auto-scaling-group \ --auto-scaling-group-name Helper_ASG --launch-template LaunchTemplateId= \ --min-size 1 --max-size 5 --desired-capacity 1 --default-cooldown 30 --default-instance-warmup 60 --vpc-zone-identifier "" ``` 10. Add a scaling policy to the Auto Scaling group based on metrics of CPU utilization (50%): ```bash theme={null} aws autoscaling put-scaling-policy --auto-scaling-group-name Helper_ASG \ --policy-name cpu50-target-tracking-scaling-policy \ --policy-type TargetTrackingScaling \ --target-tracking-configuration '{"TargetValue": 50.0, "PredefinedMetricSpecification": {"PredefinedMetricType": "ASGAverageCPUUtilization"}}' ``` **Full example:** ```bash theme={null} aws ec2 stop-instances \ --instance-ids i-03642a8f7975ac99f # Wait until the instance stops by running the following command: aws ec2 wait instance-stopped --instance-ids i-03642a8f7975ac99f aws ec2 create-image \ --instance-id i-03642a8f7975ac99f \ --name "Helper-ImageName" \ --tag-specifications "ResourceType=image,Tags=[{Key=Name,Value=Helper-ImageName}]" # Copy the , for example: "ImageId": "ami-0df8cbefb46402855" # Wait until the AMI creation finishes by running the following command: aws ec2 wait image-available --image-ids ami-0df8cbefb46402855 aws ec2 create-launch-template --launch-template-name Helper_Template --version-description Helper_Template_v1 \ --launch-template-data '{"NetworkInterfaces":[{"DeviceIndex":0,"SubnetId":"subnet-0b184d86f43ece9ce","Groups": ["sg-0629670940885903c"]}],"ImageId":"ami-0df8cbefb46402855","InstanceType":"c5a.2xlarge"}' # Copy the , for example: "LaunchTemplateId": "lt-02d7a3f031e94d432" aws autoscaling create-auto-scaling-group \ --auto-scaling-group-name Helper_ASG --launch-template LaunchTemplateId=lt-02d7a3f031e94d432 \ --min-size 1 --max-size 5 --desired-capacity 1 --default-cooldown 30 --default-instance-warmup 60 --vpc-zone-identifier "subnet-0b184d86f43ece9ce" aws autoscaling put-scaling-policy --auto-scaling-group-name Helper_ASG \ --policy-name cpu50-target-tracking-scaling-policy \ --policy-type TargetTrackingScaling \ --target-tracking-configuration '{"TargetValue": 50.0, "PredefinedMetricSpecification": {"PredefinedMetricType": "ASGAverageCPUUtilization"}}' ``` ## Create an Auto Scaling Group using AWS Console ### Gather Incredibuild Initiator Information 1. Sign in to the AWS Management Console. 2. Navigate to the **EC2 > Instances**. 3. Select the Initiator instance and copy the following info: * **VPC ID** * **Subnet ID** AWS Auto Scaling Groups screenshot

4. **Select the Security tab and copy the required Security Group.** AWS Auto Scaling Groups screenshot

### Create an AMI from Helper Machine 1. Sign in to the AWS Management Console. 2. Navigate to **EC2 > Instances.** 3. Select the Helper instance and choose **Instance state > Stop instance**. When prompted for confirmation, choose **Stop**. It can take a few minutes for the instance to stop. 4. To create the AMI, select the Helper instance, select **Actions > Image and templates > Create image**. 5. Enter the **Image name**. 6. Select **Tag image and snapshots together**. AWS Auto Scaling Groups screenshot

7. Click **Create image**. 8. Copy the **Image name** for future use. AWS Auto Scaling Groups screenshot

### Create Template from AMI 1. In **EC2 > Instances**, select **Launch Templates > Create launch template**. 2. Enter the **Launch template name**. 3. Copy the **Launch template name** for future use. AWS Auto Scaling Groups screenshot

4. Under **Application and OS Images (Amazon Machine Image),** paste the AMI **Image name**. AWS Auto Scaling Groups screenshot

3. Press **Enter**. The **Choose an Amazon Machine Image (AMI)** page appears. 4. Select your AMI image and then click **Select**. AWS Auto Scaling Groups screenshot

5. Scroll down and configure the **Network settings**: * From the **Subnet** dropdown, select the **SubnetID**. * From the **Common security groups** dropdown, select the **Security Group ID** of the grid. AWS Auto Scaling Groups screenshot

6. Click **Create Launch template**. AWS Auto Scaling Groups screenshot

7. Once created, a **Success** message appears. AWS Auto Scaling Groups screenshot

### Create Auto Scaling Group from Template 1. In EC2, select **Auto Scaling > Auto Scaling Groups**. 2. Click **Create Auto Scaling group**. A wizard appears. 3. In **Step 1 - Choose launch template or configuration**, configure the following: * Enter the **Auto Scaling Group name**. * From the **Launch template** dropdown, select the template to be used for machine creation in the Auto Scaling group (created in Create Template from AMI). AWS Auto Scaling Groups screenshot

4. Click **Next**. 5. In **Step 2- Choose instance launch options** configure the following: * In the **Instance type requirements** area, select the base model for the Helper machine to be used for scaling. Either select a specific AWS instance type or configure it manually. AWS Auto Scaling Groups screenshot

* (Optional) In the **Instance purchase options** area, select **On-Demand** or **Spot** instances. AWS Auto Scaling Groups screenshot

* In the **Network** area, select the **VPC ID** and **Subnet ID** copied from the Initiator configuration. AWS Auto Scaling Groups screenshot

6. Click **Next**. 7. In **Step 3(optional) - Configure advanced options**, click **Next.** 8. In **Step 4 (optional) - Configure group size and scaling**, provision the scaling of the Helpers group: * In the **Group size** area, enter the **Desired capacity of the group**. * In the **Scaling** area, enter the **Min desired capacity** and **Max desired capacity** (minimum and maximum number of Helpers in the grid). * Select the **Target tracking scaling policy**. * From the **Metric type** dropdown, select **Average CPU utilization**. This metric is used for sizing. * Enter a **Target Value**, for example, 50, meaning 50% CPU usage. * In **Instance warmup**, enter the number of seconds that it takes for a newly launched instance to warm up. Until its specified warmup time has expired, an instance is not counted toward the aggregated EC2 instance metrics. In this example, 60 seconds. AWS Auto Scaling Groups screenshot

9. Click **Next** until you get to the end of the wizard. 10. Click **Skip to review** at the bottom of the page. 11. In the **Review** page, scroll down and click **Create Auto Scaling group**. The **Auto Scaling Groups** page appears. AWS Auto Scaling Groups screenshot

12. Note that the first Helper instance is started (**Updating capacity**). 13. Click on group **Name**. 14. On the **Details** tab, scroll down to **Advanced configurations** and click **Edit**. 15. Set the **Default cooldown** to **0** second. 16. Click **Update**. This means that instances are terminated immediately when there is no need for scale capacity. AWS Auto Scaling Groups screenshot

17. On the **Activity** tab, view the log of the latest actions. Each time an additional instance is created or terminated, it appears in the log. AWS Auto Scaling Groups screenshot

18. On the **Instance management** tab, view the details of the active instances. For example, when a new Helper is created. The new Helper instance also appears in the Coordinator Management page a short while later. AWS Auto Scaling Groups screenshot

19. Now start the build with Incredibuild. When the CPU utilization on the Helper machine exceeds 50%, new instances are created. 20. Navigate to your group in **EC2 > the Auto Scaling groups**, and view the **Activity history** in the **Activity** tab. AWS Auto Scaling Groups screenshot

## Create an Auto Scaling Group with Mixed Instance Types ### Create an Auto Scaling Group with Mixed Instance Types using AWS Console 1. To create an Auto Scaling group with mixed instance types, in **Step 2- Choose instance launch options** click **Override launch template**. AWS Auto Scaling Groups screenshot

2. Configure the specific **Instance types requirements** and the **On-Demand** and **Spot** instances. AWS Auto Scaling Groups screenshot

### Create an Auto Scaling Group with Mixed Instance Types using AWS CLI * To create an Auto Scaling group with mixed instance type in AWS CLI: ```bash theme={null} ASG_NAME="Helper_SCL_ASG" LAUNCH_TEMPLATE_ID="lt-0b66a45ee243120f9" MIN_SIZE=1 MAX_SIZE=5 DESIRED_CAPACITY=1 COOLDOWN=30 WARMUP=60 SUBNETS="subnet-0b184d86f43ece9ce" INSTANCE_TYPE_1="c5a.2xlarge" INSTANCE_TYPE_2="c6a.2xlarge" aws autoscaling create-auto-scaling-group \ --auto-scaling-group-name "$ASG_NAME" \ --mixed-instances-policy '{"LaunchTemplate":{"LaunchTemplateSpecification":{"LaunchTemplateId":"'"$LAUNCH_TEMPLATE_ID"'"},"Overrides":[{"InstanceType":"'"$INSTANCE_TYPE_1"'","WeightedCapacity":"1"},{"InstanceType":"'"$INSTANCE_TYPE_2"'","WeightedCapacity":"1"}]},"InstancesDistribution":{"OnDemandBaseCapacity":0,"OnDemandPercentageAboveBaseCapacity":100,"SpotAllocationStrategy":"lowest-price"}}' \ --min-size $MIN_SIZE \ --max-size $MAX_SIZE \ --desired-capacity $DESIRED_CAPACITY \ --default-cooldown $COOLDOWN \ --default-instance-warmup $WARMUP \ --vpc-zone-identifier "$SUBNETS" ``` ## Create an SCL Helper for the Auto Scaling Group ### Create an SCL Helper A VPC, subnet and security groups are required to connect the Auto Scaling group instance to an existing Incredibuild grid (Coordinator/Initiator machines) and to your machine for AMI updates. 1. In the AWS Management Console, navigate to **EC2 > AMI Catalog**. 2. Filter, locate, and select the required AMI. AWS Auto Scaling Groups screenshot

3. In the upper-right corner, click **Launch instance from AMI**. The **Launch an instance** page appears. 4. Complete the following: * **Name:** Helper-SCL * **Instance type:** c5a.2xlarge * **Key pair (login):** If you already have a key pair that you use to connect to your instances, select it. Otherwise, choose **Create new key pair**. A `.pem` file will be downloaded and can be used to connect to the instance. * **Network settings:** Select the appropriate VPC, subnet, and security group(s). * **Configure storage:** At least **20 GB**. 5. Click **Launch instance**. AWS Auto Scaling Groups screenshot

6. Copy the **Instance ID** (for example, **i-0b0edfce604652eff**) and open the instance details page. 7. Once the instance reaches the **Running** state, note its **Public IPv4 address**. Use SSH or SCP to connect to the instance. On Windows, it is recommended to use a Bash-based SSH client, such as Git Bash or MobaXterm. 8. Before proceeding, obtain the Incredibuild console installer (`ibsetup__console.exe`). Ensure that the installer version matches the Incredibuild version deployed in your grid. 9. Create a folder named **SCL** and copy the installer into it. 10. Within the **SCL** folder, create a file named **Dockerfile** with the following contents: ```dockerfile theme={null} FROM ubuntu:22.04 # Declare build-time arguments ARG HELPER_CORES ARG SERVICE_PORT ARG DEFAULT_SERVICE_PORT=31105 ARG HELPER_PORT ARG DEFAULT_HELPER_PORT=31106 ARG COORD_IP # Make them available as environment variables ENV HELPER_CORES=$HELPER_CORES \ SERVICE_PORT=$SERVICE_PORT \ DEFAULT_SERVICE_PORT=$DEFAULT_SERVICE_PORT \ HELPER_PORT=$HELPER_PORT \ DEFAULT_HELPER_PORT=$DEFAULT_HELPER_PORT \ COORD_IP=$COORD_IP # Specify a workdir to better organize files RUN mkdir -p /ib-agent WORKDIR /ib-agent # Update package lists and install necessary packages RUN apt-get update && \ apt-get install -y --no-install-recommends wget software-properties-common gnupg2 winbind xvfb curl vim # Add WineHQ GPG key and store it in /usr/share/keyrings/ RUN wget -nc https://dl.winehq.org/wine-builds/winehq.key && \ gpg --no-default-keyring --keyring /usr/share/keyrings/winehq-archive-keyring.gpg --import winehq.key # Add the WineHQ repository with the signed-by option RUN echo "deb [signed-by=/usr/share/keyrings/winehq-archive-keyring.gpg] https://dl.winehq.org/wine-builds/ubuntu/ jammy main" > /etc/apt/sources.list.d/winehq.list # Update package lists and install Wine RUN dpkg --add-architecture i386 && \ echo 'quiet = on' >> ~/.wgetrc && \ apt-get update && \ apt-get install -y --no-install-recommends wine-stable=8.0.2~jammy-1 wine-stable-amd64=8.0.2~jammy-1 wine-stable-i386=8.0.2~jammy-1 # Install winetricks and configure Wine #RUN apt-get install -y winetricks && \ # winetricks msxml6 wininet RUN apt-get install -y winetricks && \ wget -O /tmp/msxml6-KB973686-enu-amd64.exe "https://s3.us-east-1.amazonaws.com/downloads.incredibuild.com/public+s3+uploads/winetricks/MSXML6-KB973686-enu-amd64.exe" && \ mkdir -p ~/.cache/winetricks/msxml6 && \ cp /tmp/msxml6-KB973686-enu-amd64.exe ~/.cache/winetricks/msxml6/ && \ winetricks msxml6 wininet # Cleanup unnecessary files RUN apt-get clean && \ rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* # Create a symbolic link for the wine command to point to wine64 binary RUN ln -s /opt/wine-stable/bin/wine64 /usr/local/bin/wine # Create x11.reg file with the specified content RUN echo 'REGEDIT4\n\n[HKEY_CURRENT_USER\\Software\\Wine\\X11 Driver]\n"UseXRandR"="N"\n"UseXVidMode"="N"' > ./x11.reg RUN /opt/wine-stable/bin/wine64 regedit ./x11.reg # Copy install required packages COPY ./ibsetup_*_console.exe ./ib_setup_console.exe COPY ./winemodal.sh ./winemodal.sh RUN chmod +x ./winemodal.sh RUN ["sh", "-c", "xvfb-run -a ./winemodal.sh wine ib_setup_console.exe /Install /Components=Agent /Agent:AgentRole=Helper /Agent:HelperType=Floating /Agent:SingleUse=True /Agent:InstallAddins=OFF /Agent:OpenFirewall=OFF /Agent:AutoSelectPorts=OFF /Agent:HelperCores=$HELPER_CORES /Agent:ServicePort=${SERVICE_PORT:-$DEFAULT_SERVICE_PORT} /Agent:HelperPort=${HELPER_PORT:-$DEFAULT_HELPER_PORT} /Coordinator=$COORD_IP"] RUN ["sh","-c","cat >./ib-group.reg <<'EOF'\nWindows Registry Editor Version 5.00\n\n[HKEY_LOCAL_MACHINE\\Software\\Xoreax\\IncrediBuild\\BuildService]\n\"Group\"=\"Universal Helpers\"\n\"GroupId\"=\"__UNIVERSAL__\"\nEOF"] # Set the CMD to start incredibuild_agent #CMD ["sh", "-c", "wine sc start incredibuild_agent; sleep 10; wine sc query incredibuild_agent | grep -q RUNNING || exit 1; sleep infinity"] CMD ["sh", "-c", "wine regedit.exe /S ./ib-group.reg; xvfb-run -a ./winemodal.sh wine sc start incredibuild_agent; sleep 10; xvfb-run -a ./winemodal.sh wine sc query incredibuild_agent | grep -q RUNNING || exit 1; sleep infinity"] ``` * **Note:** The Helper is created in the **Universal Helpers** build group, and it will be able to allocate Helper cores to all available Initiators in the grid. 11. Create a file named **winemodal.sh** with the following contents: ```bash theme={null} #!/bin/bash -u export WINEDEBUG=-all setupLogFile="./winemodal.log" echo "$(date +'%Y-%m-%d %T'): executing $@" >> $setupLogFile $@ ``` 12. Tar (or 7zip on Windows) the SCL folder with the three files: * **winemodal.sh** * **Dockerfile** * `ibsetup__console.exe` 13. In Bash, for example, Git Bash: ```bash theme={null} # Copy the relevant files to the new instance created in the previous step # Provide the location of the pem file and SCL.tar files scp -i "Desktop/WINDOWS/AWS_keys/yuri_temp.pem" Desktop/CLOUD_HELPERS/SCL.tar ubuntu@10.133.9.86:. ``` 14. Once the file is copied to the instance, connect to the instance with SSH: ```bash theme={null} ssh -i "Desktop/WINDOWS/AWS_keys/yuri_temp.pem" ubuntu@10.133.9.86 ``` 15. Once logged in, run the following commands: ```bash theme={null} sudo apt update && sudo apt install -y docker.io sudo usermod -aG docker $USER exit ``` 16. Connect again with SSH: ```bash theme={null} ssh -i "Desktop/WINDOWS/AWS_keys/yuri_temp.pem" ubuntu@10.133.9.86 ``` 17. Once logged in, run the following commands: ```bash theme={null} tar -xvf SCL.tar cd SCL docker build \ --build-arg HELPER_CORES="$(nproc)" \ --build-arg COORD_IP="10.133.27.8" \ -t scl . # You may use the following commands for testing purposes # docker run -d --network host --name scl scl # docker rm -f scl sudo tee /etc/systemd/system/scl.service <<'EOF' [Unit] Description=SCL Container After=docker.service Requires=docker.service [Service] Restart=always ExecStartPre=-/usr/bin/docker container prune -f ExecStart=/usr/bin/docker run --rm \ --name scl \ --network host \ scl ExecStop=/usr/bin/docker stop scl [Install] WantedBy=multi-user.target EOF sudo systemctl daemon-reload sudo systemctl enable scl.service sudo poweroff ``` ### Create an AMI from SCL Helper using AWS Console 1. Sign in to the AWS Management Console. 2. Navigate to **EC2 > Instances.** 3. Select the Helper instance and choose **Instance state > Stop instance**. When prompted for confirmation, choose **Stop**. It can take a few minutes for the instance to stop. 4. To create the AMI, select the Helper instance, select **Actions > Image and templates > Create image**. 5. Enter the **Image name** (**Helper-SCL**). 6. Add a new tag with `Key=Name` and `Value=Helper-SCL`. 7. Click **Create image**. ### Create an AMI from SCL Helper using AWS CLI * To create an AMI from SCL Helper in AWS CLI: ```bash theme={null} aws ec2 stop-instances \ --instance-ids i-0b0edfce604652eff # Wait until instance stops: aws ec2 wait instance-stopped --instance-ids i-0b0edfce604652eff aws ec2 create-image \ --instance-id i-0b0edfce604652eff \ --name "Helper-SCL" \ --tag-specifications "ResourceType=image,Tags=[{Key=Name,Value=Helper-SCL}]" # Note , for example: "ImageId": "ami-06c5b4e97014be3d9" # Wait for AMI creation to finish by running the following command: aws ec2 wait image-available --image-ids ami-0e6d0cc30a40e4ad4 ``` ## Create a Windows Helper for the Auto Scaling Group ### Create a Windows Helper A VPC, subnet and security groups are required to connect the Auto Scaling group instance to an existing Incredibuild grid (Coordinator/Initiator machines) and to your machine for AMI updates. 1. In the AWS Management Console, navigate to **EC2 > AMI Catalog**. 2. Filter, locate, and select the required AMI. 3. In the upper-right corner, click **Launch instance from AMI**. The **Launch an instance** page appears. 4. Complete the following: * **Name:** Helper-WIN * **Instance type:** c5a.2xlarge * **Key pair (login):** If you already have a key pair that you use to connect to your instances, select it. Otherwise, choose **Create new key pair**. A `.pem` file will be downloaded and can be used to connect to the instance. * **Network settings:** Select the appropriate VPC, subnet, and security group(s). * **Configure storage:** At least **20 GB**. 5. Click **Launch instance**. 6. Copy the **Instance ID** (for example, **i-0b0edfce604652eff**) and open the instance details page. 7. Once the instance reaches the **Running** state, connect to it using RDP. 8. Download the **Remote Desktop File**. 9. Click **Get Windows password**, provide the **.pem** key file, and decrypt the administrator password. 10. Use the downloaded **RDP** file and the decrypted password to connect to the Windows instance using an RDP client. 11. Before starting, prepare the Incredibuild installation file for either GUI or console installation. Use the same version that is installed in your Incredibuild grid (`ibsetup_.exe` or `ibsetup__console.exe`). 12. Copy the installation file to the Windows instance and run the Incredibuild installation using either the GUI or the command prompt. 13. If using the command prompt: ```bash theme={null} set "HELPER_CORES=%NUMBER_OF_PROCESSORS%" set "COORD_IP=10.133.27.8" set "HELPER_PORT=31105" set "SERVICE_PORT=31106" ibsetup_10.32.0.11685_console.exe /Install /Components=Agent /Agent:AgentRole=Helper /Agent:HelperType=Floating /Agent:SingleUse=True /Agent:InstallAddins=OFF /Agent:OpenFirewall=OFF /Agent:AutoSelectPorts=OFF /Agent:HelperCores=%HELPER_CORES% /Agent:ServicePort=%SERVICE_PORT% /Agent:HelperPort=%HELPER_PORT% /Coordinator=%COORD_IP% /Agent:Group="Universal Helpers" ``` * **Note:** The Helper is created in the **Universal Helpers** build group, and it will be able to allocate Helper cores to all available Initiators in the grid. 14. Once the Incredibuild installation is completed, clear the machine state. In CMD run: ```bash theme={null} "C:\Program Files\Amazon\EC2Launch\ec2launch.exe" sysprep ``` 15. After the RDP session disconnects, create the AMI from the instance. ## Create a Linux Helper for the Auto Scaling Group ### Create a Linux Helper A VPC, subnet, and security groups are required to connect the Auto Scaling Group instance to an existing Incredibuild grid (Coordinator/Initiator machines) and to your machine for AMI updates. 1. In the AWS Management Console, navigate to **EC2 > AMI Catalog**. 2. Filter, locate, and select the required AMI. AWS Auto Scaling Groups screenshot

3. In the upper-right corner, click **Launch instance from AMI**. The **Launch an instance** page appears. 4. Complete the following: * **Name:** Helper-LIN * **Instance type:** c5a.2xlarge * **Key pair (login):** If you already have a key pair that you use to connect to your instances, select it. Otherwise, choose **Create new key pair**. A `.pem` file will be downloaded and can be used to connect to the instance. * **Network settings:** Select the appropriate VPC, subnet, and security group(s). * **Configure storage:** At least **30 GB**. 5. Click **Launch instance**. 6. Note the **Instance ID** (for example, i-0b0edfce604652eff) and open the instance details page. 7. Once the instance reaches the **Running** state, note its public IP address and connect to it using SSH. 8. Prepare the Linux Incredibuild installation package. Use the same version that is installed in your existing Incredibuild grid (`ibsetup_linux_.run`). 9. Copy the installation package to the instance using SCP. ```bash theme={null} scp -i ".pem" ibsetup_linux_.run ubuntu@:. ``` 10. Connect to the instance using SSH. ```bash theme={null} ssh -i ".pem" ubuntu@ ``` 11. Make the installer executable and install the Helper. ```bash theme={null} sudo chmod +x ibsetup_linux_.run sudo ./ibsetup_linux_.run \ --action install \ --helper enabled \ --license-type SUVM \ --coordinator-machine \ --data-dir /etc/ ``` 12. After the installation completes, prepare the instance for AMI creation by clearing machine-specific information. ```bash theme={null} sudo systemctl stop cloud-init sudo cloud-init clean --logs --seed sudo truncate -s 0 /etc/machine-id sudo rm -f /var/lib/dbus/machine-id sudo shutdown -h now ``` ### Create an AMI from Linux Helper using AWS Console 1. Sign in to the AWS Management Console. 2. Navigate to **EC2 > Instances.** 3. Select the Helper instance and choose **Instance state > Stop instance**. When prompted for confirmation, choose **Stop**. It can take a few minutes for the instance to stop. 4. To create the AMI, select the Helper instance, select **Actions > Image and templates > Create image**. 5. Enter the **Image name** (**Helper-LIN**). 6. Add a new tag with `Key=Name` and `Value=Helper-LIN`. 7. Click **Create image**. ### Create an AMI from Linux Helper using AWS CLI * To create an AMI from Linux Helper in AWS CLI: ```bash theme={null} aws ec2 stop-instances \ --instance-ids i-0b0edfce604652eff # Wait until instance stops: aws ec2 wait instance-stopped --instance-ids i-0b0edfce604652eff aws ec2 create-image \ --instance-id i-0b0edfce604652eff \ --name "Helper-LIN" \ --tag-specifications "ResourceType=image,Tags=[{Key=Name,Value=Helper-LIN}]" # Note , for example: "ImageId": "ami-0e6d0cc30a40e4ad4" # Wait for AMI creation to finish by running the following command: aws ec2 wait image-available --image-ids ami-0e6d0cc30a40e4ad4 ``` ## Optimize the AWS Auto Scaling Group with SCL or Windows To optimize the AWS Auto Scaling group, run **ibc\_onprem**, a python script, as a service on the Coordinator machine. This builds on the existing Auto Scaling group configuration by scaling Helpers based on actual Coordinator demand, enabling faster and more responsive provisioning than standard Auto Scaling behaviour. 1. Create an AMI from the Incredibuild Helper (SCL or Windows). 2. Copy the AMI ID, for example, `ami-017821fb4a6cdf386`. 3. Create the Auto Scaling group with the following AWS CLI script, updating the `AMI_ID`, `SUBNETS`, and `SECURITY_GROUPS` values based on your existing grid config, as well as any optional values as described below: ```bash theme={null} #!/bin/bash set -e ASG_NAME="Helper_WIN_ASG" AMI_ID="ami-017821fb4a6cdf386" SUBNETS="subnet-0b184d86f43ece9ce" SECURITY_GROUPS="\"sg-06c37393593df9fea\", \"sg-0a65bb9651c5525ba\"" INSTANCE_TYPE_1="c5a.2xlarge" INSTANCE_CPUS_1=8 INSTANCE_TYPE_2="c6a.2xlarge" INSTANCE_CPUS_2=8 # % of spot instances (100% of spot instances - choose ON_DEMAND_PERCENT=0) ON_DEMAND_PERCENT=0 MIN_SIZE=8 MAX_SIZE=40 DESIRED_CAPACITY=8 # NOT TO EDIT LT_NAME="${ASG_NAME}_Template" LT_VER_DESC="${ASG_NAME}_Template_v1" COOLDOWN=30 WARMUP=60 echo "=== Creating Launch Template ===" LAUNCH_TEMPLATE_ID=$( aws ec2 create-launch-template \ --launch-template-name "$LT_NAME" \ --version-description "$LT_VER_DESC" \ --launch-template-data "{ \"NetworkInterfaces\": [ { \"DeviceIndex\": 0, \"SubnetId\": \"$SUBNETS\", \"Groups\": [$SECURITY_GROUPS] } ], \"ImageId\": \"$AMI_ID\", \"InstanceType\": \"$INSTANCE_TYPE_1\" }" \ --query 'LaunchTemplate.LaunchTemplateId' \ --output text ) echo "Launch Template ID: $LAUNCH_TEMPLATE_ID" sleep 5 echo "=== Creating AutoScaling Group ===" aws autoscaling create-auto-scaling-group \ --auto-scaling-group-name "$ASG_NAME" \ --mixed-instances-policy "{ \"LaunchTemplate\": { \"LaunchTemplateSpecification\": { \"LaunchTemplateId\": \"$LAUNCH_TEMPLATE_ID\" }, \"Overrides\": [ {\"InstanceType\": \"$INSTANCE_TYPE_1\", \"WeightedCapacity\": \"$INSTANCE_CPUS_1\"}, {\"InstanceType\": \"$INSTANCE_TYPE_2\", \"WeightedCapacity\": \"$INSTANCE_CPUS_2\"} ] }, \"InstancesDistribution\": { \"OnDemandBaseCapacity\": 0, \"OnDemandPercentageAboveBaseCapacity\": $ON_DEMAND_PERCENT, \"SpotAllocationStrategy\": \"lowest-price\" } }" \ --min-size $MIN_SIZE \ --max-size $MAX_SIZE \ --desired-capacity $DESIRED_CAPACITY \ --default-cooldown $COOLDOWN \ --default-instance-warmup $WARMUP \ --vpc-zone-identifier "$SUBNETS" ``` Optional parameters: * **INSTANCE\_TYPE\_1** and **INSTANCE\_TYPE\_2**: The instance types, and if you specify different instance types, also update the **INSTANCE\_CPUS\_1** and **INSTANCE\_CPUS\_2** values. * **ON\_DEMAND\_PERCENT**: Thepercentage for On-Demand instances. The remaining instances will be created as Spot instances. For 100% Spot instances, leave the parameter unchanged as **0**. * **MIN\_SIZE/MAX\_SIZE:** The minimum and maximum size for the Auto Scaling group.The maximum is a multiple of the minimum and must correspond to the CPUs specified for **INSTANCE\_CPUS\_1** and **INSTANCE\_CPUS\_2**. For example, if there are 16 CPU cores in both machines, the minimum is 16 and the maximum is a multiple of 16 (for example, 96). * **DESIRED\_CAPACITY**: The initial capacity when the Auto Scaling group is created without any load (this usually corresponds to the number of CPUs in a single Helper machine). AWS Auto Scaling Groups screenshot

```bash theme={null} # EC2 Auto-scaling autoscaling:SetDesiredCapacity autoscaling:DescribeAutoScalingGroups ``` AWS Auto Scaling Groups screenshot

```bash theme={null} # Use ASG name you used when creating the ASG (in the example it is Helper_WIN_ASG) # --desired-capacity should be set to the same value you used during ASG creation aws autoscaling set-desired-capacity --auto-scaling-group-name Helper_WIN_ASG --desired-capacity 8 aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names Helper_WIN_ASG --query "AutoScalingGroups[0]" ``` 9. Verify that the results do not include **AccessDenied** errors with **identity-based policy allows** requirements. 10. Copy **ibc\_onprem** ([https://github.com/Incredibuild-RND/ibc\_onprem](https://github.com/Incredibuild-RND/ibc_onprem)) to the Coordinator machine. 11. Update the `config.ini`, editing the `[AWS]` section, providing your AWS Region name (`region_name`) and Auto Scaling group name (`asg_name`), as well as any optional values as described below: ```bash theme={null} [AWS] region_name=us-east-1 asg_name=Helper_WIN_ASG [SCALE] period_sec=10 evaluation_periods=3 scale_in_protection_sec=120 [APP] log_file_path=ibc_onprem.log debug=False ``` Optional parameters: * **period\_sec**: How frequently (in seconds) the program polls the Coordinator for demand for Helper cores. * **evaluation\_periods:** How many evaluation periods(default: **3**) before making a decision to scale-out/scale-in or remain on same number of Helper cores. * **scale\_in\_protection\_sec:** To prevent “thrashing” (rapid scale in/out cycles) of instances, set this to 120 seconds or more. 12. To apply the configuration changes, restart the **ibc\_onprem** utility. Install the pip requirements and run `ibc_onprem.py` as a service, for example, using `nssm.exe`: ```bash theme={null} nssm.exe install ibc_onprem \python.exe \ibc_onprem.py nssm.exe set ibc_onprem Start SERVICE_DEMAND_START nssm.exe start ibc_onprem ``` ### Delete Auto Scaling Group and Launch Template * To delete an Auto Scaling group and Launch Template in the AWS CLI: ```bash theme={null} aws autoscaling delete-auto-scaling-group --auto-scaling-group-name Helper_WIN_ASG --force-delete aws ec2 delete-launch-template --launch-template-name "Helper_WIN_ASG_Template" ``` ### How Required Helper Cores are Calculated Helper core requirements are calculated separately for each Build Group. Only Build Groups with active builds are included in the calculation. For each active Build Group, the service retrieves the **incredibuild\_needed\_helper\_cores** and **incredibuild\_available\_helper\_cores** metrics. The total required Helper cores are calculated as the difference between the sum of the required helper cores and the sum of the available helper cores across all active Build Groups. Build Groups allow resources to be allocated independently, while the Universal Helpers Build Group can assist builds from any Build Group. To prevent over-provisioning, the service accounts for available on-premises Universal Helpers. It calculates the number of available Universal Helper cores as the reported available helper cores minus the Helper cores already provisioned in the AWS Auto Scaling Group. This value is then subtracted from the total required Helper cores to determine the additional cloud capacity that must be provisioned. ### Using ibc\_onprem in Parallel with ASG Dynamic Target Tracking Using the **ibc\_onprem** in parallel with Auto Scaling Group dynamic target tracking is possible when most of the cores are allocated by the Auto Scaling group, and core usage is not “spiky” over the day. * In this case, add an Auto Scaling Group dynamic target tracking: ```bash theme={null} aws autoscaling put-scaling-policy --auto-scaling-group-name "$ASG_NAME" \ --policy-name cpu50-target-tracking-scaling-policy \ --policy-type TargetTrackingScaling \ --target-tracking-configuration '{"TargetValue": 50.0, "PredefinedMetricSpecification": {"PredefinedMetricType": "ASGAverageCPUUtilization"}}' ``` In **many** cases **ibc\_onprem** and dynamic target tracking may compete to set desired capacity. **Example** The Auto Scaling Group is configured with a capacity of **40 Helper cores**. ## Optimize the AWS Auto Scaling Group with Linux To optimize the AWS Auto Scaling group, run **ibc\_onprem**, a python script, as a service on the Coordinator machine. This builds on the existing Auto Scaling group configuration by scaling Helpers based on actual Coordinator demand, enabling faster and more responsive provisioning than standard Auto Scaling behaviour. 1. Create an AMI from the Incredibuild Helper (Linux). 2. Copy the AMI ID, for example, `ami-017821fb4a6cdf386`. 3. Create the Auto Scaling group with the following AWS CLI script, updating the `AMI_ID`, `SUBNETS`, and `SECURITY_GROUPS` values based on your existing grid config, as well as any optional values as described below: ```bash theme={null} #!/bin/bash set -e ASG_NAME="Helper_LIN_ASG" AMI_ID="ami-0b96ef5ea8e2b2252" # LIN AMI SUBNETS="subnet-0b184d86f43ece9ce" SECURITY_GROUPS="\"sg-02d2bdc6681c5b1c7\", \"sg-0a65bb9651c5525ba\"" INSTANCE_TYPE_1="c5a.2xlarge" INSTANCE_CPUS_1=8 INSTANCE_TYPE_2="c6a.2xlarge" INSTANCE_CPUS_2=8 # % of spot instances (100% of spot instances - choose ON_DEMAND_PERCENT=0) ON_DEMAND_PERCENT=0 MIN_SIZE=8 MAX_SIZE=40 DESIRED_CAPACITY=8 # NOT TO EDIT LT_NAME="${ASG_NAME}_Template" LT_VER_DESC="${ASG_NAME}_Template_v1" COOLDOWN=30 WARMUP=60 echo "=== Creating Launch Template ===" LAUNCH_TEMPLATE_ID=$( aws ec2 create-launch-template \ --launch-template-name "$LT_NAME" \ --version-description "$LT_VER_DESC" \ --launch-template-data "{ \"NetworkInterfaces\": [ { \"DeviceIndex\": 0, \"SubnetId\": \"$SUBNETS\", \"Groups\": [$SECURITY_GROUPS] } ], \"ImageId\": \"$AMI_ID\", \"InstanceType\": \"$INSTANCE_TYPE_1\" }" \ --query 'LaunchTemplate.LaunchTemplateId' \ --output text ) echo "Launch Template ID: $LAUNCH_TEMPLATE_ID" sleep 5 echo "=== Creating AutoScaling Group ===" aws autoscaling create-auto-scaling-group \ --auto-scaling-group-name "$ASG_NAME" \ --mixed-instances-policy "{ \"LaunchTemplate\": { \"LaunchTemplateSpecification\": { \"LaunchTemplateId\": \"$LAUNCH_TEMPLATE_ID\" }, \"Overrides\": [ {\"InstanceType\": \"$INSTANCE_TYPE_1\", \"WeightedCapacity\": \"$INSTANCE_CPUS_1\"}, {\"InstanceType\": \"$INSTANCE_TYPE_2\", \"WeightedCapacity\": \"$INSTANCE_CPUS_2\"} ] }, \"InstancesDistribution\": { \"OnDemandBaseCapacity\": 0, \"OnDemandPercentageAboveBaseCapacity\": $ON_DEMAND_PERCENT, \"SpotAllocationStrategy\": \"lowest-price\" } }" \ --min-size $MIN_SIZE \ --max-size $MAX_SIZE \ --desired-capacity $DESIRED_CAPACITY \ --default-cooldown $COOLDOWN \ --default-instance-warmup $WARMUP \ --vpc-zone-identifier "$SUBNETS" ``` Optional parameters: * **INSTANCE\_TYPE\_1** and **INSTANCE\_TYPE\_2**: The instance types, and if you specify different instance types, also update the **INSTANCE\_CPUS\_1** and **INSTANCE\_CPUS\_2** values. * **ON\_DEMAND\_PERCENT**: Thepercentage for On-Demand instances. The remaining instances will be created as Spot instances. For 100% Spot instances, leave the parameter unchanged as **0**. * **MIN\_SIZE/MAX\_SIZE:** The minimum and maximum size for the Auto Scaling group.The maximum is a multiple of the minimum and must correspond to the CPUs specified for **INSTANCE\_CPUS\_1** and **INSTANCE\_CPUS\_2**. For example, if there are 16 CPU cores in both machines, the minimum is 16 and the maximum is a multiple of 16 (for example, 96). * **DESIRED\_CAPACITY**: The initial capacity when the Auto Scaling group is created without any load (this usually corresponds to the number of CPUs in a single Helper machine). ```bash theme={null} sudo /opt/incredibuild/management/set_debug_level.sh coordinator allocation sudo /opt/incredibuild/etc/init.d/incredibuild restart # EC2 Auto-scaling autoscaling:SetDesiredCapacity autoscaling:DescribeAutoScalingGroups ``` AWS Auto Scaling Groups screenshot

```bash theme={null} # Use ASG name you used when creating the ASG (in the example it is Helper_LIN_ASG) # --desired-capacity should be set to the same value you used during ASG creation aws autoscaling set-desired-capacity --auto-scaling-group-name Helper_LIN_ASG --desired-capacity 8 aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names Helper_LIN_ASG --query "AutoScalingGroups[0]" ``` 11. Verify that the results do not include **AccessDenied** errors with **identity-based policy allows** requirements. 12. Copy **ibc\_onprem** ([https://github.com/Incredibuild-RND/ibc\_onprem](https://github.com/Incredibuild-RND/ibc_onprem)) to the Coordinator machine. 13. Update the `config.ini`, editing the `[AWS]` section, providing your AWS Region name (`region_name`) and Auto Scaling group name (`asg_name`), as well as any optional values as described below: ```bash theme={null} [AWS] region_name=us-east-1 asg_name=Helper_LIN_ASG [SCALE] period_sec=10 evaluation_periods=3 scale_in_protection_sec=120 [APP] log_file_path=ibc_onprem.log debug=False ``` Optional parameters: * **period\_sec**: How frequently (in seconds) the program polls the Coordinator for demand for Helper cores. * **evaluation\_periods:** How many evaluation periods(default: **3**) before making a decision to scale-out/scale-in or remain on same number of Helper cores. * **scale\_in\_protection\_sec:** To prevent “thrashing” (rapid scale in/out cycles) of instances, set this to 120 seconds or more. 14. To apply the configuration changes, restart the **ibc\_onprem** utility. Install the pip requirements and run `ibc_onprem.py` as a service, for example, using `nssm.exe`: ```bash theme={null} nssm.exe install ibc_onprem \python.exe \ibc_onprem.py nssm.exe set ibc_onprem Start SERVICE_DEMAND_START nssm.exe start ibc_onprem ``` ### Delete Auto Scaling Group and Launch Template * To delete an Auto Scaling group and Launch Template in the AWS CLI: ```bash theme={null} aws autoscaling delete-auto-scaling-group --auto-scaling-group-name Helper_LIN_ASG --force-delete aws ec2 delete-launch-template --launch-template-name "Helper_LIN_ASG_Template" ``` ### Using ibc\_onprem in Parallel with ASG Dynamic Target Tracking Using the **ibc\_onprem** in parallel with Auto Scaling Group dynamic target tracking is possible when most of the cores are allocated by the Auto Scaling group, and core usage is not “spiky” over the day. * In this case, add an Auto Scaling Group dynamic target tracking: ```bash theme={null} aws autoscaling put-scaling-policy --auto-scaling-group-name "$ASG_NAME" \ --policy-name cpu50-target-tracking-scaling-policy \ --policy-type TargetTrackingScaling \ --target-tracking-configuration '{"TargetValue": 50.0, "PredefinedMetricSpecification": {"PredefinedMetricType": "ASGAverageCPUUtilization"}}' ``` In **many** cases **ibc\_onprem** and dynamic target tracking may compete to set desired capacity. **Example** The Auto Scaling Group is configured with a capacity of **40 Helper cores**.