Creating a Profile XML File

The profile XML file is used to define how Automatic Interception Interface should handle the various processes involved in a distributed job.

An Example

In this simple example, our goal is to distribute a main process run by MainProcess.exe. Our main process runs several sub-processes (DummySubProcess.exe) and ends with a link step (link.exe).

In this scenario, we would probably only want to distribute execution of sub-processes (DummySubProcess.exe). Distributing execution of the single link.exe task to a remote machine would involve sending a relatively large amount of data (.obj files, etc.) across the network, so it would probably be best to let this task always run on the initiating machine.

Syntax

The following profile.xml file can be used to run Automatic Interception Interface:

<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<Profile FormatVersion="1">
    <Tools>
        <Tool Filename="MainProcess" AllowIntercept="true" />
        <Tool Filename="DummySubProcess" AllowRemote="true" />
        <Tool Filename="link" AllowRemote="false" />
    </Tools>
</Profile>

This command specifies that DummySubProcess.exe is to be executed 10 times:

IBConsole  /command="MainProcess.exe DummySubProcess.exe 10"  /profile="profile.xml"

Explanation

• MainProcess.exe is the main process that is running. Because this process spawns sub-processes whose execution we want Incredibuild to distribute (DummySubProcess.exe and link.exe), this process is marked with the AllowIntercept attribute set to True.

• As we want to distribute execution of DummySubProcess.exe to remote machines, DummySubProcess is specified with AllowRemote set to True".

• As we want link.exe to run locally, it is specified with AllowRemote set to False. Omitting this line from the profile.xml file would have the same effect, as the default for both AllowIntercept and AllowRemote is False.

Usage

Once your profile.xml file is ready, it can be used with the following tools to distribute the build process using  Automatic Interception Interface:

• Custom tools that accelerate the build time of a predefined set of standard Make and other build tools and supported compilers, and in Visual Studio, to intercept a tool (in custom steps) that does rendering (see Accelerating Custom Build Steps, Custom Build Events and Custom Build Tools in Visual Studio).

• IBConsole.exe

Samples and Resources

The following is a detailed specification of the Profile XML file format. In addition to the information below, note the following useful resources:

• Under the Incredibuild installation folder, you will find a standard .xsd schema file ("Schema for Profile file.xsd") that you can use to validate an existing Profile XML file. .xsd files can be read by MS Visual Studio and many XML-related editors.

• Several pre-built working samples of projects (including Profile XML files) using Automatic Interception Interface are installed with Incredibuild.

Profile XML File Specification

The Profile XML file structure is as follows:

<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<Profile FormatVersion="1">
    <Tools>
        <Tool ATTRIBUTES />
        ...
    </Tools>
</Profile>

• The Tool element defines the tool or tools that Incredibuild should handle during the distributed process.

• Multiple Tool elements may be included under the Tools element.

A Tool element should be added for the following cases:

Tool Element Attributes

1. Filename: The name of the tool (filename extension is optional). Note that wildcards (e.g. "TOOL*.exe") are allowed to support a single Tool definition that covers multiple tool names.

2. AllowRemote: Execution of Tools with this attribute set to "True" are distributed to remote machines.

   <Tool Filename="cl" AllowRemoteIf="-c,/c" />

3. AllowRemoteIf: Conditional Value to determine if a task should be distributed, or marked for execution on the local Agent, based on its command line. If the Value specified with this attribute is found in the tasks command line, the task will be marked as allowed to be distributed. The following is an example of how to use it: When the above is used in a profile files, only tasks of cl.exe that have the “-c” or “/c” string (without quotes) in the command line will be distributed to remote machines.

4. AllowIntercept: Tools with this attribute set to "True" are monitored by Automatic Interception Interface. Any tool responsible for direct or indirect execution of remotely executed Tools (Tools with AllowRemote set to "True") should have AllowIntercept set to "True".

5. DeriveCaptionFrom: This attribute controls the caption that appears in the task's build bar in the build monitor Progress View. The attribute accepts two pre-defined values:

   • firstparam – will echo the first parameter from the task's command line.

   • lastparam – will echo the last parameter from the task's command line.

6. SingleInstancePerAgent: Tools with this attribute set to "True" never run concurrently on a single Agent – even if the Agent has multi-processor or multi-core hardware. Note that this attribute does not prevent tools from running on different Agents at the same time.

7. AllowRestartOnLocal: For tools with this attribute set to "True", the initiating Agent will be allowed to apply the performance optimization activated by the "Restart remote processes on local machine when possible" Agent Settings option.Important: This attribute should not be set to True for tools that write incrementally to output files, as this might leave a "half-baked" output file if the local instance of the tool is terminated.

8. SuccessExitCodes: This attribute specifies the exit code(s) that will be considered as success exit codes for the specified tool. Value is a comma-separated list of numerical values. Ranges are also allowed (e.g.: '2..19'). If omitted, the default success exit code for the tool will be 0.

9. WarningExitCodes: This attribute specifies the exit code(s) that will be considered as warning exit codes for the specified tool. Tasks returning this exit code are displayed as a yellow bar in the Build Monitor Progress page and are counted as warnings in the Summary page. Value is a comma-separated list of numerical values. Ranges are also allowed (e.g.: '2..19').

10. IdentifyTaskOutput: This attribute should be used whenever double-clicking on a progress bar in the Build Monitor Progress display doesn't work for a certain tool whose execution is distributed using either Submission Interface or Automatic Interception Interface. When specified using the "True" attribute, Incredibuild attempts to add textual identifiers to the tool's output to help locate it in the build's output text. The identifiers are removed from the final build output text. The following is a usage example:

    <Tool Filename="cl" AllowRemote="true" DeriveCaptionFrom="lastparam" IdentifyTaskOutput="true"/>

11. AutoRecover: comma-separated list of strings which, when found in tool output, cause Incredibuild to re-assign the task to another Agent and discard the previous output.

12. TimeLimit: Time limit, in seconds, after which a remote task is automatically canceled and re-assigned to a different Agent.

13. VCCompiler: When set to True (the default is False), this tool and its sub-processes are handled in the same way as the Visual Studio C/C++ compiler in terms of PDB and PCH handling. Use this property only when attempting to distribute execution of the Visual Studio C/C++ compiler through the IB Interfaces.The VCCompiler option is only available with the Visual Studio C/C++ Build System extension package.

14. AutoReserveMemory: This attribute specifies that the tool utilizes memory mapping of files, typically used by compilers for implementing support for precompiled headers. The attribute should specify the file extension(s) of the files to be used with memory mapping (typically the precompiled header file extension). For example, to specify that a tool called "gcc" uses memory mapping for creating precompiled headers with the ".gch" extension, gcc's definition in the profile XML might look like:

    <Tool Filename="gcc" AllowRemote="true" DeriveCaptionFrom="lastparam" AutoReserveMemory="*.gch"/>

    To specify more than one file extension, enter comma-separated values, e.g.: "*.gch,*.pch". <

15. AdditionalOutputMask: This attribute specifies that the tool will return additional files to the initiating machine according to the provided mask. For example, to specify that a tool called "gcc" will return all the .log files it generates to the initiating machine, gcc's definition in the profile XML should look like:

    <Tool Filename="gcc" AllowRemote="true" DeriveCaptionFrom="lastparam" AdditionalOutputMask="*.log"/>

Note:

• It is possible to create a file list that will be returned to the initiating machine by default. This can be done by creating a registry string value named AdditionalOutputMasks under HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Xoreax\Incredibuild\BuilderIn order to specify more than one file extension, enter comma-separated values, e.g.: "*.log,*.dat".

Important Notes

• The main build tool executable should be defined in the profile XML with the AllowIntercept attribute set to "True" (see XML profile files in thesample projects). Note that if you are running IBConsole.exe with the "/COMMAND=" syntax, the tool specified in the /COMMAND argument (or cmd.exe if the tool is a .bat file) is implicitly given the AllowIntercept behavior.

• If the process to be executed remotely is spawned by a child process of the main build tool, the entire hierarchy of processes starting from the main build tool should be specified with the AllowIntercept attribute set to "True". For example, if the main build tool is "make.exe", which executes "runner.exe", and "runner.exe" executes "MyTool.exe" which is the process whose execution we wish to distribute, then both "make.exe" and "runner.exe" must be specified with the AllowIntercept attribute set to "True".

• Executables spawned by a tool specified with AllowRemote set to "True" should not be specified with AllowRemote set to "True". Remote child processes will be handled automatically by Incredibuild.

• Incredibuild can distribute custom steps, execute tasks, and special build tools using the profile xml only with Visual Studio 2010 and later. For further information, take a look at the last section in the BuildConsole section.

Automatic Interception Interface XML Schema

This section describes the XML schema that is used to build the configuration file for Automatic Interception Interface. A detailed explanation of the XML schema can be found in the Using XML Interface section .

The XML file defines the behavior of the integration using Interception Interface, including those processes that can be distributed and those that will run locally. The XML file describes a set of rules that apply to tasks being executed by Incredibuild. A rule applies to all tasks sharing the same name. For example, the XML defines the task "Render" as AllowRemote, A job may include several instances of Render; each will be processed remotely according to the rule. IBConsole uses this XML file as a profile, which defines the way the processes will be handled.

<Tool Filename="SubProcess" AllowRemote="true" DeriveCaptionFrom="lastparam" IdentifyTaskOutput="true"/>

Example 1

To specify that a tool named LogMe returns all the .log and .dat files that it generates to the initiating machine, LogMe should be defined as follows in the profile XML:

<Tool Filename="LogMe" AllowRemote="true" DeriveCaptionFrom="lastparam" AdditionalOutputMask="*.log"/>,*.dat

Example 2

Assuming that LogMe returns all kinds of files, but you are only interested in the .log files that it returns. In this case, LogMe should be defined as follows in the profile XML:

<Tool Filename="LogMe" AllowRemote="true" DeriveCaptionFrom="lastparam" OutputFileMasks="*.log"/>  

Note: Tool is the name of the process to which the rule applies.