Cross-platform build automation for UE4 projects with Jenkins
Building UE4 projects for Apple devices on Windows is only possible if they are “Blueprint only”. As soon as you are using a single C++ source file, or Blueprint nativization, the project has to be built on Mac OS. While there are cross compilers available for Windows, using Apple SDKs on other systems violates Xcode EULA: “You are expressly prohibited from separately using the Apple SDKs or attempting to run any part of the Apple Software on non-Apple-branded hardware.“
So you need at least a Mac Mini (or similar) for Apple builds of non-trivial UE4 projects. If you are using Jenkins build automation, this causes quite some problems, because Jenkinsfiles usually contain shell calls which are not cross-platform compatible.
Scope of this post
This post is about UE4 cross-platform build automation with Jenkins on Windows and Mac OS. It also applies to game console builds (on Windows) and iOS builds (on Mac OS). If you are looking for Jenkins build automation instructions on Windows only, you may prefer to read this multiplatform UE4 Jenkins guide on Windows or this UE4 legacy Jenkins guide.
For non-automated remote builds, read this documentation topic about the remote compiler.
Build Tools included in UE4
A cross-platform Jenkinsfile requires cross-platform build scripts. The command line build scripts supplied by UE4 are located in
| Tool Name | Also Known As | Used For | Remarks |
|
Build.bat (Win) Build.sh (Mac/Linux) |
UBT, Unreal Build Tool | Generating project files, compiling. | It is also possible to directly call UnrealBuildTool.exe, but this does not work cross-platform. |
|
RunUAT.bat (Win) RunUAT.command (Mac) RunUAT.sh (Linux) |
UAT, Unreal Automation Tool | Building & packaging projects. | This script builds AutomationTool.exe if it does not yet exist. |
| UE4Editor-Cmd | Unreal Editor Command Line | Running Commandlets. | Commandlets are used to implement custom checks. |
| UnrealFrontend | Creating profiles for building & packaging. | Command line usage is broken, so it cannot be used by Jenkins. |
The simplest tool to set up cross-platform builds (game and server) is UnrealFrontend, because it can be used with platform independent profile files. However, running UnrealFrontend with parameters -run=launchprofile -profilename=”profile” as documented in the source code leads to a non-trivial crash. The corresponding bug report is not being considered by Epic Games Support.
This means that unfortunately, UnrealFrontend cannot be used with Jenkins.
Jenkins & Powershell Core
As analysed in the previous section, the Jenkinsfile needs to call the corresponding build scripts depending on the platform. This can either be done in a custom Jenkins plugin (to be created), or by adding a shell call. However, by default there is no portable shell available in Jenkins * there is “bat” (cmd.exe) on Windows and “sh” on Mac/Linux.
A solution for this issue is to install Powershell Core on all Jenkins nodes and use “pwsh” in the Jenkinsfile to call the build scripts. You may need to work around the problem that the Powershell Core setup has no signature on Mac OS, but other than that it works like a charm.
In some cases you need to combine escaping characters for Groovy and Powershell, which leads to quotes looking like this:
`\"
Custom Jenkinsfile steps
The usual build steps for UE4 projects are as follows:
-
UBT: Generate project files (Visual Studio / Xcode / whatever)
-
UBT: Build editor DLLs of the game (they are required for some of the later steps)
-
UAT: Build & package the game
In addition to that, the example Jenkinsfile runs a Commandlet to Compile all Blueprints:
startProcess(getUnrealEditorCmdPath(), "`\"${env.WORKSPACE}/${env.BUILD_PROJECT_NAME}.uproject`\" -run=CompileAllBlueprints -IgnoreFolder=/Engine -Unattended")
This is a very fast way to pre-check all Blueprints of the game, even if they are currently unused and would not be considered by the general build. Other checks like Linter can also be run like this (-run=Linter).
Cross-platform Jenkinsfile
A sample cross-platform Jenkinsfile (based on git with git-lfs) is shown below.
Setup checklist:
-
Deploy at least one Jenkins node for each platform, use labels “windows” and “mac” & install Powershell Core.
-
Adjust BUILD_PROJECT_NAME (the name of the UE4 project, default is the job name).
-
Adjust getBuildArchivePath and getUnrealEngineRoot using the corresponding locations on your Jenkins nodes.
-
Adjust the scm “checkout” command according to your needs (this may also require credentials to be set in Jenkins).
The sample Jenkinsfile is also available on github.
pipeline
{
agent none
parameters
{
choice(
name: 'PLATFORM_FILTER',
choices: ['windows', 'mac', 'all'],
description: '')
choice(
name: 'ENGINE_VERSION',
choices: [ 'UE_4_24_3' ],
description: '')
choice(
name: 'BUILD_CONFIGURATION',
choices: [ 'Development', 'Shipping' ],
description: '')
booleanParam(
name: 'COMPILE_GAME',
defaultValue: true,
description: '')
}
environment
{
BUILD_PROJECT_NAME = "$env.JOB_BASE_NAME"
}
options
{
// Default checkout may not have git-lfs and may have bad timeouts.
skipDefaultCheckout(true)
}
stages
{
stage('BuildAndDeploy')
{
matrix {
agent {
label "${PLATFORM}"
}
when { anyOf {
expression { params.PLATFORM_FILTER == 'all' }
expression { params.PLATFORM_FILTER == env.PLATFORM }
} }
axes {
axis {
name 'PLATFORM'
values 'windows', 'mac'
}
}
stages
{
stage('Preparation')
{
environment
{
BUILD_ARCHIVE_DIR = getBuildArchivePath()
}
steps
{
// Checkout code changes.
checkout poll: false, scm: [$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'CleanBeforeCheckout'], [$class: 'GitLFSPull'], [$class: 'CheckoutOption', timeout: 60], [$class: 'CloneOption', noTags: false, reference: '', shallow: false, timeout: 60]], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'scm', url: 'https://bitbucket.org/replace_with_your/project.git']]]
// Create project files (e.g. Visual Studio Solution).
pwsh script: startProcess(getUnrealBuildToolPath(), "-projectfiles -project=`\"${env.WORKSPACE}/${env.BUILD_PROJECT_NAME}.uproject`\" -game -engine")
// Build editor dlls for project.
pwsh script: startProcess(getUnrealBuildToolPath(), "${env.BUILD_PROJECT_NAME}Editor " + getUnrealPlatformName() + " Development `\"${env.WORKSPACE}/${env.BUILD_PROJECT_NAME}.uproject`\" -engine")
// Create folders for later steps.
pwsh script: 'if (!(Test-Path -Path "$env:BUILD_ARCHIVE_DIR/$env:BUILD_PROJECT_NAME")) { md "$env:BUILD_ARCHIVE_DIR/$env:BUILD_PROJECT_NAME" -Force }'
}
}
stage('Compile Blueprints')
{
steps
{
pwsh script: startProcess(getUnrealEditorCmdPath(), "`\"${env.WORKSPACE}/${env.BUILD_PROJECT_NAME}.uproject`\" -run=CompileAllBlueprints -IgnoreFolder=/Engine -Unattended")
}
}
stage('Build Game')
{
when
{
expression
{
params.COMPILE_GAME
}
}
environment
{
BUILD_ARCHIVE_DIR = getBuildArchivePath()
}
steps
{
pwsh script: 'Remove-Item "$env:BUILD_ARCHIVE_DIR/$env:BUILD_PROJECT_NAME/WindowsNoEditor" -Force -Recurse -ErrorAction SilentlyContinue'
pwsh script: startProcess(getUnrealAutomationToolPath(), getBuildArgsClient() + " -targetplatform=" + getUnrealPlatformName())
}
}
}
}
}
}
}
def getBuildArchivePath()
{
if (env.PLATFORM == 'mac') { return '/Volumes/X5/Packages' }
else if (env.PLATFORM == 'windows') { return 'C:/Projects/Archive' }
else { throw new Exception('Unsupported platform') }
}
def getUnrealEngineRoot()
{
if (env.PLATFORM == 'mac') { return "/Volumes/X5/Shared/$env.ENGINE_VERSION" }
else if (env.PLATFORM == 'windows') { return "C:/Projects/$env.ENGINE_VERSION" }
else { throw new Exception('Unsupported platform') }
}
def getUnrealBuildToolPath()
{
if (env.PLATFORM == 'mac') { return getUnrealEngineRoot() + '/Engine/Build/BatchFiles/Mac/Build.sh' }
else if (env.PLATFORM == 'windows') { return getUnrealEngineRoot() + '/Engine/Build/BatchFiles/Build.bat' }
else { throw new Exception('Unsupported platform') }
}
def getUnrealAutomationToolPath()
{
if (env.PLATFORM == 'mac') { return getUnrealEngineRoot() + '/Engine/Build/BatchFiles/RunUAT.command' }
else if (env.PLATFORM == 'windows') { return getUnrealEngineRoot() + '/Engine/Build/BatchFiles/RunUAT.bat' }
else { throw new Exception('Unsupported platform') }
}
def getUnrealEditorCmdPath()
{
if (env.PLATFORM == 'mac') { return getUnrealEngineRoot() + '/Engine/Binaries/Mac/UE4Editor-Cmd' }
else if (env.PLATFORM == 'windows') { return getUnrealEngineRoot() + '/Engine/Binaries/Win64/UE4Editor-Cmd.exe' }
else { throw new Exception('Unsupported platform') }
}
def getUnrealPlatformName()
{
if (env.PLATFORM == 'mac') { return 'Mac' }
else if (env.PLATFORM == 'windows') { return 'Win64' }
else { throw new Exception('Unsupported platform') }
}
def getBuildArgsClient()
{
return "BuildCookRun -project=`\"${env.WORKSPACE}/${env.BUILD_PROJECT_NAME}.uproject`\" -nocompileeditor -nop4 -cook -stage -archive -archivedirectory=`\"$env.BUILD_ARCHIVE_DIR/$env.BUILD_PROJECT_NAME`\" -package -clean -compressed -pak -prereqs -distribution -nodebuginfo -build -target=$env.BUILD_PROJECT_NAME -clientconfig=$env.BUILD_CONFIGURATION -utf8output -compile"
}
def startProcess(String program, String parameters)
{
String cmd = '$batch = Start-Process -FilePath "' + program + '" -ArgumentList "' + parameters + '" -Wait -PassThru -NoNewWindow -ErrorAction Stop; exit $batch.ExitCode'
echo cmd
return cmd
}
Known Issues
- If no build node is available for one of the platforms, the build will stall, even if a platform filter is specified as parameter. This is a known problem in Jenkins, because the platform filter is evaluated on the specific node (which is not available). Currently, the only known workaround is to abort the build.
Epic, Epic Games, Unreal and Unreal Engine are trademarks or registered trademarks of Epic Games, Inc. in the United States of America and elsewhere.