Automating Build Pipeline Creation using Azure DevOps Services REST API

//Automating Build Pipeline Creation using Azure DevOps Services REST API

Azure DevOps is a powerful tool for any organization, especially one dedicated to a robust CI/CD pipleline.  Getting a working pipeline for a repository can take only a few minutes, especially if using a tried and true template.  But what if instead of 5 or 10 pipelines, you want to set up a thousand?  I came across this use case in a recent engagement.  One pipeline may take 5 minutes, but to do the same thing a thousand times would be tedious to say the least.  So, to the rescue, PowerShell and Azure DevOps Services REST API.

The Problem

There are a set number of pipeline variations that could be used by numerous newly setup projects in Azure DevOps.  Manually setting them up would take too much time and they vary little between project.  These factors make this ideal for creating a PowerShell script.

Solution

Pipelines were set up as desired in an initial test project using the GUI.  This will generate the base template we can apply to all other projects.  Once that is done it’s the API’s turn.  The first step is to retrieve the build definition we just created using the API using a GET request.

This will get you the full build definition.  Doing this call in PowerShell will give you an object we can manipulate as needed.  The call will look something like this:

$Url = "$BaseURL$Project/_apis/build/definitions/" + $Id + "?api-version=2.0"
Invoke-RestMethod -Uri $Url -Headers $headers -Method Get -ContentType application/json;

Why do we want to manipulate it?  Because this is going to return too much information for our generic template.  Specifically in this case removing the following block:

id : 2
name : CleverName
url : https://Clevername.visualstudio.com/Cleverproject/_apis/build/Definitions/2?revision=3
uri : vstfs:///Build/Definition/2
revision : 3
createdDate : 2018-08-16T18:17:35.947Z
project @{Interesting nonsense}

Thankfully, we can easily remove these in PowerShell.  A simple foreach loop can remove the properties we don’t need.

foreach($value in $values) {
$response.PSObject.Properties.Remove($value)
}

Now we have a stripped down object which is in the correct format for creating new build definitions.  Depending on the project you are pushing to you’ll want to update Json to reflect the new name.

$response = ($json | ConvertTo-Json -Depth 3).Replace($OriginalProject, $NewProject)

Now we can send in our request to Azure DevOps using the below request and PowerShell script (as defined by Microsoft):

$Url = "$BaseURL$Project/_apis/build/definitions?api-version=2.0"
Invoke-RestMethod -Uri $Url -Headers $headers -Body $body -Method Post -ContentType application/json;

Now we have our initial pipeline in a new project.  We can repeat this as many time as needed.

Conclusion

Even if you don’t encounter this specific problem, I believe this illustrates the power of the Azure DevOps Services REST API.  With a few lines of PowerShell we can automate a task that may have taken weeks to do manually.  And this is just a single example.  In the original project unique service endpoints were created for each project, the builds were modified to run against their own unique endpoint, and builds were automatically triggered afterwards for verification.  I’d encourage you to read through the Microsoft reference to see the various tasks that can be accomplished using the API.

Many tasks can be automated utilizing the API, and as your organization grows the value of that automation will grow as well.  It can add complexity, but Nebbia consultants can assist you with this process.

By |2018-12-03T17:01:50+00:00November 29th, 2018|

About the Author:

Ryan is a DevOps and Test Automation Consultant. He started in the software industry as a QA and production support engineer where he developed a passion for efficiency through automation. Over the last few years he has specialized in converting the manual testing and release tasks of organizations into a robust CI/CD pipeline. When he is not at work, he enjoys taking his children to Disney World and having them occasionally throw their toys at him.