Connecting Azure DevOps with Lifecycle Services for Release pipelines

A week ago, Microsoft annouced the release of a new Azure DevOps Task. Followed by more details the setup from the announcements author, Joris de Gruyter. And with the help of this post by Business Application MVP, Munib Ahmed, I sat down and ran through the setup a couple of days ago.

I wanted to briefly add some additional thoughts as well, some considerations of my own, while we wait for the official documentation.

This new feature is very quick and easy to setup, and is something everyone should adopt sooner than later. It shaves off the time spent downloading the complete build artifact somewhere and then uploading it to the Dynamics Lifecycle Services Project Asset Library. After a successful build of the full application you will get the package automatically "released" and uploaded to the asset library.

We expect more "tasks" to be added, allowing us to setup a release pipeline that also let us automatically install a successful build on a designated target environment. So getting this setup now, and have the connection working, will set the stage for adding the next features as they are announced.

Here are some of the requirements and considerations while you set this up:

• You need to register an Azure Application of type Native (public, multi-tenant). While it is said you can use the preview experience to register the app in the Azure Portal, I had to use the "non-preview" experience, to ensure I got a correct Native Azure app registration, and not a Web app. While you can add the necessary permissions setup (user_impersonation), you need to run the admin consent for the permissions to work. If you are setting it up, and you are not Global Admin or Application Admin, then you will need to get someone else with necessary permissions to run the admin consent part. 
• The connection also requires user credentials as part of the setup. This should not be a just any user, if you think about it. You don't want the connection to break just because the password was changed, or the user was disabled. Also multi-factor (or two-factor) authentication will not work here. So you might want to create yourself a dedicated user for this connection. The user does not need any licenses, just a normal cloud user you have setup and logged on with at least once. Also the user needs to be added to the Lifecycle Services project with at least Team member permissions (access to upload to Asset Library). Log on to LCS with the user once and verify access.
• When you create the release pipeline for the first time, you will need to install the Azure DevOps task extension. Search for "Dynamics" and you should find the "Dynamics 365 Unified Operations Tools". If you are doing the setup with a user that has access to many different Azure DevOps organizations (ie. you're the partner developer doing this cross multiple customers), you will need to make sure you install it on the correct Azure DevOps Organization. When it is installed, you will have to refresh the task selection to see the new available task(s), including the new task named "Dynamics Lifecycle Services (LCS) Asset Upload". 
• When configuring the task, you will want to use variables to replace parts of the strings, like the name of the asset, the description, and so on. When you run a release, one of the steps actually list out all the available variables, though with a slightly different syntax. Have a look at the list of available variables on this article, plus the top on how to see the values they are populated with upon a run.

If you already have a successful build ready, go ahead and setup the release pipeline and run it once to see it succeed - or fail. If it fails while trying to connect, it could be one of the following errors:

• AADSTS65001: The user or administrator has not consented to use the application with ID '***' named 'NAME_OF_APP'. Send an interactive authorization request for this user and resource.
  Here you have not successfully run the admin consent. Someone with proper permissions needs to run the "Grant permissions" in the Azure Portal.
• AADSTS50076: Due to a configuration change made by your administrator, or because you moved to a new location, you must use multi-factor authentication to access.
  This is most likely because the user credentials used on the connection is secured with multi-factor authentication. Either use a different account without MFA, or disable it. Most likely it is on for the account for a reason. 

I would strongly encourage everyone to set this up, and feel free to use the community forums, Twitter or ask on the comments section, if you run into issues.

Considerations when "upgrading" Dyn365FO from 8.0 to 8.1

Version 8.0 of Microsoft Dynamics 365 for Finance and Operations was released summer 2018. Just a few months later, in October, version 8.1 was released. If you have environments running version 8.0, let it be development environments, demo environments, or even production and (tier 2+) sandboxes, you might be thinking about getting them "upgraded" to 8.1.
It's not really an upgrade, but actually rather an update.

The overall process is actually a lot easier compared with coming from 7.x. I did a series of posts on how to get started here:
https://yetanotherdynamicsaxblog.blogspot.com/2018/11/upgrade-from-7x-to-8-series-post-1.html

Microsoft outlines the process in one single article here:
https://docs.microsoft.com/en-us/dynamics365/unified-operations/dev-itpro/migration-upgrade/appupdate-80-81

Why update?

One of the main differences between 8.0 and 8.1 is the latter version will be a lot easier to service with updates. Version 8.0 still supports individual application hotfixes, meaning you will download and apply them, put them in VSTS, just as you would with 7.x. You could argue the possibility to pick individual hotfixes and avoid taking all updates is a good thing, but in fact it is not the way forward. Instead of thinking that you may have to avoid hotfixes, and potentially have to "roll back" updates that breaks you, you need to shift to a mindset where any ongoing issues are immediately reported back to Microsoft which allows them to ship new updates that resolves any issues, not only for you, but for us all. With that mindset, you will want to take the 8.1 version, which does not allow for individual hotfixes, but instead gives you everything cumulative at the point you pick updates. This is also how "One Version" will behave, and on April 2019 you will be getting updates in this fashion.

So in effect, when servicing 8.1+ you get only one update tile, and it contains everything, and you download everything cumulative. You'll use the complete update package to patch your environments, and there is no need to put the updates in VSTS either. Things are just so much easier.

Development and build environments

Even though Microsoft has a Software Deployable Package that does the update from 8.0 to 8.1 in the Shared Asset library in LCS, it is recommended that you deploy new 8.1 build and development environments. Why is that, you may ask. For a development environment, you will have both source code and a runtime (code compiled). Your 8.0 development environment might even have been updated with hotfixes, added back in time. Part of the process is to remove any 8.0 updates, and start from scratch with 8.1. So when you start removing already committed Microsoft application updates form Azure DevOps (VSTS), you cannot avoid this to also reflect your local copy of the source code.

But you do not need to compile Microsofts packages, so who cares if the code is wrong? Well, what if you want to debug, extend, view code? Even though you do not need to recompile Microsofts packages, you run the risk of having invalid, incomplete or even erroneous code on your development environment. So it just follows your best option is to redeploy a new set of development boxes and of course build box(es), and depending on your choice of server size and storage, the deploy of new servers they might be ready for you within 3-4 hours.

But before you connect the newly deployed development environments to the source code, it is paramount that you prepare a new 8.1 branch, which is clean from updates. It may contain 8.0 extension modules, but not any Microsoft modules. You can prepare all of this while the new environments are being deployed.

Non-development environments

What about demo, test and sandboxes? Well, typically you do not care about the source code on the demo boxes (even though it might be there), and as for acceptance test sandboxes, where you do not even have Visual Studio, it definitely doesn't matter. These environments you could just go ahead and update using the Software Deployable Package.

Well, unfortunately it might not just be that simple. If the environment has other non-Microsoft packages installed, LCS will prevent you from simply apply the update package. You may have some ISV-solutions or some package you've created and released, and then installed on the environment, through LCS.
LCS knows about this, and can list the non-Microsoft packages installed. In fact, if you try apply the update package, LCS will stop you, and list the packages blocking you.

Error: "Modules on the environment do not match with modules in the package. Missing modules: [...]"

In order to continue, you will need to get a pre-compiled version of these modules where they were built on a 8.1 environment. Depending on your scenario, that either means getting the 8.1 version from a vendor or partner, or simply just get your package built and released through your new and shiny 8.1 boxes.

As it is stated in the upgrade guide, you are recommended to prepare yourself one single build release of all the extension modules and packages. When you have the 8.1 package ready in the Asset Library, you can simply merge it with the update package, and execute the update.

If all your demo and test environments where using the same set of non-Microsoft packages and modules, you'll simply reuse the same merged package to update all of them.

Happy updating!

Initial steps to troubleshoot failed environment servicing

On the topic of patching and updating an existing D365 Operations environment I will refer to the online documentation.

• Upgrade Finance and Operations to the latest platform update
• Process for moving to the latest update of Finance and Operations
• Apply a deployable package to a Finance and Operations environment

There are also some great community posts that aims to help you, and you may want to check out.

• Upgrades, Updates and Hotfixes in AX7
• Installing deployable packages with Powershell

I expect more posts to show up. As it is of writing this, installing updates can be a bit tedious and cumbersome. 

I will use this post to share a recent update that failed on a Platform Update. A Platform Update is expected to be a fairly straight forward and safe operation. You simply import the update to your assets in LCS and apply it to your environment (assuming you're running your environment in the cloud). I will not discuss On-Premise in this post. 

I had an environment running application 1611 with Platform Update 7. I was trying to install Platform Update 10. After it failed on several attempts, I started to try investigate why it failed. 

Here are the steps I took.

1) Identify which step failed. In my case it was step 13. (Not exactly my lucky number)

2) Find the runbook output (normally under C:\RunbookOutput) and find the PowerShell Script that fails. I simply searched the log for "13"

3) Open PowerShell ISE in Admin mode and open the PowerShell Script. You will find the script in the J:\DeployablePackages folder, and you can match the GUID from the log with the Runbook folder. The Scripts will be located in a standardized folder path.

4) Run the script and let it fail. From there you can add breakpoints and try run again and step through to try see why it failed. Use whatever you find as information when you contact Microsoft Support. Some updates fails, but should not fail, and it is important that anyone with support agreements make sure to report findings back to Microsoft. 

Now, in my particular case, the script did not fail when I ran it manually. It succeeded. I can only guess to why that is the case, but after going back to LCS and letting the update "Resume" it eventually finished with all the upgrade steps successfully. 

In any case, the initial steps above can help you push through a failing update and potentially lead you to the answer why an update unexpectedly failed.