Saturday, February 16, 2013

Upgrading AX2012 to AX2012 R2 Part 2

Two months ago I wrote a few words around the in-place upgrade from AX2012 RTM (v6.0/v6.1) to AX2012 R2 (v6.2). While working with this upgrade, I found that the guide did not tell the whole story. In fact there were several steps that would seem impossible to complete. After a lot of testing, communication with Microsoft Partner Support and with participants at the the Community Forum, I feel I finally can say the upgrade works. I hope it helps to hear this from someone who merely followed the guide, struggled some but finally got through.

The AX Support Blog asks us to be sure to follow the guide that is dated 12th of February or later.

MSDN Version
Technet Version

While the guide is covering a lot of the steps involved in a successful upgrade, I still would like to share some of my own experience. There are still things with the guide that needs to be updated/changed. Anyone who wants to attempt this upgrade should be prepared to see several errors and issues they can safely ignore, but then again there will be some things which will be symptoms of a bigger problem.

Check your setup logs while upgrading and keep an eye on the Event Log. Setup logs are always located in the same folder, normally under \Program Files\Microsoft Dynamics AX\60\Setup Logs\. If you get in trouble, these logs will be one of the first places to look.

Upgrading the code

Before you start your code upgrade, you need to make sure the RTM version both compiles X++ code and CIL compile. This will ensure that you know later on if compilation errors during the code upgrade is due to own customization and not something you brought with you from your source environment. Obviously, any change to the code in the source after you have duplicated PROD to TEST and DEV will have to be reproduced after the upgrade is done. I would simply recommend freezing the RTM code during the upgrade.

You also need to understand that as soon as you run the R2 Setup against your database, and you install the R2 Microsoft models, you have introduced a lot of new elements which will have environment unique IDs. This means you will have a new modelstore that is your single source for the code upgrade process. During installation against the RTM database, your old modelstore will remain in the database, and the new R2 modelstore will be installed in a new database. The guide explains all of this, but just understand that during this split, your new R2 modelstore is unique for this customer environment. To put it another way, if you upgrade the database twice, the first modelstore will be incompatible with the business data in the second one. The guide also seem to convey that there will be multiple Microsoft layers, but in fact a lot of the old code in FPK and SYP will be moved to SYS. There are exceptions though, like the beta version of Data Migration Framework.

If you don't get the installation wizard step that allows you to chose between export or import of the R2 modelstore, it is most likely due to the installer not finding either a reference to a RTM AOS or Client on the system. I found that the reference to a RTM Client is the same as for a R2 Client, from the setup perspective. In other words, this step will pop up if there is either a RTM AOS or Client on the system, or you if you have a R2 Client installed. Just so you know.

The in-place upgrade guide starts off with duplicating PROD to TEST, and later to duplicate TEST to DEV. I had TEST and DEV on the same server, which worked fine for me. The guide further tells us to run a kernel compile after the R2 TEST AOS is installed against the R2 TEST database. Some have asked "What does kernel compile do?" I got this answer from Microsoft Support:

"Unlike what may be interpreted from some documentation pages (“Perform a full kernel compilation by using the following command: Ax32.exe -startupcmd=kernelcompileall”), this command does not compile the kernel; it does a full AOT compilation, i.e. it compiles the application. However, instead of using X++ classes to control that compilation and log the results, this operation is done by a kernel function. During minor version upgrade, majority of the class required to be recompiled, including many system class/compiler related class. The kernel compile is required for minor version upgrade as it force a full compile on every class in AOT, including those system/compiler class, and are executed on server tier. Without running the kernel compiler ax won’t even function properly as the base application class are not even compiled to begin with."

So basically after running setup against the database and having the R2 models installed, our modelstore is in such a bad shape, it is crippled and will not even be able to compile using its own system classes. Using kernel compile, we can get those system classes compiled and ready for the next steps in DEV. The whole process takes extra time, which I assume is mostly due to all the errors and issues the compiler will find. You are also asked to restart the AOS before and after doing the kernel compile. Please do.

When the modelstore is compiled by the kernel, you will export it as a base for your DEV environment. This is the fun part, going through each layer and make sure they get R2 compatible. The guide explains how to do this in a good way, just make sure you follow it. Since this is DEV, you are not too worried about loss of business data. You also want to set all batchjobs to STATUS = 0 in the database. This will hinder them from creating disturbing errors here and there. There will be a lot of errors and warnings from the database, but all you care about is getting the code and metadata free of compile errors. Even though the code upgrade check list has several steps, you will most likely not complete all of them. Some steps will not be marked as finished. Just focus on getting the AOT to compile and finally CIL to compile, all while making sure your element IDs remain the same throughout the process. Any AX2012 developer should by now understand how to make sure element IDs remain the same in a modelstore. 

One final important step when the modelstore is done, is to set the "installmode" on the modelstore. This is crucial for success. Make sure you set the "installmode" on the correct modelstore.

Upgrading the data
When you are done upgrading and preparing the R2 modelstore, it is time to test the data upgrade. You will start off by upgrading TEST and if everyone in your team is happy with the upgraded data, you can plan to upgrade PROD. You will use the same modelstore for both. Again, it is crucial that the modelstore is in "installmode" or you can forget about getting a successful data upgrade in production. The installation of the AOS will fail, and when you try open the client you will see several hundreds of errors in the event log. 

One thing the guide barely mentions is the new R2 database will have a different type of table per type inheritance mode. During the data upgrade the database will be in a Table-Per-Type mode. This makes the data upgrade scripts work properly. If you want to learn more about what this is, please have a look at this blog post for starters. You can check if your business database is in the right mode by looking for the stored procedure named TABLEPERTYPEINHERITANCEMODE. During data upgrade, you will be prompted to run a huge SQL-script, and this script will at the end set the inheritance mode to it's correct state after the data has been successfully upgraded. Eventually the R2 business database will have a Table-Per-Hiearchy mode. If you wonder how this stored procedure got into the database in the first place, I can reveal it happens during installation of the R2 AOS. Setup will figure out if the underlying database is on an older version (6.0/6.1) and create it in the database. The in-place guide seems to claim that you need to explicitly be in an upgrade mode when you are installing the AOS, but my testing shows that setup will discover the version of the database schema and add the procedure if the underlying database is pre-R2. 
In other words, there is a phase where you have a business database in the "wrong" table per type inheritance mode. This phase starts when the AOS installs the procedure and ends somewhere late in the data upgrade check list. 

During data upgrade, a series of jobs will be run by the batch framework. To save you some trouble, you should update the tables BATCH and BATCHSERVERGROUP with correct AOSID values. You can however get around this before running the jobs by modifying their settings in the upgrade cockpit. I prefer running a SQL skript that fix batch before I start.

The guide also mentions that you should change Recovery mode on the model database to "Simple" while upgrading. This will prevent SQL to log every single transaction to the model database. I would argue having the recovery model to "Full" on this database is only necessary for a development environment, depending on your overall backup and recovery strategy. 

Summary
Sorry for the wall of text. Here is a "brief" summary:
  • Make sure your RTM code compiles, CIL compiles and database is synced. 
  • Freeze the RTM code when you start the R2 upgrade.
  • When installing the R2 models the first time, this can not be redone later on without having to do another full code upgrade and data upgrade. 
  • Follow the guide when upgrade the models to ensure that the element IDs remain unchanged within the modelstore, or be prepared for loss of business data. Those who do not care for the business data  can upgrade layers without having to reimport the "broken" modelstore between each layer upgrade.
  • The compile and synchronize steps while doing code upgrade will not be marked as complete. Ignore this and just focus on getting the code to compile.
  • When you have prepared the modelstore, make sure to set the installmode and export a backup of the modelstore. This modelstore will be the one you import back to TEST and later on to PROD.
  • Always stay aware of what modelstore you are working against. Don't be fooled by the modelstore that is also located in your business database.
  • Make sure your batch settings are prepared before you initiate the jobs that will upgrade your business data.
  • Follow the data upgrade checklist so you don't lose business data.
  • If you get stuck, ask for help!
I would estimate less than a week for the whole process on a AX2012 RTM without any customization, but you could probably manage to pull it off in just a couple of days with some practice. However, with enough code customization and dependencies to external services, the process will take longer.

I would like to congratulate Microsoft on a well made in-place upgrade. It must have been a lot of hard work to get it where it is today. Hopefully we will see most RTM versions of AX2012 be upgraded to R2 during this year.