Where is my Modelstore - AX2012 and AX2012 R2

This post is just a friendly reminder to myself and everyone who get tricked by an environment that has been upgraded from the initial release of Dynamics AX2012, namely RTM, with or without Feature Pack, to either Release 2 or any of the consecutive releases.  What trick am I referring to? I am thinking about the fact that unless you've removed the old Modelstore, you now have two different Modelstores for each environment.

You will have the upgraded Modelstore in the separate Modelstore Database, but you will also have the old Modelstore inside the Business Database. It does not do anything, except occupy space. Well, there is one more annoying thing it does to me. I find myself going back and forth between RTM to R2 environments and sometimes I forget to use the correct Database name when running PowerShell commands. Imagine installing a new model to a R2 environment, only to later discover that you installed it into the idle RTM Modelstore inside the Business Database and not to the Modelstore Database. It doesn't happen ALL the time, but I've done it more than once.

So in the interest of clarity, here is a drawing explaining where the Modelstores are located. Left hand side is the previous version where everything was in one database. On the right hand side you have the new setup with two databases. And for upgraded environments, you may or may not have that remnant Modelstore sitting inside the Business Database.

The experienced AXers may argue that if you use the Config parameter when operating the Modelstore through PowerShell you will always work against the Modelstore Database, but since I often run these commands from other servers than the AOS Server I mostly use the Server and Database parameter for all operations.  :-)

Setup Life Cycle Services

If you haven't had time to check out Life Cycle Services (LCS) for Dynamics AX yet, then you should know you're missing out on some pretty cool features. Microsoft provides this service for Customers and Partners and I would encourage IT pros to check it out. You should at least install and test this in your own sandbox and test environment.

The LCS has been around for a while now. It was covered by the Technical Conference back in October 2012 and the service has evolved more since then. It was officially released summer 2013 and Microsoft has been improving it since. You can stay updated on latest news and changes by subscribing to the LCS Blog here: http://blogs.msdn.com/b/lcs/

I didn't set aside enough time to check it out right away when it was published, but lately I have been testing it and I wanted to post a short article on my findings. Life Cycle Services is covering a lot of ground, but I will focus on System Diagnostics collection part.

So you begin this by creating a new project on the Life Cycle Services website. Log on with the Microsoft Account (aka. LiveID) associated with your Customer or Partner account. Navigation is simple and intuitive, so I'll assume you figure out how to create your new project. You will have to add additional team members manually, but it is just as easy as creating a new project.

The idea here is to have a project where you can collect diagnostics from one or multiple environments.

When the project is created, you can open it and then Open the System Diagnostics tile. By default, you will be thrown in the Admin section and from there you can download the installer of the LCS Service. Installing the Service is pretty straight forward. Just follow the steps in the guide and you should be good. It takes just a few minutes. I opted for using the business connector as the Service Account. If the installer says the service couldn't start, just head over to Group Policy and make sure the service account actually is allowed to run as a service. If you want this to be a smooth experience, you probably should go through the security setup mentioned in the install guide. If you do implement permissions locally after the service has been installed and started, remember to restart the service. Otherwise it will not pick up any changed permissions.

The next step is to upload environment(s) to the project. You use the LCS Environment Discovery Tool for this. Start by typing in a name for the environment. This will be visible in the LCS website so make sure to use a name that makes sense. Type in SQL Server instance and the name of the business database. Then hit "Discover" and observe if the tool discovers the instance. If it fails, you've most likely forgot to grant the service account read access to the database. If it succeeds, simply press "Upload" to have the environment be available in the LCS website. You can redo this for more environments if needed.

In order to keep the website updated on the environments health, you should add a task that uploads updated information to the website. Just use the Generate command button to create the command line, and add a scheduled task for it. Make sure it runs even though you're not logged in.

I should add that I recently got a message on one of my other projects stating that the LCS had updated the on premise tool. I simply downloaded, stopped the service, ran setup, reused the existing certificate, and it was updated. Lovely!

And that is it. I expect more improved content and features on the LCS website in the future.

Installation of Management Reporter 2012 for AX 2012

This post will take you through the basic steps of installing, configuring and a basic test of Management Reporter 2012 for AX 2012. This tool is now easily available after AX 2012 R2 CU7 as part of the Dynamics AX Setup and the tools has also been translated and made available for multiple languages.

The Reporting tool has a lot of features and Microsoft has release several videos to cover some of the functionality. Technet also describes in detail how to install and setup this tool, and you can also download a comprehensive set of whitepapers and documents covering a lot of ground for this tool.

For the sake of simplicity, I will run you through the setup with some screen shots. I am assuming you're the tech guy which has been given the task by the financial consultant to get this tool installed as soon as possible.
As with all UI, there might be minor changes in the upcoming versions.

The guide will be in three sections:

1. Installing the Server components
2. Installing the Client components
3. Testing!

Before you begin you should have prepared a new domain user which will be used by the service to connect and collect data from AX. I will assume you have full access to the AOS and SQL Server and that you are on AX2012 R2 with CU7 (or higher). Needless to say, but you can obviously do all the three steps above on the same machine - if that is your swag.

Estimated installation time? We should be done within an hour. :-)

1. Installing the Server Components

Run AXSetup and choose to install "Management Reporter"

Type in the details for the AOS you want to the Management Reporter Service to connect to. This is neat if you want to diversify what AOS you want to handle any load.

Select the SQL Server Instance and Database holding the business data. In my example setup threw an error before I was able to select an actual server running a SQL Server Instance. Maybe they'll fix that in some update.  

Enter the credentials for the service account which the Management Reporter Service will run as. I chose to use the same service account as the AOS.

Type in some additional details for the Management Reporter Service. Default port is 4712. The first database entry points to the configuration database. The second to the data mart database, which will hold report data. I just added a suffix so I can test installing multiple instances later on. 

Finally type in the credentials for the domain user which will be used to connect to AX and collect data. Either let the installer set it up or choose an existing AX User ID.

Done!

Before we continue, let us check some things. You should have a new shortcut named "Configuration Console" on the Start menu. Open the Console, locate  "Data Mart Integration" and hit Refresh. Observe that things look promising. It might take a few minutes before data has been collected fully to the Data Mart Database. 

Oh, and if you wonder how that link from AX works, you should also have an entry in LedgerParameters.ManagementReportUrl for the company that has an active integration. Setup should have added that. If not, you will have to do some manual steps from this Configuration Console, but that is not the scope of this post.

2. Installing the Client Components

Run AXSetup and chose to install "Management Reporter Report Designer". This will install both the Designer and the Viewer. You can install these on the terminal server if that fits your requirements. 

Done!

3. Testing!

Let us do some testing now that we have things installed!

The Designer

First, let's test the Designer. Locate the shortcut "Report Designer" under the Start menu. 

Enter the url to the server and port where the Management Reporter is running and connect. This is a one time per user per machine thing (as far as I know).

Select a company and hit the "Set As Default".

Select the first report and hit "Generate" on the toolbar.

Observe while the report is generated. Let it complete and wait for the report to be launched in a new window.

And the report should open. Yea, I hid some of the numbers from my demo.

The Viewer

Now for the Viewer. Locate the shortcut "Report Viewer" under the Start menu.

This tool might also need to have its connection properly set. The setting is under Tools, Connection.

Double click the generated report located in the Report Library and observe it loads an integrated view of the report.

And that is it. Now tell me, did it take you more than an hour to get this up? Next up is to inform the financial consultants that you're done and they can start hammering the tool for awesome reports. 

Final note

Have someone with access to the customer license (VOICE) download the license for the Management Reporter. It is its own license file, and you only have to copy+paste its content into the Registration form under Tools. If you don't do this step, you are limited to less than a handful of users, which is fine for testing, but not the real deal.

Also, don't bother to add users manually. These will be populated from AX dependent on what users have access to what. This is all documented in the Installation Guide for the AX provider.

Data are being held up to date by using SQL Server Change Tracking. If you however notice data not being refreshed and updated properly, you will find steps to resolve this in the already mentioned guide.

I hope this little guide helped you get going with Management Reporter 2012 for Dynamics AX 2012 R2 (CU7 and upwards). There are plenty of guides out for both installing, configuring and using this Reporting tool - I just felt like making one myself as well.

Remember to upgrade the modelstore schema

I was preparing for an RTM to R2 upgrade the other day and while I was just making sure the RTM was prepared and ready for being upgraded I noticed an error I haven't seen before. As soon as I opened the About-dialog, the SQL Server would throw an error.
Here is an extract of the error:

"FASTFIRSTROW" is not a recognized table 
hints option. If it is intended as a parameter to a table-valued function or to the CHANGETABLE 
function, ensure that your database compatibility mode is set to 90.

A quick search on the net and I found out others had seen this error too. Since I was doing this upgrade on a fresh installed SQL Server 2012 I was now bit by a deprecated keyword "FASTFIRSTROW". This hint is now replaced by FAST n (TechNet).

Easy fix was simply just to update the modelstore schema. Apparently, whoever upgraded this RTM didn't read the CU3 instructions properly. One of the necessary steps in this upgrade was to reinitialize the schema. You can basically reinitialize the schema whenever and as often as you like. You will not lose your modelstore data - it is just a schema update. Not every CU contains schema changes, so read the upgrade instruction on partnersource when doing an upgrade.

My preferred way of updating the schema is this simple PowerShell command:

Initialize-AXModelStore -Server MySQLServerNameAndInstanceName -Database MyModelStoreDatabaseName

When I think about it, I would like the upgrade software do this initialization as part of the upgrade. From the top of my head I can't see any reason why not.

Error during installation of Enterprise Portal for AX2012 R2 on SharePoint 2013

I want to just quickly share this with the community. I have not yet narrowed down why this failed, but I have a feeling this might be something I won't encounter too many times when installing Enterprise Portal.

I wanted to setup and install Enterprise Portal for AX2012 R2 (CU6) on SharePoint 2013 and I had already ensured that SharePoint was upgraded with latest upgrades. This SharePoint server was setup by a third party and prepared as just a regular SharePoint Server. I had to prepare the Business Connector as Managed Account. I also created a new Web Application designated for Enterprise Portal. Just to be safe I also created a root Site Collection and tested it ok.

Unfortunately, when I tried to install Enterprise Portal I constantly hit this error:
"Microsoft SharePoint 2010 is not installed or running. Please run the prerequisite utility for more information. Operation is not valid due to the current state of the object."

Now, obviously SharePoint 2010 is not installed, but SharePoint 2013. I believe the last part of the error is the important part. I did decompile the code to see if I could understand why it failed, but didn't find anything obvious.

Instead I found the necessary clue in the setup log. You know everytime you run setup a log is written to disk, normally under "C:\Program Files\Microsoft Dynamics AX\60\Setup Logs\". The log for this run showed me that when it was iterating over the Web Sites, probably for the presentation of them into the drop down list, it failed and threw the error. The last site it was traversing in the log was the default one, "SharePoint - 80".

So I head back to IIS and stopped the Web Site causing Setup to fail and reran setup again. Sure enough, the error was gone and I could successfully select the Enterprise Portal Web Application and continue setup.

Maybe this was just bad luck, or perhaps this will help someone.

You got to love AXBuild Compiler Tool

Initial testing of AXBuild 

I finally got around to play with the new compiler tool for Dynamics AX 2012 R2 and it really is a game changer. My first attempt of compiling a fresh install of AX2012 R2 with CU7 took a little less than one hour. This was tested on a virtual server with enough RAM and just two logical (virtual) 3.6Ghz processors. Not the result I expected, so I went back to the host system and changed Power Options from Balanced to High and added one more logical processor. The tool will take number of processors, multiply by 3 and divide by 2 to give a count of "workers". Max number of workers are 32, in case you try to run this on a system with more than 21 processors. Lol.

Second attempt now took half an hour, which is good, but not quick enough. Apparently I wasn't completely alone on the system, so after kicking out other variables (aka. colleagues) I was down to around 16 minutes for a full compile. Now this is having the databases hosted on another virtual server, so there are some speed lost in chit-chat, but I am thrilled.

Testing backward compatibility

The next thing I wanted to check was whether or not I could actually (ab)use this new compiler methodology to compile older R2 modelstores, like CU6. Surely a bit far-fetched, and perhaps not really supported, but why not give it a go.

I ran the command line utility and threw in a parameter pointing to a CU6 modelstore. The tool started compiling, but it compiled against the CU7 modelstore and not the one I pointed at. So the tool will take the AOS configuration and compile against the modelstore database defined on the AOS configuration. Not really what I aimed for.

I then tested compiling a CU6 modelstore with a CU7 AOS and I got the same result as described here.
I already had a CU6 AOS prepared for testing if I could start this AOS against a CU7 compiled modelstore, but it fails with the following error:

Object Server 01:  Fatal SQL condition during login. Error message: "The internal time zone version number stored in the database is higher than the version supported by the kernel (5/4). Use a newer Microsoft Dynamics AX kernel."

Upgrade kernels!

I mean, the kernel should be backwards compatible, so having CU7 kernel for client and service running against an older application version of R2 is supported, so the lesson learned is to upgrade kernel binaries to CU7 and enjoy a new and improved compiler tool!

If you run axupdate.exe you have the option to only upgrade the services and client components:

Well served by Microsoft on this one!

About your elements Origin, ID and Handle

This post will be about the elements Origin, their ID and finally the Handle. These are important to grasp as AX developer when creating elements in the Application Object Tree (AOT) and copying them around between environments. I hope it will help both new and old AX developers.

The Origin is made up of a Global Unique ID (GUID), and it will follow the element created throughout its lifetime, from the very first brilliant idea, through all sorts of versions and bug fixes and until is is one day obsoleted; from cradle to grave. This Origin will be the same for this element in all installations, environments, versions and variations. It is unique and special, and it so for a purpose.

Take this table as an example:

The Origin of this table will follow this table from my development environment and into any other installation. If I copy this table using XPO, Model or ModelStore, this is the Origin it will carry everywhere. This also means that if I create "MyTable" in my AX, and you create your "MyTable" in your AX, you can be certain they will have different Origin, and essentially not be the same table.

Take a look at how the Origin is part of the element description when I export it to a XPO.

The same is true if I make an axmodel-file having this table.

So why is this so important? Well, most seasoned Dynamics AX developers have at some point gotten their hands burned on element ID conflict, element cache become invalid, change of element ID causing loss of data and so on. AX 2012 attempts to solve this by letting us forget about element ID altogether and instead focus on Origin. The goal is for us developers only to worry about Origin as identification of Elements.

Great, but what is "Element ID"? Element ID is a unique number identifying elements in the AOT. This ID is used a lot in AX for historical and good reasons. It is a way for the system to quickly identify any element for things like indexing, cache, relations, usage, etc. It works like charm, and is a lot more efficient than using the actual name, path or some other non-numeric reference to the element. Surely a GUID is not a very efficient way to index and organize a long list of elements - just think about it.

Element ID comes into play as soon as en element is created in the AOT, but an important difference compared to the Origin is the fact that the Element ID may perfectly well be different between environments. Element ID is strongly tied to the business data, so if the Element ID is changed or made invalid, you risk loss of data. There are however ways around this, but they are not discussed in this post.

Let us have a look at how Origin works when Elements get their ID in AX during import of a XPO or installation of a Model (axmodel). Details can be found in this Technet Article.

The system will automatically try to reuse Element ID based on Origin, or it will create a new Element ID if necessary. There is also this LegacyID, which is a way to make some IDs backward compatible with previous versions of AX, if really necessary.

As an example, take a look at good old CustTable, number 77. :-)

Also take notice of the Origin and compare it to the Origin in your AX environment. Is it the same? It sure is!

Ok, so what is then Element Handle? It is unique ID identifying every single element in the AOT that is meant to be uniquely identified. It is also the ID being used as key and foreign key in the tables containing the information about the AOT elements (ModelElement, ModelElementData, etc). The system assigns Element Handles just as it does RecIds.

Does that mean Origin and Element ID doesn't cover all possible elements in identifying them? The answer is NO! Take table methods as an example. It does not have an Origin, nor does it have en Element ID. It does have en Element Handle.

Try run the SQL underneath in your AX environment and see what handles you have for CustTable and a custom table.

SELECT ELEMENTHANDLE, ROOTHANDLE, NAME, AXID, ORIGIN FROM MODELELEMENT 
JOIN ELEMENTTYPES ON MODELELEMENT.ELEMENTTYPE = ELEMENTTYPES.ELEMENTTYPE
WHERE NAME IN ('CUSTTABLE', 'MYTABLE') AND ELEMENTTYPENAME IN ('TABLE')

You may want to replace "MyTable" with a custom table of your own. Notice how my ElementHandle of CustTable differs from yours.

So let's wrap this up with some highlights:

• Origin is your cradle to grave ID for your new Elements. It is the only ID you really need to worry about.
• Element ID works pretty much like before, except you no longer keep it between environments. System creates this ID uniquely for you.
• Element Handles uniquely identifies every single identifiable element in the AOT and is provided by the system.
• Not all elements have Origin or Element ID, but all elements have a Handle.
• Finally - Business data is tied to Element ID and not Origin. Don't ever get tempted to mix one modelstores business data with another modelstores business data.

If you ever wonder what elements have ElementHandle but no Origin or ElementID (AxID), you could try run these queries on your modelstore database:

-- Get all element types having Origin but no Element ID
SELECT DISTINCT ELEMENTTYPES.ELEMENTTYPENAME
 FROM MODELELEMENT JOIN ELEMENTTYPES ON MODELELEMENT.ELEMENTTYPE = ELEMENTTYPES.ELEMENTTYPE
 WHERE AXID IN (0) AND ORIGIN NOT IN ('00000000-0000-0000-0000-000000000000') 
 
-- Get all element types having no Origin 
SELECT DISTINCT ELEMENTTYPES.ELEMENTTYPENAME
 FROM MODELELEMENT JOIN ELEMENTTYPES ON MODELELEMENT.ELEMENTTYPE = ELEMENTTYPES.ELEMENTTYPE
 WHERE ORIGIN IN ('00000000-0000-0000-0000-000000000000') 

Thank you for reading!