I want to just quickly share this with the community. I have not yet narrowed down why this failed, but I have a feeling this might be something I won't encounter too many times when installing Enterprise Portal.
I wanted to setup and install Enterprise Portal for AX2012 R2 (CU6) on SharePoint 2013 and I had already ensured that SharePoint was upgraded with latest upgrades. This SharePoint server was setup by a third party and prepared as just a regular SharePoint Server. I had to prepare the Business Connector as Managed Account. I also created a new Web Application designated for Enterprise Portal. Just to be safe I also created a root Site Collection and tested it ok.
Unfortunately, when I tried to install Enterprise Portal I constantly hit this error:
"Microsoft SharePoint 2010 is not installed or running. Please run the prerequisite utility for more information. Operation is not valid due to the current state of the object."
Now, obviously SharePoint 2010 is not installed, but SharePoint 2013. I believe the last part of the error is the important part. I did decompile the code to see if I could understand why it failed, but didn't find anything obvious.
Instead I found the necessary clue in the setup log. You know everytime you run setup a log is written to disk, normally under "C:\Program Files\Microsoft Dynamics AX\60\Setup Logs\". The log for this run showed me that when it was iterating over the Web Sites, probably for the presentation of them into the drop down list, it failed and threw the error. The last site it was traversing in the log was the default one, "SharePoint - 80".
So I head back to IIS and stopped the Web Site causing Setup to fail and reran setup again. Sure enough, the error was gone and I could successfully select the Enterprise Portal Web Application and continue setup.
Maybe this was just bad luck, or perhaps this will help someone.