ERROR: Failed to set up BAM database(s).
Database 'BAMAnalysis' has compatibility level larger than 1103 with StorageEngineUsed set to InMemory. Tabular databases with compatibility level at 1200 or above must use StorageEngineUsed set to TabularMetadata.
This error is caused by the SSAS instance was installed using tabular mode, and BizTalk need a multi-dimensional mode SSAS instance.
The solution is to change the \Program Files\Microsoft SQL Server\MSAS14.MSSQLSERVER\OLAP\Config COnfig\msmdsrv.ini file to use <DeploymentMode>0</DeploymentMode>, then restart the instance. It will change from a tabular instance to a multi-dimensional instance