I was looking at the Organization Insights tonight ( https://technet.microsoft.com/en-us/library/mt763947.aspx ), and I just realized that it seems to be a very special solution:
When looking at that dashboard, don’t you think it would be great if we were able to get access to that information from Web API? Or, possibly, if you could set up some alerts/notifications? Usually, all the solutions we will deploy to Dynamics will follow the same approach:
- They will create some custom entities to store solution-specific data
- They will create dashboards, reports, and other UI components so you had a way to work with that data
In case with Organization Insights, it seems that we are getting the presentation “layer”, as usual, but we are not, actually, getting the data “layer”.
Technically, Dynamics 365 / CRM has never had direct access to the database/ hard drive. So, the kind of information you see on that dashboards is not, really, available in the regular Dynamics solutions:
- We can’t use SDK/Web API to estimate storage usage
- Which means we also don’t have access to the storage details per table
- We don’t have any statistics on the number of API calls
- Etc
In other words, Organization Insights solution is not, really, your regular Dynamics solution – it’s, likely, more an external component that is embedded into Dynamics through that dashboard.
So, where is it taking the data from?
I used Fiddler to capture the traffic quickly, and here is what showed up in the log:
There seem to be a separate service running on a different url (different from my Dynamics organization url), and, it seems, that’s where Organization Insights solution is getting all the data from.
By the way, I am almost certain it’s not a coincidence that both organization insights service and Dynamics organization have the same domain name: crm3.dynamics.com.
Notice how the log starts from my organization url, then it goes to the crminsights.crm.dynamics.com (where it seems to be doing some sort of discovery), and, from there, it goes to the crminsights.crm3.dynamics.com (where, it seems, it’s finally getting the actual data).
I am guessing crminsights.crm3.dynamics.com is, really, some sort of an external web service which is more or less independent from Dynamics, and which has access to the server-side statistics/information which is not available from within the Dynamics. It might be possible to sort of piggy back that service if we wanted to utilize Organization Insights data from within Dynamics, although, unless Microsoft actually documents the usage of that service, that kind of solution might not be very reliable in the long term.
There is another interesting implication, though.
There was a question in the community forums recently where somebody was trying to compare Organization Insights statistics on the AuditBase table with the information you can see in the Audit Log Management in Dynamics:
https://crmtipoftheday.com/2015/03/20/auditingsize/
There can be quite an impressive difference between those two numbers – if I ‘m not mistaken, in that particular case it was about 10 GB under Log Management vs about 35 GB reported by the Organization Insights.
Why? It turned out Organization Insights was looking at the size of the database index as well (it’s also counted against your storage allocation, so it definitely makes sense to include index size into the overall storage usage numbers); however Log Management is not looking at the size of the database index. This kind of difference is not only impressive, but it can be costly to ignore it.
So, Organization Insights solution does bring in quite a bit of unique insight which we can’t really get form anywhere else when working with Dynamics 365 (online). It does it by utilizing some kind of external web service – apparently, we are not supposed to be using that service directly.. not yet, at least. Since this solution is utilizing a special web service hosted in the cloud, there is, probably, no intention to make it available to the on-premise customers. That said, it’s a very useful solution, especially for Dynamics online clients, since it’s, probably, the only way to figure out the exact details of Dynamics storage utilization without creating a support ticket with Microsoft.