1. Unrealistic expectations
An organization that just starts throwing data at novice users is facing a serious risk of poor-quality reports. It will be very lucky if these users with different qualifications wind up with non-misinterpreted data without first learning the basics of reporting.
For instance, a happy user creating their first report on total sales in a historic period might end up with average numbers instead of a SUM, knowing nothing about default aggregations for various measures. Or on the contrary, they can submit inflated numbers. There is also risk of data inconsistency that might affect weighted averages when they need to be displayed with different levels of granularity.
Further on, a non-power user might rest satisfied with just a casual analysis that has supported their initial beliefs. The confirmation or cherry-picking bias trap is not something an untrained user is necessarily aware of, especially when under pressure to explain a certain pattern.
2. Reporting chaos
Self-service BI doesn’t mean zero IT involvement. Letting users toy around with data with no governance from IT usually leads to reporting anarchy.
With no governance, there could be redundant reports from different users working in silos and delivering the same analysis or reports from different users analyzing the same metrics but using different filters and hence delivering conflicting results. Reports from different departments can rely on different naming conventions for quantity, value, or time or use the same terms but not necessarily the same definition. Multiple versions of the same database, errors in databases that are never fixed, the creation of objects used only once … The list is endless.
Governance is not something that a data-driven organization can boycott in the world of self-service. No matter how badly a company wants to free users for conducting their own analysis, IT still needs to be involved to maintain high data quality and consistency.
3. Lack of adoption
Truth is, not everyone likes to work hard. Most business users just want a simple dashboard that will give them the numbers. Valuable insights, however, often lie levels deeper that go beyond plain business performance analysis.
Another psychological factor that may hold back an efficient self-service BI is resistance to change. It is not uncommon for many organizations in the early stages of their self-service BI journey to see frustrated business users coming back to BI or IT to request a report as they did in the good old times. Older approaches are safer.
Unfriendly self-service BI environment setups also might be a problem. What may seem for IT or BI teams to be an easy-to-use tool for collecting and refining results can have an overwhelming and demotivating amount of features for a casual user without technical skills. Pivotal tables and spreadsheets might be dull, but users are quick to revert to them
when getting stuck.