Do you think Business Intelligence (BI) is something only large companies have? Do you think of it as a fancy name for reporting? On the contrary, Business Intelligence is much more than report writing. And it is very much something that small and medium businesses can use.
Before we talk about BI specifically, let’s talk about how reporting evolves. Although every organization is different, you should recognize the general trend.
Reporting Level 0: Built-In Reports
Most organizations begin with applications that include pre-built reports.
Depending on the software, built-in reports can range from being useless to being all you need. Some applications only include simple reports that you have to learn to live with. Others may allow you to customize reports through a user interface. The best systems allow you to connect to the underlying data, or at least export raw data to Excel.
There is nothing wrong with relying on built-in reports if they meet your needs, but they can feel limiting. One day you likely will have needs that built-in reports cannot help with.
Figure 1: Example of built-in report
Reporting Level 1: Excel & SQL
At the same time, many organizations start using SQL. SQL allows you to choose the data you want to extract and organize it in a way that makes sense to you. By writing SQL, you also become much more familiar with the underlying database. In turn, this gives you a much deeper understanding of the data.
The best part of both Excel and SQL is that they are easy to learn. A talented individual with access to Google can become proficient. Formal training is not required.
Figure 2: Example of SQL code
Reporting Level 2: Wizardry
Complex business logic also slows down report creation. You are no longer creating a quick data extract. Instead, you have to replicate logic from one or more reports. Mistakes are easy to make. At the same time, the number of reports proliferates.
Over time, business logic changes, but earlier reports do not get updated. The “can do anything” benefits of Excel & SQL that were once revolutionary start to hold you back and slow you down.
Reporting Level 3: The Flood
Since you only have a few expert report builders, the next logical step is to try and rank incoming requests. Recognize this?
Figure 3: Example report request. How long will it be before it gets done?
You send an email to a centralized address or fill in a report request form. You then wait weeks or months for a response. The process tries to discourage frivolous or low-value requests. In truth, most claims are not trivial. Business users adapt and learn to explain better the value of what they want. However, there are still more requests than can be completed.
Reporting Level 4: Bypassing IT
When it becomes clear that IT cannot build every report, the next logical step is to bypass IT. Talented individuals in the business gain direct access to the data they need.
While it sounds like self-service reporting, it is not. Yes, business users can now build reports. However, in reality, this is the worst of both worlds.
- Burdensome. The business user still has to know & learn SQL to be effective. It continues to be on IT’s shoulders to train and support the self-service business users, whose skill levels can vary.
- Error Prone. Mistakes are still easy to make, regardless of who builds the report. Maintaining complex business logic across many different reports remains a challenge.
- Insecure. Direct access to the raw data often means complete & unlimited access to everything. Any security that exists in the system gets bypassed. If the person building their own reports is a manager, this might be okay. The reality is, subordinates often run reports. Soon lots of people have access to more than they should.
- Risky. Software applications don’t function the same as reporting tools. A user running a large data extract during business hours can slow or bring down the entire system. They may even be unaware that they are the cause of the outage. (So yes, that means they keep doing it until someone tells them to stop.)
The Artisanal Reporting Gap
At this point, each report is still being lovingly hand-crafted by an artisan. We have done as much as we can to improve report throughput. We’ve tried expanding access to the data. We’ve tried reducing the number of report requests. However, only a small percentage of report requests continue to be delivered. At the same time, power users in the business are bypassing IT as much as possible.
The simple truth is: artisanal report writing does not scale.
You could hire more report writers, but if you do, the chances are that demand will increase too.
One option is to reject report requests that aren’t valuable, but most are of value. Very few organizations are getting too much benefit from their data. Report request processes add a lot of administrative burdens and little value.
Do not despair. There is a path forward.
Reporting Level 5: Business Intelligence
Business Intelligence addresses many of the pain points of artisanal reporting.
With BI, you take your data and store it in a dedicated reporting tool. In this separate tool, you also apply your business logic to the data. That business logic is only needed once, and it becomes available to all reports. In other words, you “clean” the data up-front, so that each report does not have to replicate business logic.
Benefits far beyond what I can list exist. Some, to name a few – you can simplify your data and improve usability, rename obscure tables & columns to make sense to end-users, and can combine related data into a single table instead of spreading it out across many.
Once cleaned & simplified, you can make your data available directly to end-users. Users then build their own reports using a drag & drop interface.
Many report requests are for the same data presented in different ways. You might have 80 reports focused on sales: sales by store, by salesperson, by product, by day of the week. All these variations are easy using drag & drop. End-users can explore the data immediately instead of having to wait for IT.
With reduced demand for artisanal reports, IT can focus on making more data available. In turn, this increases the value for all end-users.
Figure 4: Example of Drag & Drop Power BI Report
Advantages of Business Intelligence over Artisanal Reporting
- Easy. Data is easy for anyone to create reports. You no longer need advanced SQL/Excel skills. You only need to be able to drag-and-drop.
- Consistent. There is no chance underlying business logic could vary between reports.
- Fast. A dedicated data system returns results that are lightning-fast. A report that looks at ten years of transactions no longer has to add up each transaction.
- Secure. The data is both accessible and secure. If there is data you don’t want people to see, you can restrict it.
- Empowering. The end-user can explore the data themselves. If a number looks wrong, the end-user can break down in different ways (by date, by product, by store, by salesperson). They no longer have to email IT and wait days for someone to investigate for them.
Disadvantages of Business Intelligence
The two main downsides to Business Intelligence:
- Not real-time. Typically, reports get updated overnight for the previous day. So, if you’re looking to spot a problem as soon as it happens, a SQL report still makes sense. In this case, the SQL is much simpler. It is looking for a specific scenario and alerting you to it – it is not adding up ten years worth of transactions.
- Not a data extract tool. After years of artisanal reporting, end-users often prefer data extracts. They prefer to load data into Excel, where they have full control. Business Intelligence is not a data extract tool – it works best when you allow it to analyze for you. Quite a change for advanced consumers of data.