How to Avoid a Rough Path to Business Intelligence Success

Leonard Murphy   •   09.17.2019

Do you think Business Intelligence (BI) is something only large companies have? Do you think of it as a fancy name for reporting? On the contrary, Business Intelligence is much more than report writing. And it is very much something that small and medium businesses can use.

Before we talk about BI specifically, let’s talk about how reporting evolves. Although every organization is different, you should recognize the general trend.

 

Reporting Level 0: Built-In Reports

Most organizations begin with applications that include pre-built reports. 

Depending on the software, built-in reports can range from being useless to being all you need. Some applications only include simple reports that you have to learn to live with. Others may allow you to customize reports through a user interface. The best systems allow you to connect to the underlying data, or at least export raw data to Excel.

There is nothing wrong with relying on built-in reports if they meet your needs, but they can feel limiting. One day you likely will have needs that built-in reports cannot help with.

Figure 1: Example of built-in report

 

Reporting Level 1: Excel & SQL

As your organization grows, you will need to start building your own reports.

Often, this is as simple as exporting data to Excel or Access and then working with it there. You might start out sorting the data by improving its usefulness or reformatting it to call attention to an essential part.

At the same time, many organizations start using SQL. SQL allows you to choose the data you want to extract and organize it in a way that makes sense to you. By writing SQL, you also become much more familiar with the underlying database. In turn, this gives you a much deeper understanding of the data.

The best part of both Excel and SQL is that they are easy to learn. A talented individual with access to Google can become proficient. Formal training is not required.

Figure 2: Example of SQL code

 

Reporting Level 2: Wizardry

Both SQL and Excel can be revolutionary for an organization. You move away from extracts of raw data and can start to apply business rules to make reports more valuable. There is no limit to the innovative things you can do.
 
The challenge is that the very same business rules make reports more complex. Excel spreadsheets eventually grow to include lots of hidden formulas and logic. SQL code becomes more advanced and complex to decipher. The number of people who understand all the intricacies behind these reports begins to diminish. As people leave the organization, reports that no-one else understands are left behind.

Complex business logic also slows down report creation. You are no longer creating a quick data extract. Instead, you have to replicate logic from one or more reports. Mistakes are easy to make. At the same time, the number of reports proliferates.

Over time, business logic changes, but earlier reports do not get updated. The “can do anything” benefits of Excel & SQL that were once revolutionary start to hold you back and slow you down.

 

Reporting Level 3: The Flood

As organizations grow, more and more people start needing and asking for data. If one department or individual has success with a report, others will want to do the same. Pretty soon, demand can’t keep up with supply.
 
Unfortunately, each report must still be lovingly hand-crafted by an expert report builder. These report artisans can only do so much in a day.

Since you only have a few expert report builders, the next logical step is to try and rank incoming requests. Recognize this?

Email Image - Talavant Business Intelligence Consulting

Figure 3: Example report request. How long will it be before it gets done?

You send an email to a centralized address or fill in a report request form. You then wait weeks or months for a response. The process tries to discourage frivolous or low-value requests. In truth, most claims are not trivial. Business users adapt and learn to explain better the value of what they want. However, there are still more requests than can be completed.

 

Reporting Level 4: Bypassing IT

When it becomes clear that IT cannot build every report, the next logical step is to bypass IT. Talented individuals in the business gain direct access to the data they need.

While it sounds like self-service reporting, it is not. Yes, business users can now build reports. However, in reality, this is the worst of both worlds.

  1. Burdensome. The business user still has to know & learn SQL to be effective. It continues to be on IT’s shoulders to train and support the self-service business users, whose skill levels can vary.
  2. Error Prone. Mistakes are still easy to make, regardless of who builds the report. Maintaining complex business logic across many different reports remains a challenge.
  3. Insecure. Direct access to the raw data often means complete & unlimited access to everything. Any security that exists in the system gets bypassed. If the person building their own reports is a manager, this might be okay. The reality is, subordinates often run reports. Soon lots of people have access to more than they should.
  4. Risky. Software applications don’t function the same as reporting tools. A user running a large data extract during business hours can slow or bring down the entire system. They may even be unaware that they are the cause of the outage. (So yes, that means they keep doing it until someone tells them to stop.)

 

The Artisanal Reporting Gap

At this point, each report is still being lovingly hand-crafted by an artisan. We have done as much as we can to improve report throughput. We’ve tried expanding access to the data. We’ve tried reducing the number of report requests. However, only a small percentage of report requests continue to be delivered. At the same time, power users in the business are bypassing IT as much as possible.

The simple truth is: artisanal report writing does not scale. 

You could hire more report writers, but if you do, the chances are that demand will increase too. 

One option is to reject report requests that aren’t valuable, but most are of value. Very few organizations are getting too much benefit from their data. Report request processes add a lot of administrative burdens and little value.

Do not despair. There is a path forward.

 

Reporting Level 5: Business Intelligence

Business Intelligence addresses many of the pain points of artisanal reporting.

With BI, you take your data and store it in a dedicated reporting tool. In this separate tool, you also apply your business logic to the data. That business logic is only needed once, and it becomes available to all reports. In other words, you “clean” the data up-front, so that each report does not have to replicate business logic.

Benefits far beyond what I can list exist. Some, to name a few – you can simplify your data and improve usability, rename obscure tables & columns to make sense to end-users, and can combine related data into a single table instead of spreading it out across many.

Once cleaned & simplified, you can make your data available directly to end-users. Users then build their own reports using a drag & drop interface.

Many report requests are for the same data presented in different ways. You might have 80 reports focused on sales: sales by store, by salesperson, by product, by day of the week. All these variations are easy using drag & drop. End-users can explore the data immediately instead of having to wait for IT.

With reduced demand for artisanal reports, IT can focus on making more data available. In turn, this increases the value for all end-users.

Figure 4: Example of Drag & Drop Power BI Report

 

Advantages of Business Intelligence over Artisanal Reporting

  1. Easy. Data is easy for anyone to create reports. You no longer need advanced SQL/Excel skills. You only need to be able to drag-and-drop.
  2. Consistent. There is no chance underlying business logic could vary between reports.
  3. Fast. A dedicated data system returns results that are lightning-fast. A report that looks at ten years of transactions no longer has to add up each transaction.
  4. Secure. The data is both accessible and secure. If there is data you don’t want people to see, you can restrict it.
  5. Empowering. The end-user can explore the data themselves. If a number looks wrong, the end-user can break down in different ways (by date, by product, by store, by salesperson). They no longer have to email IT and wait days for someone to investigate for them.

 

Disadvantages of Business Intelligence

The two main downsides to Business Intelligence:

  1. Not real-time. Typically, reports get updated overnight for the previous day. So, if you’re looking to spot a problem as soon as it happens, a SQL report still makes sense. In this case, the SQL is much simpler. It is looking for a specific scenario and alerting you to it – it is not adding up ten years worth of transactions.
  2. Not a data extract tool. After years of artisanal reporting, end-users often prefer data extracts. They prefer to load data into Excel, where they have full control. Business Intelligence is not a data extract tool – it works best when you allow it to analyze for you. Quite a change for advanced consumers of data.

 

Conclusion

As you can see, Business Intelligence is more than writing reports. Business Intelligence is about empowering end-users (reaching all ends of the business) to write reports. It gives users safe access to their data. And it provides an easy-to-understand interface to build reports. IT ceases to be a bottleneck that slows down access to vital business data. Now, anyone with drag-and-drop skills can explore their business data without delay.
 
If you work in IT and are drowning in reports, I encourage you to learn more about BI.
 
If you work outside IT and it takes a long time for you to get the data you need, advocate for BI solutions.
 
Having BI capabilities is a competitive advantage, giving the business the ability to spot problems and identify trends. It enables fact-based decisions and quicker results. It is a win-win for both IT and the business.
 
If you like the sound of Business Intelligence, get in touch with Talavant. We have traveled this path many times with many different organizations and can help you.

Leonard Murphy

Senior Consultant

Share This Article


At Talavant, our goal is to help you discover, ask and answer the right questions – so you get more value from your data to drive your business forward. You’ll gain a competitive edge with faster, more relevant analytics. It’s all part of our focused, holistic approach that goes beyond tools.

Ready to get started?

Contact Us