8.2 Foundations of Business Intelligence and Analytics

Valuable Insights

If data truly make up the “new oil,” then that analogy can go a step further. Raw data—like crude oil—are completely useless until trained individuals use the proper tools to refine them. Once they refine the data, then business leaders can glean valuable information from that data. Simply put, the data are initially too overwhelming for humans to manually sift through. Data analytics automates the sifting process by deploying various analytics tools and algorithms to identify and highlight important information.

In the current business landscape, decision-making must be based on accurate and currently relevant information. Business intelligence and analytics play a vital role in this process. By consolidating data from various sources and transforming that data into meaningful insights, BI equips decision-makers with a comprehensive understanding of the organization’s current state and its prospects. With analytics, decision-makers can evaluate different scenarios, perform predictive modeling, and simulate potential outcomes. These capabilities enable them to make informed decisions with less uncertainty.

Visualization

Another benefit of BI tools is the ability to produce a visualization, or a graphical representation of data and information to facilitate understanding, analysis, and communication of insights and trends. Many executive leaders may have a firm grasp on fundamental statistics, but they may lack the training and experience to derive real meaning from large amounts of data. This is where visuals are helpful for communicating vital information. Figure 8.6 shows a dashboard that is an example of an effective way to provide visual expressions from multiple data sources at once. A dashboard can facilitate easier communication of sometimes complex ideas more effectively, especially to an audience with minimal experience in technical fields.

Figure 8.6 A dashboard tool brings metrics from multiple sources into one page for quick comparison. (credit: modification of work “Data visualization” by “Kabuttey”/Wikimedia Commons, CC BY 4.0)

Key Performance Indicators and Benchmarking

Business leaders are often analyzing how an organization is doing, and key performance indicators (KPIs) are an important part of that analysis. A key performance indicators (KPIs) is a measurable value that demonstrates how effectively a company is achieving its key business objectives and goals. Related to that is the concept of benchmarking, which is the comparison of an organization's performance against industry standards and competing businesses to identify areas for improvement and drive performance optimization. Business intelligence tools make it possible to identify KPIs and benchmark levels centered around efficiency, profit, and other metrics and assess the performance of the project or initiatives in real time.

Inventory Management

Business intelligence tools make life easier in terms of purchasing, procurement, and inventory management. Companies can generate reports on shipping and receiving and automate the process of ordering materials before they are below a certain threshold. Business intelligence also tracks outbound materials, so organizations can identify purchasing trends and reduce or eliminate wasteful spending. One way to quickly observe trends, costs, and other metrics is through a visual tool similar to the dashboard shown in Figure 8.6 that would instead show multiple metrics such as the on-hand quantity of products, the value of goods issued, and links to tasks such as ordering new stock.

Customer Analytics

The process of customer analytics represents a step forward in creating value from data by analyzing and synthesizing information about the customer, providing a customer-centric focus, and providing decision-making support. Through the careful analysis of customer data, business leaders can better understand the expectations, habits, and preferences of potential customers. These data are used to build consumer engagement and loyalty, improve the performance of marketing campaigns, and even identify additional distribution channels. It is helpful if customer analytics provides predictive recommendations.

For example, suppose you run a clothing store, and you wish to track the popularity of a specific design of pants. After you run some data queries on social media, mobile, and cloud media communications using web crawlers, you might discover a group of people with similar traits or features, as they are referred to in analytics. The results reveal that the typical customer is between twenty and twenty-four years old, and that the pants sell online 80 percent of the time (rather than in a physical store). This information indicates where your marketing efforts should be focused. In this case, the company should promote the pants online with less emphasis on the physical location and should direct ads to consumers under the age of thirty. To make that determination, an organization can use BI tools to conduct a recency, frequency, and monetary (RFM) analysis, which is the task of customer segmentation or grouping based on their purchasing habits. Essentially, four pieces of information are needed to create RFM scores:

• identity: depersonalized information, such as a customer ID
• recency: how long ago the last purchase was made
• frequency: how many purchases or conversions the customer made over a specified period
• monetary: total spent

Note that RFM is not strictly a tool used by for-profit companies. It can be beneficial for a variety of organizations, including hospitals and places of higher education. In health-care settings, RFM analysis can help prioritize patients based on their recency of visits, frequency of appointments, and monetary value of services utilized, facilitating targeted outreach and resource allocation. An RFM analysis can help with patient retention by classifying patients based on recent visits and service usage to design personalized follow-ups or wellness programs, and it can help with service optimization by prioritizing high-value patients for loyalty programs or preventive care services. In the education sector, RFM can help with targeted student engagement by identifying students who are most engaged (high frequency and recency of interaction) to tailor resources or interventions, and in donor analysis by recognizing high-value alumni donors based on donation patterns for optimized fundraising efforts. Using this information, you would then define an RFM score on a scale of 1 to 6 for each customer. An RFM analysis primarily focuses on behavioral data rather than personal traits like age or income level. While RFM analysis does not directly incorporate personal traits, it can indirectly reflect certain characteristics of customers based on their purchasing behavior.

Business Intelligence Tools Offer Competitive Advantage

In an environment where companies scramble to outdo one another, BI tools offer a competitive advantage crucial for sustainable success and also provide a pathway toward achieving this advantage. For people working in an increasingly competitive market, the ability to make fast decisions is part of the job, but knowing those decisions are based on hard evidence provided by data-driven insights offers greater confidence. By harnessing the power of big data, businesses can uncover hidden opportunities, identify emerging trends, and anticipate market shifts. This enables them to stay ahead of their competitors, respond quickly to changing customer needs, and capitalize on new business prospects.

By recognizing the benefits of BI and analytics, understanding the part these tools can play in decision-making, and leveraging them to gain a competitive advantage, organizations can position themselves for success in today’s data-centric commerce environment. By deploying BI analytics tools, organizations gain superior insights into their operations, customers, and competitive landscape; find new potential customers; and make more well-informed decisions.

Customer data analytics can revolutionize business performance and customer loyalty. Data-driven customer insights are invaluable, but successful deployment requires a strategic focus on ROI and customer-centric innovation. Data are no longer limited to reporting; data are now deeply embedded in operational and decision-making systems. For example, companies such as Amazon, Google, and Netflix use data as a product to drive innovations like recommendation engines. Personalized offers and experiences make customers feel valued, increasing their loyalty to the company. Analytics is not just a tool; it is a strategy that aligns with a company’s objectives and operations.

Decision-Making

Business intelligence and analytics provide organizations with the necessary insights to make informed and strategic choices. By analyzing data from multiple sources, organizations can identify trends, patterns, and correlations that impact their operations. These insights enable decision-makers to evaluate different scenarios, assess risks, and determine the most effective course of action. For instance, a retail company analyzing customer purchasing patterns through BI might discover that certain products experience increased demand during specific seasons, prompting them to adjust inventory levels and tailor promotional strategies accordingly, resulting in optimized sales performance and customer satisfaction.

Time-Series Analysis

Time-series data consists of information collected on the same topic over a specified period. Examples can include the employment rate in a country over one year, the stock price of a specific company over the last year, or the attrition rate at a college from the fall through the following summer. Any data recorded continuously at different time intervals is considered time-series data. For example, Figure 8.7 shows a chart of time-series data from the National Park Service that compares horse population growth and foal production over several decades.

Figure 8.7 A time-series graph can compare multiple sets of data over the same period, such as the horse population growth and foal production on Assateague Island National Seashore, Maryland. The blue line represents the horse population, and the red line represents foal births. (credit: modification of work "Population Growth and Foal Production" by NPS/National Park Service, Public Domain)

Continuous time-series data refers to a stream of information that is collected or recorded over time without interruptions. It’s essentially taking measurements or observations regularly, such as every minute, hour, or day, to track how something changes over a period. It could be, for example, monitoring temperature every hour throughout the day to observe how it fluctuates.

Decision Trees

Contemporary management challenges are not influenced by isolated decisions but rather by a series of decisions. Business leaders recognize the importance of how decisions made today may have a profound impact on future conditions. One analytics tool that speaks to this concept involves the use of decision trees. A decision tree in BI or data analytics is a decision-making tool that uses a tree structure diagram in which branches represent choices and their outcomes. They start with a question, then branch out based on the answers to subsequent questions, finally leading to a decision or prediction. For instance, in retail, a decision tree might help decide which customers are likely to buy a product based on factors like their age, purchase history, and location, helping businesses target their marketing efforts effectively.

Decision trees provide a framework to visualize the potential cause-and-effect relationship between decisions and future outcomes. They present a visual guide to show decision-making processes and future outcomes. The parts of a decision tree include the following:

• root node: the beginning, where the decision tree starts
• leaf node: the end, or the final output node
• splitting: dividing from decision nodes into subnodes according to the given conditions
• subtree: a subsection or branch

To better appreciate decision trees, consider Figure 8.8, which shows how it is possible to break down the decision of what drink to buy from a coffee shop. The first root node involves deciding between tea and coffee. If the customer decides to buy tea, they want it to be herbal. If the coffee shop does not have herbal tea, they want it to be iced. What if the coffee shop doesn’t carry tea at all? What beverage will they drink then?

Figure 8.8 A decision tree can step through a user’s choices for deciding on a drink at a coffee shop. (attribution: Copyright Rice University, OpenStax, under CC BY 4.0 license)

Marketing

Business intelligence and analytics also play a critical role in understanding customer behavior, preferences, and market trends. By analyzing customer data, organizations can develop targeted marketing campaigns, personalized promotions, and tailored product offerings. Targeted marketing involves knowing who your audience is and providing services accordingly. For example, Rakuten Travel understands that international customers prefer a clean, simple user interface, whereas potential customers from their home country of Japan typically prefer a busier page with more options, and Rakuten directs users to the appropriate version of the site accordingly.⁶ Additionally, BI and analytics help organizations assess the effectiveness of marketing initiatives, track campaign performance, and measure customer satisfaction, enabling them to optimize their marketing strategies for maximum impact. In today’s digital age, organizations leverage advanced BI and analytics technologies to assess the effectiveness of their marketing initiatives, track campaign performance, and measure customer satisfaction in ways that were not possible a decade or so ago.

For assessing the effectiveness of marketing initiatives, modern organizations harness the power of predictive analytics, machine learning algorithms, and data visualization tools.

When it comes to tracking campaign performance, real-time analytics platforms and marketing automation software play a crucial role. These tools provide organizations with immediate feedback on KPIs such as click-through rates, conversion rates, and engagement metrics. By monitoring these metrics in real time, organizations can make timely adjustments to their campaigns to optimize performance and maximize impact.

Furthermore, measuring customer satisfaction has been revolutionized by the advent of sentiment analysis tools and customer feedback platforms. These technologies allow organizations to analyze customer feedback from various channels, including social media, surveys, and online reviews. By understanding customer sentiment and identifying areas for improvement, organizations can enhance the overall customer experience and strengthen customer loyalty.

In essence, the integration of modern BI and analytics technologies enables organizations to assess the effectiveness of their marketing initiatives and track campaign performance and measure customer satisfaction with unprecedented accuracy and efficiency.

Financial Analysis

BI and analytics prove invaluable in financial analysis. Organizations can use these tools to consolidate and analyze financial data, identify cost-saving opportunities, detect anomalies or fraud, create sales projections, and optimize budget allocation. In Figure 8.9, a projection is made by analyzing historic sales data and extrapolating potential future sales in units over time. By gaining a comprehensive view of their financial performance, organizations can make data-driven decisions that improve profitability and financial stability.

Figure 8.9 A time-series chart can project potential sales based on historical data. (attribution: Copyright Rice University, OpenStax, under CC BY 4.0 license)

Supply Chain Analysis

The process of supply chain analysis is crucial for optimizing operational efficiency and ensuring timely delivery of goods and services. Business intelligence and analytics enable organizations to track inventory levels, monitor supplier performance, analyze demand patterns, and identify areas for cost reduction and process improvement. Companies can leverage BI to examine the benefits of decisions and identify opportunities to reduce costs using tools such as a cost-benefit analysis, which is a systematic approach to assessing the costs and benefits of a proposed project, investment, or decision to determine its feasibility and potential ROI. For example, metrics obtained from the supply chain can help better understand driver behavior and lead to more efficient routes. This data-driven approach enhances supply chain visibility, streamlines logistics, and ultimately improves customer satisfaction. Business intelligence tools enable organizations to transform raw data into actionable insights, empowering them to make informed decisions, adapt to market dynamics, and maximize their potential for success.

Tools for Visualization

Data visualization is a key aspect of data analysis and communication. Effective data visualization not only helps to convey complex information but also aids in decision-making by providing a clear and intuitive understanding of the data.

In terms of modeling and analysis, tools such as Excel, R, and Python can be useful. They provide a wide range of statistical and analytical functionalities that enable users to explore and analyze datasets. Data analytics professionals apply quantitative and qualitative data analysis techniques, understand statistical concepts, and use these tools to build models for predictive analytics and decision support. Today, there are many options for visuals that are typical static charts, but there are also newer interactive charts that allow viewers to explore the data in greater detail or with different parameters. Demonstrations like this can have a strong impact on an audience.

Data Mining

Another important concept, data mining, involves the extraction of valuable information and patterns from large datasets. Data mining can be applied to solve real-world problems and support decision-making processes. One remarkable success story in data mining comes from Netflix’s recommendation system. Using a custom algorithm, the streaming company analyzes billions of data points to predict what content a viewer may like.⁷

Professionals who develop proficiency with tools will be able to work with data effectively, conduct quantitative and qualitative analysis, apply data mining techniques, and present findings in a visually compelling manner. This knowledge can further enable you to uncover insights and KPIs, make informed decisions, and contribute to the success of an organization in the field of BI and analytics.

Predictive Analytics

The use of statistical modeling, data mining, machine learning, and other analytical techniques to forecast future outcomes based on historical data patterns is called predictive analytics. The key principle is to identify meaningful relationships and patterns within the data that can be used to make predictions. This involves understanding concepts such as training and testing data, feature selection, model evaluation, and accuracy assessment.

To learn how predictive analytics works, consider this question: If you study more hours, will your midterm exam score increase? In other words, you want to determine whether there is a positive relationship between the number of hours studied and the score on midterm exams. Although the answer to this question might seem obvious, it is an effective scenario to demonstrate predictive analytics.

To illustrate prediction and regression, suppose the dataset comes from a group of ten people who take a fifty-question exam and provide the number hours they spent preparing for the exam. Each question is worth one point. If you were to chart the results for each participant with the x-axis representing the time they spent studying and the y-axis representing the resulting grade, it would be possible to generate a visualization like the one in Figure 8.10. This demonstrates regression, which is a statistical analysis method used to quantify the relationship between variables and to make predictions. Note that analysts would typically use regression to form a hypothesis on a dataset that is much larger than our sample population of ten. This smaller example is used for illustrative purposes only.

Figure 8.10 This regression analysis shows the results from a hypothetical study exploring the correlation between time spent studying and test scores. (attribution: Copyright Rice University, OpenStax, under CC BY 4.0 license)

Simple linear regression is a method that presents the relationship between variables as a linear equation on a graph, for example, predicting house prices based on features like size, location, and number of bedrooms. It involves plotting x and y points along a line and determining whether there is a relationship between the variables, and by what margin. If the plotted data follow an upward trend from left to right, there is a positive correlation. Figure 8.10 shows a positive correlation between time spent studying and exam scores.

In a positive correlation, both variables are moving in a positive direction together. There is one dependent variable (the y-variable), which is the score on the exam, and one independent variable (the x-variable), which is the time spent studying.

There are three important points to remember about linear regression:

• To get an ideal solution, you need data from the whole population. Since that may not be feasible, you could pull a sample to represent the population.
• After acquiring the data, you must choose a relevant model. This can be a daunting task at first but can be done by considering the volume of data, determining whether it is continuous and deciding whether to perform classification or prediction.
• After modeling, you can form predictions.

In linear regression, the term “linear” implies that as the value of one item increases, the other is changing in parallel. Consider the equation for a line: y = mx + b, where the following is true:

• y is a dependent variable (outcome). This is the predicted value. In this example, the y-variable is the exam score.
• x is an independent variable. It is usually time or some other linear value. In this example, the x-variable is the time spent studying.
• m is the slope of the line.
• b is the y-intercept value.

In the equation of a line, y is a function of x. To make a solid prediction, you need to find the values of m and b. The variable m represents the slope of the line, which is the rate of change in the dependent variable (y) per unit change in the independent variable (x). In simpler terms, it shows how much y increases (or decreases) for each additional unit of x. So, if m is positive, it means that as x increases, y tends to increase, and if m is negative, it means that as x increases, y tends to decrease.

The variable b represents the y-intercept of the line, which is the value of y when x is equal to zero. In other words, it gives the starting point of the line on the y-axis.

In the example of time spent studying (x) and exam scores (y), the slope (m) would show how much the grade tends to increase (or decrease) for each additional hour of study time. The intercept (b) would represent the grade a student might get if they didn’t study at all (x = 0).

The variables m and b are not degrees of correlation but rather parameters that help to define the regression line and understand the relationship between the variables. They provide crucial information about the direction, steepness, and starting point of the line that best fits the data.

In Table 8.1, the values from the study of ten participants with the number of hours studied and the number of correct answers on the fifty-question exam are shown.

After checking some new x values (time) to predict the scores, it becomes possible to form a prediction based on new y values. Now, you can identify predicted scores and how much they vary from the actual score, expressed in terms of error. Refer to Table 8.2 for the computed values, and view Figure 8.11 for how the predictions would plot on a graph.

Figure 8.11 Values from the study are plugged in and calculated, forming predicted grades that can be plotted on a graph. (attribution: Copyright Rice University, OpenStax, under CC BY 4.0 license)

Notice that the values of the predicted score surpassed the maximum score of the fifty-question exam. The implication is that studying thirty hours or more would result in getting a higher than perfect score, which obviously is not possible. This highlights the important fact that no predictive model is perfect. However, the example does demonstrate positive correlation.

Business Intelligence Reporting

The process of collecting, analyzing, and presenting data in a format that communicates insights derived from BI analysis to support decision-making within an organization is called business intelligence reporting. It focuses on transforming raw data into meaningful information and insights through an interactive dashboard, reports, and visualizations. The goal is to provide stakeholders with accurate, relevant, and timely information to support strategic, tactical, and operational decision-making.

Data scientists and analysts play a critical role in designing and developing BI reports. They identify KPIs, define data requirements, select appropriate visualizations, and create reports that cater to the specific needs of stakeholders. This process involves data modeling, report design, and development of data-driven visualizations. Data visualization tools like Tableau, Microsoft Power BI, or custom-built solutions play a vital role in aiding managers in decision-making processes.

Here are several ways in which BI reporting supports managers:

• access to real-time and accurate information
• performance monitoring and KPIs
• data visualization and analysis
• identification of trends and opportunities
• data-driven decision-making

Despite its advantages, BI reporting also has potential disadvantages that organizations may encounter:

• Implementation can be complex and costly.
• Business intelligence reporting relies heavily on data accuracy, so data quality and successful integration are important.
• Organizations may become overly reliant on technology and infrastructure.
• Business intelligence reporting involves handling sensitive business data, so it is necessary for organizations to maintain privacy and security.