The Big Data Fairy Tale

By Roel Castelein

Fairy tales usually start with ‘Once upon a time ...' and end with ‘... And they lived long and happily ever after'. But nobody explains ‘how' the heroes live long and happily ever after. Big data (analytics) promise to transform your business, but just as in fairy tale endings, big data will not explain ‘how' to transform your organization. In my view, big data might spark some behavioral change or open people's minds, but it will not transform organizations. At best, big data evolves organizations. Let's look at the concept and a concrete example to draw conclusions.


What big data analytics does is take a bunch of data, analyze and visualize it, and then derive insights that potentially can improve your organization or business. Based on these insights the actual transformation can begin, but it requires more than just big data. Let's have a look at a classic example of data analytics; the reduction of crime in New York under Mayor Giuliani with the help of CompStat.

CompStat is a data system that maps crime geographically and in terms of emerging criminal patterns, as well as charting officer performance by quantifying criminal apprehensions. The key to success was not the data or analysis, but that the organizational management that used the data and analysis was effective. Processes, structures and accountability were setup to drive the transformation. In weekly meetings, NYPD executives met with local precinct commanders from the eight boroughs in New York to discuss the problems. They devised strategies and tactics to solve problems, reduce crime, and ultimately improve quality of life in their assigned area. CompStat tracked the results of these strategies and tactics, and whether they were successful or not. Precinct commanders were held accountable for the results.

Drawing upon my own experience, I know how difficult an organizational transformation is. Even if you have the data and the analysis that shows things need to change, it requires much more than data analysis. Let's assume that the data uncovers opportunities for improvement, either in reducing cost or in increasing revenue. The next step is to design the changes in processes, in people's roles, in org charts and in the systems. This usually entails a two pronged approach; communicate the change in org charts, processes and roles, and engrain these changes in the systems to track the change results. This tracking creates a feedback loop, necessary to manage the transformation.

Another challenge in the big data transformation message is finding the right people. Ideally the team leading the transformation needs to understand an organization's data, enriched with outside data, then know how to do data analysis, and once the results are there, strategically communicate the change to get everybody on board. Next, the transformation team needs to set up a tracking and feedback process that holds participants accountable for the transformation results. And when participants do not play along, have an escalation process in place, with the possibility for punitive measures.

In the same way that Giuliani fired one of the precinct commanders when he showed up drunk at the first CompStat meeting, big data systems require a complementary management philosophy to ensure whatever transformational insights are derived get implemented and controlled.

So, when the advertisements claim that big data will transform your business, remember that big data brings the potential for transformation, not the actual transformation. That still requires commitment and hard work, just like ‘living long and happily ever after'. That's why they are called fairy tales.

How Big Data Brings BI and Predictive Analytics Together

Big data is breathing new life into business intelligence by putting the power of prediction into the hands of everyday decision-makers.


For as long as anyone can remember, the world of predictive analytics has been the exclusive realm of ivory-tower statisticians and data scientists who sit far away from the everyday line of business decision maker. Big data is about to change that.
As more data streams come online and are integrated into existing BI, CRM, ERP and other mission-critical business systems, the ever-elusive (and oh so profitable) single view of the customer may finally come into focus. While most customer service and field sales representatives have yet to feel the impact, companies such as IBM and MicroStrategy are working to see that they do soon.

Big Data Moves Analytics Beyond Pencil-Pushers
Imagine a world in which a CSR sitting at her console can make an independent decision on whether a problem customer is worth keeping or upgrading. Imagine, too, that a field salesman can change a retailer's wine rack on the fly based on the preferences that partiers attending the jazz festival next weekend have contributed on Facebook and Twitter.
Big data is pushing a tool more commonly used for cohort and regression analysis into the hands of line-level managers, who can then use non-transactional data to make strategic, long-term business decisions about, for example, what to put on store shelves and when to put it there.
However, big data is not about to supplant traditional BI tools, says Rita Sallam, Gartner's BI analyst. If anything, big data will make BI more valuable and useful to the business. "We're always going to need to look at the past…and when you have big data, you are going to need to do that even more. BI doesn't go away. It gets enhanced by big data."
How else you will know if what you are seeing in the initial phases of discovery will indeed bear out over time. For example, do red purses really sell better than blue ones in the Midwest? An initial pass through the data may suggest so—more red purses sold last quarter than ever before, therefore, red purses sell better.
But this is a correlation, not a cause. If you look more closely, using historical transaction data gleaned from your BI tools, you may find, say, that it is actually your latest merchandise-positioning-campaign that's paying dividends because the retailers are now putting red purses at eye level.
That's why IBM's Director of Emerging Technologies, David Barnes, is actually more inclined to refer to the resulting output from big data technologies such as Hadoop, map/reduce and R as "insights." You wouldn't want to make mission-critical business decisions based on sentiment analysis of a Twitter stream, for example.

Infographic explains Lifetime Value of a Customer

++ Click Image to Enlarge ++



Source: How To Calculate Lifetime Value

Nokia flop

Nordic Choice Business Intelligence

Here is a video where shows the job that me and BI collegues from Nordic Choice Hotels have done last 2 years.
Also we collaborated with Platon (www.platon.net) in UX and Dashboard Designing of our Business Intelligence Solution.



A Visionary Choice - Nordic Choice Hotels Business Intelligence vision from Platon on Vimeo.

Data Analytics see the World differently

Data and Analytic Professionals See the World Differently from PARTNERS Conference on Vimeo.

A BI Architectures Approach to Modeling and Evolving with Analytic Databases

Major shifts converging in today's BI environment bring the opportunity to discover new answers to old questions about what BI architectures are about and how they are designed.



August 21, 2012

By John O'Brien, Principal, Radiant Advisors



Whether you have been a BI architect managing a production data warehouse for many years or are embarking on building a new data warehouse, the new analytic technologies coming out today have never been so powerful and complex to understand. In fact, with so many analytic technologies available on the market, they are somewhat overwhelming as we struggle to make sense of what to do with them and which ones to use with our existing environments.



This is good for BI architects because it brings us back to BI architecture fundamentals, key data management principles, pattern recognition, and agile processes, along with BI capabilities that challenge classic BI architecture best practices to design what clearly makes sense to meet the demands of business today.



There are major shifts converging in today's BI environment, and these changes bring with them the opportunity to discover new answers to old questions about what a BI architecture is all about and how an architecture is designed. As I explore these questions in this article, I will focus on three main themes: BI architectures are strategic platforms that evolve to their full potential; good architectures are based on recognizing data management principles (patterns and so-called best practices are discovered later); and BI architecture design is purposeful at every stage of development and technology decisions follow this purpose.



Architecture Maturity and Information Capabilities



We have all seen the research that says BI architectures evolve into robust information platforms and value over time. BI teams have focused on increasing value through maturing their data warehouses from operational reporting to data marts to data warehouses and finally to enterprise data warehouses. This evolutionary approach is typical when balancing pressing tactical needs for information delivery with strategic development, and is found in many companies where business demands for information drive towards a data warehouse platform.



The classic data warehouse is the last thing to be built, if ever, because the emphasis remains on quicker delivery first and information consistency later. Unfortunately, this leads to data warehouses that reflect current information needs and doesn't foster the evolution of a mature analytics culture.



Instead, an architecture based on BI capabilities focuses on nurturing the analytic culture of the business community by first educating user communities about the BI capabilities available and then on business subject data that is delivered via BI capabilities. This approach centers the data warehouse architecture on BI capabilities such as information delivery; reporting and parameterized reporting; dimensional analytics for goals achievement; and advanced analytics for gaining insights, to name a few. These discussions recognize that the same consistent data has many usage patterns, behaviors, and roles in the decision process. A BI architecture that is designed in this way ensures that data models and chosen analytic technologies are best suited to their intended purpose.



However, this BI-capabilities approach is contrary to some BI architects' belief that there should be an all-in-one data warehouse platform in the enterprise.