The Intelligence of a Business

“The Intelligence of a Business” is the title I had envisioned for my book published about a month ago (June 21, 2024). However, my book was instead given the name, Enterprise Intelligence. I do like the title–I approved it and understand how it could be more “marketable”. But it doesn’t capture the soul of the book.

Nonetheless, the book I ended up writing is the book I intended to write. The only thing I might have done differently is to have dived deeper into technical details. I didn’t intend for this to be a “cookbook”. I shot for the depth that provides a technical sense of plausibility for the techie half of my intended audience.

Instead of the cookbook approach, my intent is to provide a comprehensive framework for integrating various well-known concepts and technologies in business intelligence, data analytics, and artificial intelligence into a cohesive enterprise strategy. Unlike “cookbooks” that focus on providing step-by-step solutions to specific problems, this book describes the synthesis of multiple established methodologies to address broader organizational challenges with this high-level approach:

• Integrative Framework: The book pulls together well-known components of the analytics landscape such as BI, Knowledge Graphs (KGs), Data Mesh, Data Fabrics, and AI to form an overarching strategy for enterprise intelligence. Each chapter introduces on of the components, then I connect them, demonstrating how they can work together to enhance decision-making and operational efficiency.
• Conceptual Cohesion: By leveraging existing knowledge and technologies, the book avoids reinventing the wheel. Instead, with the exception of LLMs, it builds on the strengths of established methods, illustrating how they can be adapted and integrated into a unified approach for modern enterprises.
• Practical Implementation: While not a step-by-step cookbook, Enterprise Intelligence provides detailed guidance on implementing the described strategies. This includes practical examples, case studies, and scenarios that demonstrate the application of these concepts in real-world settings.

This book is about how many big subjects fit together, offering high-level implementation guidance. Consider my blog, Enterprise Intelligence: Integrating BI, Data Mesh, Knowledge Graphs, and AI, to be the comprehensive book description I really wanted but wasn’t markety and wouldn’t fit on the book jacket. Cookbook-level details on technical aspects will come in later volumes, once the foundational ideas have hopefully been digested a bit.

Notes:

• Enterprise Intelligence is available at Technics Publications and Amazon. If you purchase the book from the Technics Publications site, use the coupon code TP25 for a 25% discount off most items (as of July 10, 2024).
• Please at least read the TL;DR of my book for a quick description of the book before moving on.
• The Enterprise Intelligence category of this blog site is a list of my blogs related to the book.

Target Audience

My book is geared towards a diverse audience, such as Chief Data Officers (CDOs), Chief Artificial Intelligence Officers (CAIOs), data architects, data engineers, project managers, or any other analytics professional who isn’t a hardcore AI researcher. These professionals, who might be familiar with some components but not all, will gain an introduction to all the parts and how all the parts fit together, with Large Language Models (LLMs) serving as the connective mortar. By using BI, which is highly-curated and established for decision support, as the spearhead of the architecture, the book aims to provide a comprehensive understanding of integrating BI and AI with modern data strategies.

My BI Misunderstanding

The original title, “The Intelligence of a Business,” reflects a misunderstanding I held for most of my long Business Intelligence (BI) career. Back in 1998, when I was rudely introduced an ugly word, OLAP, I misunderstood that the “I” in BI is akin to the “I” in CIA, not the “I” in AI. In the CIA, “intelligence” refers more to gathering data and human analysts making sense of it.

My misconception was that while BI involves data gathering, processing, and analysis, I thought the real goal of BI was on a road to enterprise-level artificial intelligence. In other words, the enterprise itself would have an artificial intelligence – like HAL. After about 25 years of building ETL pipelines, data warehouses, and OLAP cubes, I realized that my blog site’s tagline for 20 years, “Putting the ‘I’ back into BI”, was misguided.

The Intelligence of a Business is a concept I had been working on for at least the last couple of decades. Here are a few very old blogs I’ve written on the theme of the intelligence of a business:

• Bridging Predictive Analytics and Performance Management: Explores the connection between predictive analytics and managing business performance.
• The Magic of the Whole is Greater than the Sum of its Parts: Written during the Holiday Season of 2013, at home in (then) cozy Boise away from the hustle of Manhattan, this post explores the synergy within a business. It is a bit cringey, but it was the Holidays when we’re all cringey.
• The Effect Correlation Score for KPIs: Discusses strategy maps and their importance in business intelligence.

The Book is Applied AI in a BI Environment

As I write this blog, it seems like we’re rolling down into the trough of disillusionment (just after the happy honeymoon) portion of the Gartner Hype cycle. For example, these resources discuss both sides of the argument:

• Goldman Sachs: Gen AI – too much spend, too little benefit
• AI is Not a Bubble – Why Goldman Sachs is wrong

I wrote my book during what could be during the height of the AI hype (around Sep 2023 through around Feb 2024). In 45 years of a software development, I’ve been through many Gartner Hype cycles of many types – beyond just AI Winters and Summers. I must admit that I thought there’s a chance this hype-cycle could be different (i.e. no trough of disillusionment).

My book does hedge on a bet between the of trough of disillusionment versus the hockey stick continuing to grow. As an architect and developer (not an AI researcher), my book is about what we can do with today’s level of AI in the enterprise. What I present doesn’t depend on a continuing hockey stick curve but also provides sanity against a continuing hockey stick curve.

The level of AI (LLMs, in particular) as I write this blog is far from the level required for “the intelligence of a business”. It would be like placing a classroom of teenagers from a gifted class in charge. But there’s a lot you can do with that if implemented in the right way.

But it’s mostly because, “gun to my head”, if I had to make a bet, I’d bet that is AGI is achieved within the next few years. We people must still hold the reigns of our agency and that the distributed network of eight billion sentient people would still a formidable force. I thought the ideas I’d been working on for about 20 years I’ve conveyed in my book is a powerful middle ground.

Original Thinking

An undercurrent theme of my book is building a focus on “original thinking” at a time when we still have the upper hand in that department against AI. Even if/when AI overtakes our collective intellectual capabilities, we’re still the sentient beings we’ve always been, with hopes and dreams, people we love, and a unique prism of experience.

We hear much about “critical thinking”, but original thinking is a level above that in terms of intellectual prowess. Critical thinking is the process of analyzing and evaluating information or arguments in a disciplined and systematic way. It involves logical reasoning, questioning assumptions, and assessing the validity of evidence. Critical thinking is exemplified in scientific methodology using controlled experiments to test hypotheses and analyze results. Or doctors analyzing symptoms, considering potential diagnoses, and determining the best course of treatment.

Original thinking involves generating new ideas, concepts, or solutions that are novel and unique. It is not merely about being creative in an artistic sense but about pioneering new ways of thinking and approaching problems. These include scientific breakthroughs such as the theory of relatively, business innovations such as the assembly line, and paradigm-shifting inventions such as LLMs.

Over the coming decades AI and its accompanying robotics will climb the hill of intelligence towards the top – first taking on the many levels of “automatable” work, then domain experts, the critical thinkers, and finally the original thinkers. As it happened with the machines of the industrial revolution, we’ve built a symbiotic relationship-well, we still need to work out some side-effects. We surrendered physical superiority to machines in terms of strength, speed, accuracy, and endurance. We even surrendered superiority in terms of memory and well-defined tasks. But we still maintain superiority in terms of critical and original thinking.

However, LLMs have breached the borders of our realm of critical thinking. Even though a good chunk of an LLM’s efforts at critical thinking are amusing but potentially dangerous hallucinations, it is still a form of critical thinking.

AI in many forms and compositions (RAG, agentic, etc.) will only make more inroads into these realms and we people need to consistently adjust in order to maintain substantial value in the workplace. Eventually it will be tough to impossible to compete one on one with AI (as happened with chess). When/if the day comes where one on one, each of us can’t compete with AI in terms of critical and original thinking, we can probably hold our superiority longer at a more collective level. That would be at the level of the enterprise.

I’ve read in many places how our natural community (hunter-gatherer days) size is about 150 people (Dunbar’s number). That sounds reasonable given the complexity level of hunter-gatherer life and technology. That’s probably what is needed to build expertise at the known jobs, maintain relationships, and to enough to foster the next generation of skilled workers.

But with the complexity level of our technology today, we need hundreds of millions of people to keep our technology-augmented society humming along. Our enterprises are organized into smaller groups closer to the magnitude of 150 (departments, teams), but the overhead of complex interfacing between the groups is much more than the simpler agreements between a few neighboring tribes.

At the enterprise level, that’s a few hundred to tens of thousands of people. It’s big enough to sustain highly complicated products used in our complex society. But it’s not so big that it’s impossibly unwieldy and most of us stand some chance to be more than a number. At that level, there is enough variety to promote a healthy level of evolution.

Corporations have always been viewed as entities. The unappealing aspect of corporate competition is that sentient humans get caught up in the battlefield, like cuts and broken bones of a battle.

People have pushed back on me when I talk about enterprises as organisms, saying corporations are not like organisms. Corporations are certainly not like people, but they are entities with purpose composed of many moving parts. While they may not look like organic creatures such as people and chickens, corporations are higher-level entities built at the direction and hands of organic beings (mostly, and at least for now). They must compete for resources (sales) to survive. They evolve and eventually even die.

In an ideal world, if a business dies, we human workers don’t die. They live to move to another opportunity, bringing their knowledge and skills to apply to different problems elsewhere. Competing at the corporate level enables vigorous competition without bloodshed being a natural part of the battle.

Although my book doesn’t offer the secrets to magical original thinking potion for us, it stresses the need to honestly live with AI as a strategic asset. That means using it to develop your own secret sauces and even something more profound than a secret sauce. If ever there were a time for all of us strive to be genuinely different against the trend to homogeneity, it’s now. Survival of the cleverest ensures our systems continue to churn in a healthy way, evolving at a steady state, giving everyone a reasonable shot at success.

Conclusion

“The Intelligence of a Business” reflects my journey and evolving understanding of Business Intelligence over the past 25 years, which began with a fortunate misunderstanding of the meaning of the “I in BI”. But this beginning of AI spreading into the psyche of the general public helps to shift my misguided understanding of BI to what I thought it was.

While my book, Enterprise Intelligence, encapsulates these ideas, it also emphasizes the importance of connecting various big concepts at a high level. AI, even at this pre-AGI level, is a transformative force, reshaping our understanding of intelligence and competition. Corporations, much like organisms, must adapt and evolve to survive. As humans, our adaptability and unique perspectives remain invaluable, even as AI continues to advance. Ultimately, the intelligence of a business lies in its ability to learn, adapt, and thrive in an ever-changing landscape.