This is the time of the year when forecasters tap their crystal balls to tell us which technologies will shine and which will fade. Given that big data became so prominent in our lives in 2013, it’s no wonder it is playing a big role in these predictions. But did anyone predict last year that big data (and metadata, and Hadoop and so on) will be so frequently discussed in mainstream, mass-audience publications in 2013? And that this sudden interest will be instigated by one man, Edward Snowden?
No one predicted Snowden because he could not be predicted. Even general trends, rather than specific events, are subject to changes that are impossible to foresee. Most predictions are what the forecasters want the future to be or simply an extension of what they are familiar and comfortable with. It may well be that the only future that we can have some degree of confidence in is the one we create ourselves.
This, of course, is what defines entrepreneurs. They typically have a somewhat higher degree of confidence in themselves—and the future they are creating—than people that are not so willing to take risks and try to make an idea succeed. While mostly associated with starting a new business, the spirit of entrepreneurship is behind new ventures everywhere. It can happen in government, in the non-profit sector, in healthcare, or in established businesses. It can also shape how people define and carry out their organizational roles.
Which brings me to the one and only valid prediction for 2014: CIOs could create their own future by leading the big data initiatives of their organizations.
In a recent webcast discussing its 2014 predictions for CIOs, IDC declared that this is “a wonderful time and a scary time to be a CIO.” This existential duality was evident also in IDC’s top two predictions, expecting CIOs to become innovators and, at the same time, denying for most of them a significant role in the adoption of big data. In the next two years, IDC predicted, “over 70% of CIOs will change their primary role from directly managing IT to become an innovation partner who delivers information insights and value-added services.” IDC also predicted, however, that “before 2017, only 40% of CIOs will rise to the challenge from CxOs to partner in strategic planning by producing business-enhancing insights from big data and analytics.”
I would argue that the only way the CIO can become “an innovation partner” is exactly by “producing business-enhancing insights” from big data. The mining of internal and external data to find better ways to operate and advance the business or to develop new products and services is what big data is all about. Big data is the definition of IT-driven innovation and the result of the confluence of three important IT trends: cloud computing, open software, and new data-manipulating skills.
These trends are usually perceived as threats to CIOs. The availability of cloud-based platforms provides an easy-to-use and cost-effective sandbox for business executives’ experiments, leading to more “shadow IT.” Open source technologies require an overhaul of traditional IT practices and processes and demand skills and expertise that are often lacking in the IT organization. And as data scientists are often hired and trained somewhere else in the organization, the IT organization may end up relegated to only a data maintenance role, leaving the extraction of value from the data to others in the organization.
Instead, CIOs could embrace, standardize, and lead the use of cloud computing.
CIOs can also lead by inventing and implementing new processes for the IT organization that accommodate open source software and rapid change while providing high-quality, continuous, and stable support. CIOs can use big data as an opportunity to update the traditional role of IT to include discovery and agility, and add a new functionality and service—data science, the uncovering of important new insights—to what they offer to the organization.
Instead of fighting chaos by resisting change, CIOs could create order out of chaos, delivering stability and creativity, the two hallmarks of innovation. As Dell’s CIO, Andi Karaboutis, told the CIO Journal: “I don’t chase shadow IT, I’m chasing innovation. It’s how our people do work.” At the time when innovation is based first and foremost on new uses of data and new insights derived from data, big data provides CIOs with a unique opportunity to lead the digital transformation of the business.