Establishing a dedicated analytic environment with the infrastructure, technologies and data required to produce insights that change the business is one of today’s hottest topics. At WebbMason Analytics, we call this environment the Analytic Platform. Often times, it is not just a conversation or decision about analytics, but also incorporates strategic company initiatives including:
- Assessment and migration to the cloud
- Access to data and security
- Data governance decisions
- The introduction of data science
- The formation of an Analytics organization
Together, all of these topics can create significant complexity when deciding on the best solution to serve as your analytic platform. While understanding the nuances of individual technology or infrastructure decisions alone can be challenging, addressing organizational changes and decisions is often a more significant undertaking.
Over the years, having worked with a number of organizations establish an analytics platform, one particular conversation seems very common: Who owns the analytic platform? Should it be IT, the new analytics group, or a hybrid approach? Addressing this question early in the conversation allows for clear boundaries to be established, which, in my opinion, improves collaboration, streamlines initial investment decisions, and significantly decreases the time required to design the analytic platform.
The two most common justifications I hear as to why IT or analytics should own the platform include:
- IT has a tremendous amount of expertise, but are too slow in adapting to changing business needs (a point of view often coming from an analytics group)
- Analytics, while great at adapting to meet new business challenges, does not have the IT expertise to manage a technology platform (a point of view often coming from the IT group)
One thing I have noticed working with analytics organizations is how tightly integrated data, technology, and analytics has become, especially in the world of cloud computing and big data. For example, if trying to run a machine-learning algorithm against a large data set requires infrastructure tuning (and with the cloud making this quick and easy) to support the memory-intensive workload, having one person that can do it all is ideal. However, in a world where analytics skills are in short supply, this is not a scalable model. What is required is a process that brings the required skills into a close collaboration in order to produce results efficiently.
So, at the end of the day, who owns what? Let’s explore a few different models and weigh the pros and cons of each.
IT owns the Analytic Platform
In this scenario, IT would own the design, implementation, administration, and support of the analytic platform. This closely mirrors how many organizations have structured traditional business systems. The analytics group would be end consumers of the platform. Many would argue that this scenario would bring the analytic platform into the traditional IT arena, ensuring that previously established processes and best-practices pertaining to IT systems are applied to the analytic platform. It would also allow for existing employees to take on the burden of managing the analytic platform and potentially reduce the need to hire new staff. A few key considerations when evaluating this scenario include:
- Do traditional IT processes and best-practices apply to the analytic platform? Many processes and best-practices were developed before the introduction of cloud computing, especially prior to Platform-as-a-Service (PaaS) offerings. Would IT impose processes that limit the adoption of these new technologies, many of which are good solutions for analytics (and necessary to manage costs effectively)?
- Will IT develop a dedicated team to support the analytic platform, or will platform requests be merged in with other IT requests?
- Does IT have the existing staff to support the analytic platform, or is new staff required regardless of who owns the platform?
- Is IT capable of supporting exploratory data analysis, in which specific requirements are not known up front? How does analytic development align with a traditional application or system development? Does IT have the incentive to adapt to new methods?
- Is the current IT team agile? Will they modify traditional waterfall approaches to accommodate the agile nature of analytics?
- How will the IT personnel interact with data analysts and data engineers in the analytics group? How will they adjust to collaborating with a highly technical user group vs. supporting a non-technical group?
There are potentially many advantages to having IT own the Analytic Platform. The question every analytic leader should ask is:
“If IT owns the analytic platform, how do we prevent the platform from becoming just another business system suffering from the traditional pain points between IT and the business.”
Analytics owns the Analytic Platform
In this scenario, the analytics group owns all aspects of the Analytic Platform, including design, implementation, administration, and support. While the analytics group may work with IT during the data ingestion process for legacy business systems, the bulk of the responsibility to develop analytic capabilities falls to the analytic organization.
This approach could have many benefits, including approaching the analytic platform with an open mind, willing to explore new technologies and processes, and not becoming burdened by the saying “This is how we have always done it”. However, this scenario requires the analytics group to really step up their game. This includes hiring, training, and organizational structure that allows the analytics group to not only produce analytic capabilities but to also operate like a traditional IT engineering group. This can be intimidating for some analytic leaders, especially those that do not come from an IT background or have significant experience working in the “IT weeds” during analytic projects.
A few key considerations when evaluating this scenario include:
- Is the analytic organization ready, or even capable, of managing IT personnel?
- Will the analytics group define and follow IT best practices to ensure the platform is reliable, scalable, and secure?
- Is additional staff required to manage the platform? If so, what additional costs would be incurred?
- How will the analytics group provide ongoing support and maintenance of the analytic platform?
- Does this shift too much of the focus of the analytics group away from generating data-driven insights to IT support?
There are potentially many advantages to having analytics own the Analytic Platform. The question every analytic leader should ask is:
“Are we capable of maturing quickly enough to handle this burden?”
Last, we visit the scenario which certain organizations will likely adopt – the hybrid approach. In this scenario, IT supports the administration and maintenance of the analytic platform, analytics has the necessary privileges and transparency to remain productive, and the two groups collaborate closely in a number of areas, including design, implementation, governance, and extensibility. We often see significant challenges in organizations adopting this approach including:
- IT is required to perform a task but does not receive the budget to hire additional personnel to support a dedicated team
- IT does not have the skills or motivation to understand the needs of analytics and the platform is not matured or managed to support efficient analytics development
- Granular roles and responsibilities are not defined, creating confusion, siloes and single decision makers
- The Analytics Platform never gets past the design phase without clear, senior leadership capable of accounting for the present day challenges while also considering the future state
While this approach is often the most likely scenario, a few key considerations include:
- Who has the final decision in the likely debates between IT and Analytics?
- Are clear SLAs and processes defined to support collaboration and ensure accountability?
- How do you provide flexibility while still supporting security? If you need an example to test this question, ask “who can install Python packages?”
- Does IT have accountability and incentives to facilitate delivery of data-driven capabilities? Does analytics have accountability and incentives to ensure a stable and secure environment?
- Is the team co-located, either physically or virtually, or do traditional “fences” still exist?
There are advantages to having a hybrid model. The question every analytic leader should ask is:
“Are we ready to tackle the organizational or cultural challenges that have traditionally created friction between IT and business stakeholder? Can we resolve these challenges quickly enough to stay competitive in the market?”
One thing I have learned is no one size fits all. Groups adopt one of these three scenarios, or combinations, all the time. The most important thing when determining who owns the analytic platform is to recognize the strengthens and weaknesses of each scenario and to develop solutions to address each weakness.