Analytic Governance

Any major discipline in IT which is considered to be a serious undertaking in an enterprise requires some sub-form of IT governance to regulate its implementation. Different IT sub-disciplines require its own governance: data governance for data architecture, implementation and maintenance; integration governance for middleware architecture and infrastructure; SOA governance for service design, versioning and funding and so on. Analytics is increasingly assuming its own space and place in an enterprise. The discipline of analytics ties in the business and IT organizations very closely; it is imperative that it does. The strategy to invest in the right area of the business where analytical capabilities and insight is required to gain a distinctive advantage should be closely coupled with the data scientists who develop the preditive models and IT system designers and implementers who orchestrate analytics driven business processes and deploy the solutions to production.

The critical significance of the analytics discipline warrants a separate, dedicated and focused analytic governance model or framework. Analytic governance is expected to be in its infancy considering the fact that analytics itself is maturing with time as we see more and more implementations being undertaken. The maturity of analytics governance is expected to closely follow the maturty of analytics itself.

Although nowhere close to any stable state of maturity, the following provides some rationale and areas where analytic governance processes and policies ought to be defined:

• Data Reconciliation - Data sets used by data scientists, to develop the analytical and preditive models, are typically in the form of extracts from the real sources and merges from various sources to get to flattened Excel-like format which is then subsequently used to develop the models. This data may not be the same in the production systems where the models are ultimately deployed. A proper data reconciliation process needs to be established and followed to ensure that the models work on the right type of data so that the correct predictions are generated.
• Model Currency - Based on the latency between model development and its deployment, the data may have changed. As an example, more data types and, or, different data sources may have been introduced. The currency of the model and its applicability at the time of deployment needs to be assessed and hence a well engineered process for the same must be developed.
• Analytics Sandbox - The data scientists require an analytic sandbox where they have the analytical tools and the data required for them to perform data mining and exploratory techniques to identify the right algorithms and models. Such activities often requires data and compute intensive executions. Proper workload must be dedicated to such computations while ensuring that such workloads do not affect the transactional systems. Workload planning, guidelines, infrastructure and best practices must be developed and implemented.
• Business Rules Vitality - The applicabilty of outcomes from predictive models are contingent upon the regulatory requirements, business policies and mandates, etc, which contextualize the correct application of the model outputs in the context of a business process. Business rules are formulated and codified to bring model outputs to the enterprise business processes. Such business rules needs to be revisited for validity and conformity on a periodic basis to ensure that any regulatory changes or internal business policies are appropriately enforced.
• Model deployment - A proper process must be designed and followed for deploying newer versions of models into production. Model versioning will assume significance when multiple versions of various models are put in production to test for best fit. Guidelines on how to reduce the latency between model development and deployment must be developed and commensurate IT infrastructure must be developed to support such capabilities.
• Communication - The use of predictive models to predict the outcome or suggest the Next Best Optimized Action will require a paradigm shift from the traditional human expertise and intuition driven 'sense and respond' mode of business operations to one which adopts a real-time, fact-driven 'predict and act' modus operandi. This cultural shift is going to be the hardest one to address as it requires humans to start thinking and behaving differently - to start relying on system predictions more than their own judgement! Unless, the value of predictive models is not socialized adequately right from the very onset, its value and adoption will pose significant cultural and adaptability challenges. A proper education and communication plan needs to be devised and followed.

I cannot enforce on the point that analytics and analytic governance is still in its infancy and both these disciplines have a long way to go before they can be etched in stone. Nonetheless, my effort, in this post has been to raise the awareness among enterprises that analytic governance is fast becoming a growing imperative.