Effective e-Government Data at the heart
Written by Pamela Weaver and Janet Pateson
SPECIAL FOCUS: BUSINESS INTELLIGENCE IN GOVERNMENT
With governments around the world spending ever-increasing amounts of taxpayers’ money on large projects, it’s vital to know that should something go wrong, staff can find out about it before the problem escalates.
By keeping data fit and healthy, governments can move to a new level of customer service and efficiency – business intelligence (Bi) can help public servants to move out of a silo mentality and share information in ways that help everyone.
But information on its own doesn’t really do anything. You have to put intelligence to work, act on it. Bi is the process of gathering, organising and distributing data so that the right people get the right facts at the right time, allowing them to make the right decisions.
Failing insight
Gartner’s BI predictions for 2009 focused on the increasing need for BI to deliver business value – although not all organisations will achieve this. Over the next four years through 2012, Gartner predicts that over 35% of the world’s top 5 000 companies “will regularly fail to make insightful decisions about significant changes in their business and markets”. Most organisations, it predicts, “do not have the information, processes and tools needed to make informed, responsive decisions due to underinvestment in information infrastructure and business tools”.
The need to remedy this lack of investment will see business units, rather than IT departments, controlling at least 40% of the total BI budget by 2012. While it’s a good thing for business units to take ownership of their information and analysis needs, there is a risk that independent IT purchases can create silos, making the problem of co-ordinated action even worse. IT departments will need to work closely with business units and maintain tight control of standards to combat this.
Gartner also predicts that, by 2012, one-third of analytic applications applied to business processes will be delivered through “coarse-grained application mashups” of existing operational and analytical applications. Overlaying analytical insights onto operational applications will put information at employees’ fingertips.
In the shorter term, through the rest of 2009 and 2010, BI spending is likely to be one of the few budget areas to survive cutbacks – although doing more with less will be a key theme, according to leading industry watcher Howard Dresner.
Visions of BI
Just as there are many systems under which countries are run, there is varied thinking on BI, which come in the form of three common visions:
• Information monarchy: Only a few people have exclusive access to all the information, drawing control and power to themselves.
• Information communism: The complete opposite of an information monarchy, all information is available to everyone.
• Information democracy: The right information is available to the right people at the right time. Not everyone can access the same data, but each individual is able to access the full complement of what is necessary to them to perform optimally and deliver the very best service possible.
South African users present something of a mixed bag in this department. According to ITWeb’s 2008 BI Survey, of those respondents who suggested that 75% of employees either made or should be making regular use of BI tools, it emerges that fewer than 11% are actually meeting that target. This, against 19% reporting a “should be using” situation.
Those aiming at 5% to 10% usage appear to be hitting the spot with more success: 19% are using, with 7.63% in the “should be” department. Meanwhile, in the information monarchy of the 1%-5% of users, almost 27% are using the tools, against 7.63% who should be.
Interestingly, top management and finance make up the bulk of South African BI user profiles at 61.8% and 67.6% respectively; IT comes in at 61% and you have to drop to below the 50% mark before you’ll find sales and marketing. This drops to 18.7% for research and development.
There is increasingly less of an emphasis being placed on the query/reporting aspects of BI. Cultural change works in tandem with this as we move away from the notion of knowledge as power, through the concept of insight resource and on into the idea of insight as a service.
Transforming public servants
BI is much more than a vehicle for disseminating information; it is a tool for achieving goals, among them measuring decisions, management issues, efficiency, leadership or performance against the overall strategy of the public service. To truly deliver on the goals of effective, citizen-centred e-government, BI should be used to transform public servants from mere receivers/generators of reports and into dynamic units capable of acting on that intelligence to drive new and improved services and initiatives.
THE IMPORTANCE OF DATA QUALITY MANAGEMENT
Strong data quality management is an important component of BI. It involves appointing employees or “stewards” to ensure the consistency and accuracy of data as it flows through their function or department. According to Gartner, these stewards should be subject experts for their departments and act as “trustees of data” (rather than taking ownership of it). Governance duties of stewards should, according to Gartner:
• Ensure the consistency and accuracy of data as it flows from one application to the next. • Implement governance tasks and achieve data quality metrics pertaining to the accuracy and completeness of information in their domain.
• Be responsible for the elements that support data sharing and master data management objectives (such as official product hierarchies, valuation models, customer segmentation profiles and preferred suppliers).
• Support ongoing profiling activities and identify issues with source systems (such as calculation routines and missing values).
• Create or update document taxonomies and actively participate in the semantic reconciliation of data models.
Even once data quality problems have been identified, those tasked with BI implementations continue to experience difficulties. Low-quality data almost invariably bites back to the extent that the enterprise will take the time out to fix it rather than suffer any further losses. Unfortunately, things don’t always run so smoothly in government circles. If it’s not fixed, public information stays broken. This is where Enterprise Data Quality Management (EDQM) or Master Data Management Frameworks (MDMFs) can come in. By forcing a top-down view of critical information, they help generate an understanding of what goes where and how it all fits together before re-engineering legacy data to make it useful in a new system.
A standard information quality programme usually consists of three essential components:
• Measurement: If you don’t measure data quality, you can’t identify what needs improving and, in turn, can’t assess whether you’ve met targets. This is the starting point for any information quality programme.
• Improvement: Identify problem areas and rectify the situation. This is an ongoing process – just once off gets you back where you started. Once you have it clean, keep it clean.
• Verification: Measure the information quality again. Measuring at the beginning and the end facilitates developing a standard.
Ultimately, organisations need to look less at technology and more at people and processes if they want to get to the heart of data quality improvement. Successful data quality management comes from engaging both business users and the IT organisation. After all, if a business
intelligence solution can’t help you make sound decisions about your company’s future – quickly, easily and with confidence, it’s neither good business nor intelligent. Gartner estimates that a 25% improvement in data quality coupled with 75% can add up to 100% business value.
Establishing data quality “hot spots” in areas where it’s most urgently required can get productivity going again rather than bogging an entire government department or function down by trying to fix everything at once. Bad information inhibits successful BI, but user understanding is a significant enabler. Poor decision-making is often a consequence of a lack of user understanding of how information should be applied than of the information itself being worthless.
ENTERPRISE REPORTING
Performed by standard ERP systems, enterprise reporting provides business information to people at all organisational levels, supply chain partners and customers. Most BI tools are able to deliver such reports and the quality has improved to offer summarised analyses, neatly presented and relevant to the user level.
Reporting is and will continue to be a key reason for BI implementations – almost 48% of those responding to ITWeb’s survey cited it as a “primary purpose” of their application – but some observers are voicing concern that users are focusing too closely on generating reports (and the mountain of paper that follows in its wake). There is also the reality that users can’t physically drill into information offered in a report – new questions require new reports... and more paper or time. As many organisations continue to rely on IT to do this, this can be time consuming – as few as 10 variables can translate into more than three million permutations, leaving a lot of unanswered questions.
This situation has seen the development of an interesting squaring off of the traditional online analytical process (OLAP) against the proprietary technology-whose-time-has-come associative query logic (AQL) (see below).
Data mining
This analytic process is geared towards the exploration of data in search of consistent patterns and relationships, which are then validated by applying the patterns to new sets of data. Tools for data mining are based on science like statistics and algorithms, but increasingly make use of technologies such as artificial intelligence, neural networks, decision trees and Bayesian network theory.
Given that prediction is the name of the data mining game, “predictive data mining” is the most common form of it used in business applications. Predictive analysis can provide insight into the sorts of business models that may work to offer insight into customer behaviour. In the past, data mining tools were sold separately, but are increasingly integrated with other BI offerings.
Alert intelligence
|
THE 5 STAGES OF BUSINESS INTELLIGENCE BI consists of five stages: 1. Data sourcing: Extracting information from as many different sources as possible – text documents, images, Web pages, tables, etc. |
Closely aligned to exception reporting, “alert” intelligence can be triggered and distributed to relevant people without the need to query on demand. Many organisations use this form of BI to put out key performance indicators (KPIs). When it comes to analysis, a variety of tools and methodologies facilitate the extraction of intelligent information from the number crunches. Among them are cube analysis, ad hoc query and analysis, and associative query logic.
Cube analysis
Cube analysis takes the humble spreadsheet into a third dimension, hence the name (it is possible to include more than three dimensions, called hypercubes). Instead of information just sitting in a database, it is automatically organised into a cubic “information array”, enabling the slicing and dicing of information and the drilling up, down or across of different data sets. Linked online to the source data, cube analysis is an OLAP, allowing users to quickly identify the root cause of a business problem.
One drawback with OLAP/cube analysis is that data is presented in a pre-aggregated way – the hierarchy for information is pre-programmed upfront, so there is a limit to how many questions can be asked of the information. Inevitably, as a user drills down into the information, questions will arise that couldn’t have been predicted before the predefined parameters were set. Consequently, it can be a very IT-focused process – new questions mean a new hierarchy of information has to be built, along with a new cube. It’s not an ideal situation for business functions to be overly reliant on IT for report generation, however good relationships are.
Relational online analytical processing (ROLAP) doesn’t require the usual pre-computation and storage of information, and is scalable when it comes to handling large volumes of data. Many commentators claim that it is too slow a process, however. Pre-aggregation has, until recently, been a necessity borne of hardware constraints, with disc storage forming a vital component of a system built around slower processors and small, costly memory. The arrangement of data into cubes hurdled some of the limitations of relational databases, chief among them their lack of suitability for handling large amounts of data at any one instant.
Ad hoc query and analysis
A tool for information explorers and power users, this allows a full investigative query into all data as well as automated slice-and-dice OLAP analysis of the entire databases – down to minor details if necessary. There is a trend away from the ad hoc project and into the realm of automated business processes, where incoming data flows can set off predetermined triggers.
Associative query logic
A significant growth area in recent years, AQL was invented in the early 1990s when its developers took a long-term gamble on their belief that not only would processor power increase in a matter of years, but memory would also become cheaper. When hardware constraints became a thing of the past, AQL was ready to take full advantage of it and the QlikView BI solution that uses it has become the fastest growing BI offering in the world, with a 90% CAGR each year for the past four years. It is also the most popular BI solution among respondents to ITWeb’s BI Survey.
Increasingly, it seems that AQL is where the market is heading now that the tools are here. It differs from OLAP in a variety of fundamental ways, most significantly in the fact that it operates in a PC’s memory (RAM). It allows a non-relational, associative database to be built and maintained in a computer’s memory, where it is stored as binary code. This architecture means that source data can be retained and readily accessed offline for analysis. The technology’s capacity to make the most of everything memory has to offer allows on-the-fly analysis in real-time.
Intelligent service delivery
BI supports many of the features of strong e-government: a single-view of the citizen, collaboration, transparency, good governance, streamlining of processes, intelligence-driven strategies, decision-making and cost-effectiveness. BI also makes it easier for governments to see how they’re performing against their own benchmarks; reporting back on programme-specific actions regularly and with relevance helps build a strong foundation on which to build new and improved services.
It seems clear that pressure to streamline and modernise the public sector to drive delivery, efficiency and value for money is finally catching up and governments around the world are turning to BI as one way of helping to achieve this.