Managing utility data: beware the all conquering ‘benchmark test’


David Socha,
Utilities Practice
Leader, Teradata
By David Socha, Utilities Practice Leader, Teradata EMEA

Managing, then actually exploiting the massive amounts of new data that the smart world will deliver to the average utility company is a huge challenge. Smart meters and the smart grid will undoubtedly mean exponential increases in the data that most utilities must deal with.  

Fortunately, it seems a number of vendors are rising to that challenge. New consortia of businesses familiar to utilities are arising almost every week, it seems.  Each provides a potential end-to-end solution, promising to cope with, and deliver value from, smart data. Recently, quotes such as “460 billion records in less than a second” or, in another case, “scalability up to 100 million meters” seem to suggest that any issues with managing these potentially overwhelming data volumes have been solved. Utilities can rest easy, assured that the vendors they have done business with over many years have things covered. Sure, it’ll probably mean a few more licences and a project or two. But they’ve solved the data problems we’re all so worried about. And they’re saying so in print. So all is well… Isn’t it?

Before we come to that conclusion, here are two things to consider. First, are those benchmark results actually applicable to your situation? And second, is internal, self-managed benchmarking all these vendors have got? In other words, have they dealt with big data anywhere else before?

Let’s take these in turn. What about the applicability of impressive benchmark numbers to your business? On the surface, they’re comforting. But are they measuring the right things – the things that will genuinely make a difference to how you operate? Or are they measuring the things that make the vendor look really good? For example “speed” in query response can be achieved by knowing in advance what the question is, then tuning the data structure to answer that particular question very fast. But this is often delivered at a cost, so that when other queries or analytics are run they perform very slowly. So, how many criteria in the benchmarking setup are compatible to your situation? If it’s only a few, does that still mean you will get the same sort of results with your requirements, technical landscape, or business constraints? I wouldn’t like to take that bet.  

Then what about that second point – is this the first time these vendors have come across big data? Publicising anonymous, internal benchmarking results, rather than real, named customer cases would seem to suggest so. Wouldn’t it be better to also talk to someone who has implemented big data solutions before? Yes, big data is new to utilities, so no vendor can name more than a few utilities they have worked with on big data projects. But multinational telcos, banks, retailers, social media providers and the like have been dealing with massive and complex data sets for many years. Might it not be worth considering and learning from how these businesses have solved their data challenges? And talking to the vendors that they trust?  

In conclusion, it’s probably a good thing that the vendors familiar to the utilities industry are taking big data challenges seriously. But it’s a risky step to take everything you read at face value. When you talk to these vendors (and talk to them you will), make sure you dig deep below the surface of impressive benchmarks. Find out how applicable the grand claims really are to your situation. And at the same time, think about vendors you might not have relationships with yet, but have that all important experience of big data, with the real, named customers to back them up.

David Socha is one of the expert speakers on data analytics at Smart Utilities Russia 2011 in Moscow, Russia, November 16-17, 2011.

David Socha is a utilities business and IT professional and Fellow of the British Computer Society. He began his career as an electricity distribution engineer on the HV network in central Scotland, subsequently moving through a number of roles in the U.K. electricity deregulation program and then into IT strategy and management. In more recent times, he has been a principal consultant and business development director with a leading global systems integrator/consultancy. He has a BEng from Heriot-Watt University in Edinburgh and an MSc from the Robert Gordon University in Aberdeen, Scotland.