Oil and Gas is one of the most data intensive industries I’ve worked with, yet the three amigos of risk – price, delivery and quality – remain present across all industries when thinking about data growth and strategies for data management, a significant issue in the E&P sector. Put simply, data is growing too fast, being held in the wrong place for the type of data it is, and wasting millions of pounds for individual organisations.
Why is this? Well, we are all aware of the demands for growth in data. For example, a subsea video that quadruples each time a new standard comes out (and just as we start to absorb the impact that high definition had with 1080p) now let’s multiply the storage again by 4k. Or, take our offices as an example with their ever expanding team email folders that continue to grow with the desire to always have to hand last month’s out of date advisory on the car park and other information we really don’t need.
This, in some part, begins to explain why the average company in our industry is growing their data “lakes” at 30% per annum! And, that rate is accelerating. The cost is not just in the storage, but in expensive data centre space, wherever you look there is cost risk and uncertainty – when will we need that next SAN upgrade? If the data is unused, not analysed or spread across multiple geographies you have another risk; quality. And if the data is unused and inaccessible you have another risk; delivery – how do you deliver value from data you can’t access and don’t understand?
So what should we do? One of the first things a business can do to help mitigate risk is grade their data – what data is critical and potentially full of untapped value, what is ‘nice to have’ in case you need it one day, and what can be sent into dark storage. Once organisations understand how their data can be tiered, based on its value potential, they can make intelligent decisions about how to store it, where to store it, and how to analyse it. The good news is that there are companies out there that can assess your data and provide you with an automated report – the reaction to these reports is always an eye opener….and then a smile when you realise how much your organisation could shave off its annual IT operations budget.
The fundamental analysis of a data usage report is access to the pie charts. These deliver digestible archive strategies that can be agreed with the business units. Archiving should be considered in the same way as how you keep food – you have a fridge and you have a freezer.
So let’s start with the Freezer: The logistics and economies of long-term archiving of data require and deserve significant consideration. Security is number one and then what data you ship, how much data there is, and how you physically get it to its archival destination. These are key considerations. Amazon Glacier is often thought of as the de-facto ‘cold storage’ archive solution, and Amazon have created a device called ‘Snowball’ to overcome the limitations of transfer over the internet, physically shipping a 50TB device to customers for them to seed data onto to be shipped back for ingest into Glacier. This is forward thinking and useable today for long term “tertiary” copies of data.
But how many companies in our industry are thinking about the short-term archive, their fridge – fresh data. Well I think a lot of senior execs would smile with the potential of this one point: your enterprise live storage costs you about $1 a gig a month – short term archive should save you between a Nickel and Dime a gig a month.
But the three amigos are ever present, so we can nail the cost with this technology, and quality risk is addressed because if you archive to a secure replicated cloud (whether private or hybrid) you’re guaranteed it will be there and that your organisations back-ups will now operate more efficiently -within their back up windows – because the archive has removed excess data. Now the final quality benefit is a bit techy – most archives put the data in an “OBJECT STORE” this means we or your teams can write secure, simple and new ways of accessing your data from an HTTPS layer. This means that, in addition to your existing applications seamlessly accessing the archive when they need it, you now have the ability to gain new insights and efficiencies from your data.
Now, since the Safe Harbour laws (these provided assurance around the transfer of data to US companies) have been ruled invalid by the European Court of Justice, yet another data challenge arises. I wanted to share with you what we are seeing in the bigger picture and that is that some solutions are now proven and available regionally to not only pool data where it legally must stay, but actually to drive down cost and improve secure access. So next time you go to the Freezer don’t forget that, I bet like me, you use fridge more often – in our world that is short-term archive.
Richard is the chief executive of brighsolid. The Warwick Business School graduate has spent 25 years in the IT sector. Richard joined brightsolid full time in 2010. Today brightsolid is a rapidly growing and vibrant cloud provider with new facilities in Aberdeen and Dundee and developments from social media to games.
Recommended for you
