The Open Data Institute (ODI) recently published a report asking ‘who owns our data infrastructure?’, and invited others to blog their thoughts and feedback on the subject. As this is a subject very close to our heart, we thought we would respond.
As the report outlines, there is huge potential and value in the numerous and varied data assets created by organisations in both the public and private sector, which if exposed responsibly, could have a huge positive impact on society and commerce in a wide variety of ways.
Emu Analytics is completely aligned with this message. After all, it is the primary purpose for which we formed our business… in order to responsibly utilise organisations’ data assets for innovative and meaningful use cases.
The need for responsibility, openness and transparency is further referenced in the ODI report, and should not be underestimated, particularly when the organisation is public facing and where the data underpinning an open data initiative is derived from the general public or customer. We wrote about this in one of our earlier blogs, ‘A Word on Privacy’.
Within their report the ODI references six expectations that they would have for organisations contributing to an Open Data society:
- Long-term sustainability
- A perceived authority
- Transparency
- Openness
- Commitment to availability
- Agility
Whilst agreeing with these fully, there is a significant consideration that often underpins all of these points, that is possibly not given the same recognition in the report. Meeting expectations for the provision of data can be a complex and costly task for many organisations. Whilst data technologies have improved significantly in recent years, so has the volume of data being generated, as has the appreciation of data privacy and security. As such the costs associated with acquiring, processing, storing and exposing data should not be underestimated.
The ODI report gives an example of applying aggregated mobile operator data to show how crowd movement can aid retailers, emergency services, charities and numerous other sectors. Again, this is an area close to our heart, but it is worth considering how financially challenging this can be, even to a mobile operator.
UK mobile operators have circa 20 to 30 million subscribers each. Their subscribers’ devices are generally on and connected to their networks 24 hours a day, 365 days a year. The volume of source data that this type of service generates as ‘exhaust’ is monstrous. Even if elements of it are already stored for billing or service management purposes, it is unlikely to be in a form that would be suitable for external exposure.
To enable a secure, consumable data stream a mobile operator must apply the appropriate anonymisation, filtering, data cleansing and aggregation rules along with offering transparency and openness to their customer base. At scale this undoubtedly incurs a huge cost burden.
The most obvious way to offset this cost is for a private organisation to commercialise this data or insight as suggested in the ODI article. In doing so, the organisation is accepting that this may impact their ability to truly offer ‘open’ data from the outset, especially as they may be in a competitive situation with their peers in this sector. Additionally at present this is not a fully-matured industry sector, so often a ‘leap of faith’ is required to commit to and invest in infrastructure when the return is unquantified. The return on investment might take some considerable time to realise, which itself is difficult for many private sector organisations.
Dr Ed Manley is a lecturer at the Centre for Advanced Spatial Analysis at University College London. He adds to this notion, “Infrastructure funding is clearly a major determinant in assuring sustainable open data feeds. While the argument for public sector investment in these resources is clear, it is harder to convince shareholders of the same moral imperative. There are, however, significant advantages for businesses looking to make the investment and open up their data. “
The private sector also has the additional challenge of not wanting to upset and alienate their customer base. They are probably aware that whatever their intentions in this area, it only takes one biased newspaper article on the use of data in a seemingly intrusive manner, to result in significantly negative PR consequences.
Articles such as this recent one on the BBC Website highlight the difficulties in promoting the use of data responsibly and trying to cope with potential scaremongering versus legitimate risk.
However as Dr Ed Manley suggests, engagement can potentially mitigate this risk too: “Businesses engaging with open data place themselves at the centre of a potential ecosystem of new services and business partners. This not only expands their influence to new, unforeseen sectors, but builds public engagement with and interest in the data, and with the business also. These services can be commercialised by tiered access, where higher demand users pay for access, or through provision of specialist consultancy services.
Twitter and Google are both excellent examples of companies who have used open access to their data and services to build an ecosystem of integrated businesses. Both companies retain a fee-paying option for higher bandwidth users, but their open datasets have enabled researchers and developers alike to create engagement in their datasets, demonstrating and strengthening their value. A commitment to open data does require investment, but taking the ‘leap of faith’ can lead to significant and sustaining long-term benefit.”
In the public sector, provision of Open Data could be mandated, but still it can require a significant investment to avoid missing expectations on ‘sustainability’, ‘authority’ and ‘availability’. Few would subscribe to a dataset that they had little confidence in or that they felt would not be readily available on an on-going basis. Ultimately, even public sector data sources are going to need infrastructure funding that has to be found from somewhere.
We at Emu Analytics hold the belief that data from organisations can be used for new, innovative, valuable and beneficial purposes and are very excited to be working in this sector. However we feel that no-one should underestimate the financial challenge that often underpins even the most cited examples, and as such some of these strategies and objectives from the ODI may take some time to come to fruition. There is a responsibility on the whole data and analytics community to tangibly demonstrate the value of open data, both commercially and socially, in order for these organisations to be able to make and justify the investment decisions.
Credits:
Dr Ed Manley is a lecturer at the Centre for Advanced Spatial Analysis at University College London. His blog on Urban Analytics can be found here: http://urbanmovements.co.uk/