The next generation of data storage and processing technologies will be critical to the success of smart cities, says James Norman.

The development of smart cities is going to become hugely important in the coming years as urbanisation puts increased pressure on the infrastructure and services of cities around the world.

The United Nations predicts that by 2050, around 70 percent of the world’s population will be concentrated in urban areas, compared with 54 percent in 2014. As a result, city authorities will need to take significant steps to address population growth, safety, traffic, pollution, commerce, culture, and economic growth.

Emerging technologies such as the Internet of Things, sensors and analytics, are going to be crucial in helping urban environments and their growing populations develop in a sustainable way, by allocating resources as effectively and efficiently as possible.

It’s already becoming clear that the development of smart cities isn’t just a luxury but something that will be crucial to the viability and success of increased urbanisation. And as smart cities become a reality, it will be important to have technology in place to support the new services and evolving needs.

A Question of Scale

An effective smart city will measure as many aspects of the urban environment as possible, generating a huge volume of data from a plethora of sources as a result.

Typically, a smart city will have sensors on street furniture that measure environmental variables such as temperature, seismic activity, humidity, and pollen and pollution levels.

By measuring these factors, the city’s infrastructure can respond to alleviate the impact of problematic developments, potentially without human intervention. For example, the impact of traffic congestion on pollution levels could be used in a system that directs drivers to alternative routes to lessen the concentration of pollution.

In a typical smart city, it has been estimated that each bus stop and lamp-post will hold an average of eight sensors. With around 250,000 connected objects in a typical urban environment, this adds up to two million sensors, all producing real-time data that needs to be processed and distilled into what is most relevant.

Dubai provides a great illustration of the volume of data that smart cities are expected to produce. As part of government-backed efforts to become a smart city, the Emirate is launching smart initiatives across infrastructure, transport, communications, financial services, urban planning and electricity.

Nearly 1,000 smart services have been mandated to go live by 2017, including the deployment of 200,000 smart meters in 2016, expanding to one million by 2020. The amount of data generated and required by these services is vast.

The sheer capacity of the latest data storage systems when used in conjunction with scale-out solutions such as EMC’s Isilon and Elastic Cloud Storage (ECS) will enable cities to develop a full range of data-driven services to address the challenges presented by increased urbanisation.

But for a city to be truly smart, its systems must be able to access and process vast amounts of data. This is where data lakes will prove crucial, as they aggregate huge volumes of data of different types and formats and ensure it can be readily accessible by advanced analytics tools. When dealing with the volume and range of unstructured data that smart cities will produce, scale-out data lakes provide the best way to make sense of the information, regardless of its form or provenance.

Generating Insight at Speed

Smart cities need to do more than just collect data though — they also need to gain insight from the data generated as quickly as possible in order for infrastructure or services to react.

This need is particularly pertinent if you consider an emergency response to an incident and the number of variables this involved.

Emergency services will need to be alerted and receive information as quickly as possible in order to help people affected and deal with any potential escalation. In addition, traffic will need to be intelligently redirected to avoid the affected area; communications networks will need to be flexible enough to direct capacity to where it’s needed; and information will need to be collated to inform citizens about the incident.

So it’s essential that smart cities make use of cutting edge data processing and analytics capabilities, and in particular, in-memory latency for external storage.

While in-memory processing is expensive and limited by the amount of data that can processed, EMC’s DSSD D5 rack-scale flash appliance can deal with huge datasets in microseconds.

This is thanks to the removal of the protocol changes normally needed when moving data between the storage system and the server. The D5 flash modules attaches directly into the PCI-E bus in the server, avoiding the need to change protocol (such as PCI to SAS or SCSI). Furthermore, DSSD has a new way of accessing data via an Object API which removes additional file system- , volume- and RAID layers in the I/O path.

This approach removes the bottleneck for input/output operations that affects other data processing techniques, and significantly decreases the time needed to process and analyse data. As a result, DSSD provides shared access to 140TB of raw data with in-memory latency, opening up new opportunities that just weren’t possible before.

The Potential of Predictive Analytics

The citizens of smart cities will expect to have access to information and personalized services at all times. An important method for dealing with these ‘digital citizens’ is predictive analytics, which offers the ability to forecast the needs of citizens and allocate resources appropriately through the interrogation of data lakes.

In a medical emergency for example, predictive analytics could direct individuals to a hospital with capacity at that moment in time to deal with their medical issue as quickly as possible, potentially reducing waiting times.

Traffic optimisation is another good use case. Authorities could predict traffic flow and the impact of a certain actions, such as schools opening or a large sporting event. This information could then be used to raise the cost of parking to deter people from travelling to the area in question.

Predictive analytics require data to be collected over a period of several years in order to gain the necessary depth of insight – another reason why data lakes and scale-out storage will be important elements of the smart city infrastructure.

Uniting Data Capabilities

The scale, complexity and range of demands that will be placed on urban infrastructure and services in the near future are huge.

They will require a robust, agile, scalable and secure ICT foundation. And the next generation of data storage and processing technologies will be critical.

Without doubt, the latest developments in highly-scalable, low latency data systems with advanced analytics capabilities support the fundamental requirements of smart cities. And when these capabilities are brought together as part of a platform, the creation of smart cities can really accelerate.

James Norman is public sector CIO at supplier EMC