It might be hard to believe – when you're waiting for a bus that finally arrives so full, you'd have been better off walking – that we are any closer to the smart cities that have been pitched to us for years.
The daily commutes in the world's largest cities easily run into millions, and managing those travel flows remains a major task for the authorities as they work towards smarter networks.
Transport for Greater Manchester (TfGM), for example, recently announced that it is working with data-warehouse specialist Snowflake to try and coordinate public transportation in the city to meet the needs and schedules of travellers.
Snowflake lets TfGM use a cloud-built data platform that aggregates information about the 5.6 million journeys made across the region's transport network every day. The data warehouse service also manages the data so that city planners can access and interpret it more easily.
SEE: Cloud v. data center decision (ZDNet special report) | Download the report as a PDF (TechRepublic)
It may sound like a simple idea, but it is a big step, especially given how TfGM used to process data before it adopted the platform.
Malcolm Lowe, head of IT at TfGM, said the authority always had lots of data but accessed it using techniques that ranged from spreadsheets to modelling tools.
"It was all very siloed and it took a lot of manual processing, and a lot of time, to get any value from it," he told ZDNet.
"With Snowflake's platform, we can access the data and carry out analyses that weren't possible before. We can find out things we never thought we could know."
More specifically, this means TfGM can establish exactly, and in real time, how many people are travelling on the transport network, in which stations, and plan accordingly to have enough trams and staff in the right place, at the right moment. In theory, this advance should limit the number of crammed carriages and angry commuters.
Last summer, the city introduced a contactless system on the Metrolink light rail service – an additional wealth of data, which enables Snowflake to gather extra insights by also tracking travellers as they tap in and out of the transport system.
Lowe explained that since then, TfGM has identified exactly when the rush hour peaks in the morning and in the evenings – which before was only estimated using anecdotal evidence. In an effort to spread the peaks, the organisation is launching a "contactless early bird" initiative, which offers off-peak prices to commuters if they tap in outside the hours identified by Snowflake.
Lowe said Snowflake's technology will allow for immediate feedback on projects like contactless early bird. "We'll be able to understand how the initiative has an impact on the network, and from there make evidence-based decisions, rather than perception-based ones," he said.
It is only the start of Greater Manchester's switch to smarter transportation, partly driven by public demand, said Lowe.
In London, planners have been using big data for a number of years already, based on contactless card payments, Oyster card tapping, as well as Apple Pay, Android Pay, and even cameras set up around the network to keep an eye on traffic flows. Every week, the city's transport organisation, Transport for London (TfL), collects 19 million smart-card ticketing transactions.
Last summer, TfL fitted 97% of the city's underground stations with Wi-Fi, and started collecting data based on travellers' phones connecting to the network.
As useful as ticketing data can be, said the organisation, it can only tell where customers enter and exit London's underground services. Wi-Fi connections, on the other hand, reflect which platforms and lines customers are using, as well as how they navigate around stations.
The data can then be used to manage disruptions more effectively, and provide users with better information to plan their journeys and avoid congestion. For instance, knowing the walking time from one platform to another, and how busy the different paths are, could come in useful for a passenger travelling with luggage.
TfL is even hoping that in the future, it can provide users with an online map reflecting the real-time business of the city's stations.
London's transport organisation is also making a point of sharing data in an open format for anyone to use. The goal is to encourage software engineers to develop new products, and particularly improved app-based services.
Lowe said in Manchester sharing data is something that TfGM is thinking about. "We are looking at democratising data," he said.
"We could potentially open it up to CityMapper or Google Maps, for example. We should think about other functions it could serve, like embedding ticket purchase in the application. But these are still early thoughts."
The idea of open and democratic data is also likely to be welcome among passengers who worry about transparency. As with many things smart, improving transportation doesn't come without tracking customers – and although data is anonymised, the increased level of surveillance is sure to make some commuters uncomfortable.
Francesca Bria, the chief of technology and digital information for Barcelona City Council, identified the need for a "citizen-owned data ecosystem" a few years ago. The Catalan city has been using data-driven technologies to improve the everyday lives of its inhabitants since 2011.
City planners have developed, for example, a more efficient orthogonal grid scheme for the bus network, fitted with bus stops that allow connections with tram and metro lines. With the new routes, there should never be more than one change needed for a passenger to link any two points in the city.
SEE: Self-driving cars: The hunt for security flaws steps up a gear
However, Bria believes that a smart city has to be achieved hand in hand with the citizen, in what she calls e-democracy. It starts with making sure smart city dwellers know exactly what data is being collected about them.
Barcelona is, in fact, one of the four pilot cities for a European project called DECODE (Decentralised Citizen Owned Data Ecosystems), an initiative that attempts to implement practical solutions to let users manage their data more easily.
Among other things, the city has launched an online dashboard that lets citizens visualise the information held about them, find out with whom it is shared and under what conditions.
For Bria, the future smart city is one that protects citizens' technological sovereignty, which she described as "citizens' capacity to participate in how the technological infrastructure around them operates and what ends it serves".
So, the concept linked to smarter bus routes and manageable crowds in the underground and it is one that city planners should keep in mind each time they deploy data-driven tools, even if it is to make the lives of commuters easier.