We need to measure innovation better. Here’s how

Creativity 2019

(Unsplash, 2019)

This article is brought to you thanks to the collaboration of The European Sting with the World Economic Forum.

Author: David Gann, Vice President, Imperial College & Mark Dodgson, Professor of Innovation Studies, University of Queensland


Innovation drives productivity, and productivity is what drives jobs growth and the wealth that pays for our health, education, defence and social security systems. In short, innovation really matters. Governments, businesses and academics obsess over it. We need to know what works, what doesn’t and why – and that means turning to measurements and metrics. But can we really measure innovation today? Are we doing it right, or are our metrics more suited to Industry 2.0 and 3.0, rather than 4.0?

In the era of Industry 4.0, all innovation is combinatorial. Leaders from England’s state-run National Health Service to former Google CEO Eric Schmidt recognise this fact. Almost no innovations can be attributed to a single source.

For example, the iPhone may have been the work of Steve Jobs, Jony Ive and their teams of brilliant engineers and designers, but it was also the product of US government-led innovations such as GPS and, of course, the internet. It would be ludicrous to attribute such innovations as the iPhone to the state alone – but at the same time, there are few truly original ideas. Most successful innovations hinge on the execution rather than the idea itself, which many others may be pursuing simultaneously. The aesthetically similar LG Prada came out shortly before the iPhone, and there were plenty of touchscreen PDA-like phones before the iPhone, but none had the iPhone’s industry-defining impact.

However, the complexity of measuring such innovations should not deter us from trying.

 The combinatorial effects of new technology will drive innovation in new and exciting ways - but how can we measure it?

The combinatorial effects of new technology will drive innovation in new and exciting ways – but how can we measure it?
Image: World Economic Forum / Accenture

Measuring innovation matters…

Good metrics can direct better policies. The OECD’s Programme for International Student Assessment (PISA), which measures school pupils’ performance in mathematics, science and reading, has improved policy debates and led to improved outcomes in some countries. It has driven Singapore’s extraordinary educational success. Statistics on the causes of mortality, meanwhile, have been instrumental in shifting health resources towards early intervention, which has improved life expectancies.

But measuring innovation is troublesome.

…but it’s hard

Indicators have to account for many different forms of innovation, with widely differing motivations, processes of development and consequences. In the past it was possible to identify innovation within particular organizations, teams or individuals; nowadays innovation is more often networked among multiple contributors, which complicates its measurement. Collecting data on innovation is hampered by the desire to ensure indicators are simple, easily accessible, comparable across nations, and cheap to acquire and compute. And these requirements do not reflect the complex and often messy realities of innovation, let alone capture whether the innovation has negative consequences, such as putting CFCs in refrigerators.

The move over the past 60 years from products to services to an increasingly experiential economy has changed the nature of research and development (R&D). Traditional measures of innovation, such as R&D investment and patents, were fine when innovation mostly occurred in large manufacturing firms, but are of limited value when much of the action lies in services, business models, and entrepreneurial start-ups. Much innovation does not rely on traditional R&D investment and processes, and many innovations are not protected by formal intellectual property rights, but by the speed of changes and secrecy around them – and this makes them difficult to measure.

A great deal of expertise has been developed around innovation surveys that ask firms whether they innovate, and in what forms. The EU’s Community Innovation Survey (CIS), for example, has coordinated national statistical agencies to collect extensive data on the innovativeness of EU regions and sectors. But self-reported innovations can be subjective and difficult to calibrate. We know of world-leading innovations in infrastructure and resources industries that are not accounted for; conversely, a firm that was recorded as innovative in a government survey turned out to be a hairdresser using a new brand of hair colouring.

Other judgement calls are needed: are innovations new to the world, new to the sector or region, or just new to the firm? Is adapting an existing product or service to a new market an innovation? While these surveys can tell us about the ITALnumbers and ITALtypes of firms that claim to be innovative, they don’t address the more important question of ITALhow firms are innovating and whether this is improving.

The nature and measurement of capital goods – the assets that are put to productive use – have changed significantly in Industry 4.0. In the past, these consisted mainly of investments in large-scale factories and production machinery, and were carefully recorded for accounting and taxation purposes, providing accurate information for governments. Nowadays, intangible investments and activities such as design are much more significant – but these are more difficult to measure. The new digital technologies of Industry 4.0, including AI and Machine Learning, virtual and augmented reality, simulation and modelling methods, and novel, small-scale ways of manipulating matter such as additive manufacturing and genome editing, for example, complicate the recording and classifications of important innovation investments.

The measurement of innovation is changing

Agencies that collect national and international statistics are highly aware of the shortcomings of their approaches and are seeking more relevant ways of measuring innovation performance. The Australian government, for example, which has been a leader in this field, is undertaking a significant review of indicators, including a comprehensive literature review and public consultations.

Important lessons have been learned in these agencies. The shortcomings of input measures, such as R&D, and averages of innovation performance, such as gross expenditure on R&D as a percentage of GDP, can hide skewed distributions and are not helpful. The opportunities for gaming the system at the national and organizational levels are being recognized. And there is awareness of the dangers when poorly determined indicators become targets for governments.

We welcome the improvement in the collection of such information, but believe this needs to be complemented by some radically different approaches.

New opportunities for measurement and insight

There is value in including insights from behavioural science. Ideas for new services, the largest component of modern economies, occur at the point of consumption. This means the behaviours of consumers and ‘user-innovators’ matter more and more. Furthermore, as AI and automation replace repetitive work, and human attributes such as creativity, intuition and empathy become more important, understanding behaviours around innovation becomes crucial. By changing the behaviours of individuals and populations, behavioural science is starting to be used to address significant global issues such as the environment and health. Governments and organizations wishing to improve innovation performance need increasingly to bring insights from behavioural psychology and economics into their metrics. By applying behavioural science to, for example, the adoption of good innovation practices and better risk management, we may get insights into future performance.

 

Data and digital science provide new opportunities for developing useful innovation metrics. Innovation statistics agencies are exploring the ways in which new sources of data can complement and supplement their work. Analysing social media sites and electronic marketplaces for ideas and employment, such as InnoCentive and LinkedIn, can provide valuable insights. While there are concerns about the self-selecting and potentially unrepresentative nature of the information collected, data-scraping and analytical tools can be used to provide useful new and real-time insights into innovation activity.

Companies such as Digital Science are using AI tools for mapping the trajectories of science, and these could be used for mapping innovation. Scientists have created algorithms and used machine learning to study complex and emerging phenomena such as health and weather forecasting, and these could be repurposed and applied to measuring innovation. Real understanding of innovation requires a deep dive into what goes on within organizations. It has to take into account how risks are assessed, decisions are made and implemented, and how the rocky roads of internal politics and organizational battles over resources are navigated. If government policy is to be informed, quantitative indicators of innovation performance must be complemented with qualitative case studies. For these case studies to be valuable they have to comply with formal research method protocols to ensure they are relevant, accurate and can be compared.

Government innovation policies have to be based upon, and directed towards improving, the performance and practices of the new industrial era. The way innovation occurs is changing – and so the indicators that measure it must respond to this new reality.

the sting Milestone

Featured Stings

Can we feed everyone without unleashing disaster? Read on

These campaigners want to give a quarter of the UK back to nature

How to build a more resilient and inclusive global system

Stopping antimicrobial resistance would cost just USD 2 per person a year

False promises to Small and Medium Enterprises

Mozambique cyclones a ‘wake-up call’ to boost resistance: UN weather agency

African Peace Facility: African Union Peace & Security Operations boosted by an additional €800 million from the European Union

With science ‘held back by a gender gap’, Guterres calls for more empowerment for women and girls

Coronavirus: Commission starts testing interoperability gateway service for national contact tracing and warning apps

UN atomic watchdog chief updates governing body on key North Korean reactor

Why did Cameron gain absolute majority? What will he do now? Will he vote ‘yes’ in Britain’s in – out EU referendum?

Can Eurozone’s uncertain growth answer the challenges that lie ahead?

Bolivia: UN underlines support for ‘credible, transparent and inclusive’ election

10 technology trends to watch in the COVID-19 pandemic

Just transition in EU regions: support to people, economy and environment

UN gears up emergency food aid for hurricane-struck region of Bahamas, as death toll rises

Appalling overall unemployment in Eurozone at 20.6%

Progress on gender equality is too slow, says OECD on International Women’s Day

EuroLat plenary in Panama: control of trade talks and fight against crime

‘Millions facing starvation’ – Global political and business leaders on the economic impact of COVID-19

Teachers launch a free ebook to help children cope with the pandemic

Foreign fighters: ‘One of the most serious dimensions’ in global counter-terrorism struggle

Korea must enhance detection and reinforce sanctions to boost foreign bribery enforcement

World’s 1.8 billion youth must ‘have a say in the future of the planet’

UN condemns deadly simultaneous terrorist attacks on military camps in Mali

Mitigating climate change in Asia-Pacific could give region an economic boost

Monday’s Daily Brief: WFP mulls ‘last resort’ Yemen aid suspension, top peacekeeping awardee announced, abuzz over Bee Day, Ebola threat ‘very high’

Stronger partnerships with post-conflict countries needed to ensure ‘path towards durable peace’: UN chief

One person dies by suicide every 40 seconds: new UN health agency report

UN global counter-terrorism strategy review an ‘overarching vision for the future’: Assembly President

Investment, not debt, can kick-start an entrepreneurial Europe

G20 LIVE: the EU trade gold rush continues as EU and Australia agree to launch Free Trade Agreement (FTA) live from Antalya Turkey

To protect trust in the 4IR, we need to become experts in distrust

COVID-19 has accelerated the digital transformation of higher education

“Smoking steam instead of tobacco, are the E-cigarettes a safer alternative?”

This Norwegian cruise line plans to power its ships with rotting fish

Three ways Finland leads the world – and education isn’t one of them

Security: better access to data for border control and migration management

‘I don’t like to give up’: veteran UN envoy reveals how two decades of quiet diplomacy gave birth to North Macedonia

Car bomb attack on National Police Academy in Colombia, ‘strongly condemned’ by UN

How do you get people to trust self-driving vehicles? This company is giving them ‘virtual eyes’

Family incomes stagnate in the EU; people excluded from ‘moderate recovery’

ECB indicates south Europeans can endure more austerity

The Brexit factor in the US-China trade war and other conflicts

UN recognises role of sport in achieving sustainable development

A Sting Exclusive: “Global Climate: Our Common Responsibility”, S&P MEP Miriam Dalli underlines from Brussels

Have we reached peak smartphone?

UN welcomes Angola’s repeal of anti-gay law, and ban on discrimination based on sexual orientation

Here’s how India can soar in the Fourth Industrial Revolution

Syria: Urgent, concrete actions needed, to protect children too young to ‘make sense of this senseless war’

Sustainable Development Goals: making the world a better place

Burned in the Amazonian forest: Your health may be in danger

DR Congo: Strengthened effort against Ebola is paying off, but insecurity still major constraint – UN health agency

What is the UN General Assembly and what does it do?

5 ways CEOs can renew their businesses for the digital age

MWC 2016 Live: Industrial world prepares to reap digital benefits

The Brussels bureaucracy blocks the Youth Guarantee scheme

The end of Spitzenkandidat: EU leaders concluded unexpectedly on EU top jobs

Why we need a new social contract for data in healthcare

In New Zealand it takes less than a day to start a business

Solutions for cultural understanding: medical students’ perspective

Africa’s Sahel must be a top priority for UN peacebuilding efforts, says commission

Earth Hour 2019: switch off your lights, and become an innovator

VW diesel scandal and climate change: can increased independent car checks lead to cleaner mobility?

More Stings?

Advertising

Speak your Mind Here

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s