Auditors have taken the European Commission to task over how it has evaluated its technology research funding for the past 12 years.
On Wednesday the European Court of Auditors roundly criticised the Commission about how it has assessed the success of European research and technological development funding programmes since 1995. Currently Commission funding for technology research is running at €7.2bn (£5.2bn) per year, while an extra €9bn (£6.5bn) was allocated to European ICT research in November 2006.
The auditors found there had been no explicit underlying intervention logic, or description of causal links between framework programme activities and expected outcomes, linked to the programmes.
"Poorly defined programme objectives and weak performance measurement undermined effective monitoring and evaluation," said a Court of Auditors statement. "The absence of a comprehensive evaluation strategy, agreed between the research Directorates-General implementing the framework programmes, resulted in inconsistent approaches between the different Commission services."
The Court of Auditors, a panel of financial experts that examines the management and overall efficiency of the European Union's bureaucracy, also castigated the Commission over its assessment of research and technological development during the current phase of funding, which will last until 2013.
"Evaluation of the framework programmes was decentralised, the existing mechanisms for co-ordination among the Directorates-General implementing the framework programmes were not effective, and the Commission's central services, in particular [the] Directorate-General [that controls] budget, had no enforcement role," said the auditors. "Inadequate methodological guidance was provided, evaluators found difficulties in gathering relevant data, and there were no evaluation studies that addressed the longer-term outcomes and impacts of the framework programmes."
As a result of these inadequacies, the Commission evaluations of the schemes were of "limited use to policy-makers", said the auditors, who recommended that framework programme intervention logic be made explicit by legislation surrounding Commission technology funding.
"Underlying assumptions should be explained, the link between scientific and socio-economic objectives clarified and appropriate performance indicators developed," said the auditors.