Multidimensional assessment of R&D performance: Evidence from the pilot evaluation exercise of Russian public research institutions
Research evaluation recently became a widely disseminated exercise aimed in the end of the day at improving the cost efficiency of public funding of national R&D sectors. In November 2013, the Government of the Russian Federation initiated a national evaluation exercise of public research institutions (PRIs) to provide information basis for development of S&T policies aimed at increasing effectiveness and strengthening the role of R&D performing institutions in economic and social development. The aim of this paper is that of providing an approach for multidimensional assessment of R&D performance based on quantitative data derived from the national evaluation exercise, specifically looking at its applicability and limitations for further analysis and preliminary differentiation of PRIs as well as for use in policymaking.
The article contains the results of the cognitive mapping procedure applied to a series of interviews with the reviewers of the Russian Foundation for Basic Research. The procedure can be qualified as a qualitative research method, which allows to produce a graphic representation of the cognitive content of the respondents’ speech. The interviews touched upon the criteria and methods used by the reviewers in evaluating research proposals, as well as the value basis of the Russian scientific community. Cognitive mapping was applied to 15 interviews, which allowed to conclude that the examined group possessed coincident beliefs in regard to the criteria used in the review of proposals, to the separation between basic science and development, and to the ways to improve the current state of science in Russia.
The global economic and political landscape is undergoing profound changes as the world enters a period of rapid transformation development strategies or adjusting their existing ones with greater prominence given to the role of innovation in the leading and underpinning development to strengthen their strategic arrangements for innovation⁃driven development, in a bid to improve their international competitiveness and seize the initiative in global competition Science, technology and innovation (STI) are recognized as the golden key to the door to growth In this trend of the times, the BRICS countries are spearheading the development of developing countries and attracting international attention with their highly innovative and distinctive development strategies Meanwhile, the BRICS as a bloc has become an exemplar of STI cooperation of developing countries.
As the rotating chair of BRICS in 2017, China will host the 9th BRICS Summit in Xiamen in September In the lead⁃up to the summit, the Ministry of Science and Technology of China (MOST) hosted the 5th BRICS Science, Technology and Innovation Ministerial Meeting in Hangzhou in July, where BRICS STI ministers had in⁃depth discussions and reached wide consensus on topics including STI policy, cooperation in priority areas, and co-funding for multilateral research projects The BRICS Action Plan for Innovation Cooperation and the Hangzhou Declaration
To support the work relating to BRICS STI cooperation under the Chinese presidency, China Science and Technology Exchange Center (CSTEC), as entrusted by MOST, established a High Level Expert Group of leading professionals The High⁃level Expert Group complied theBRICS Innovative Competitiveness Report 2017, in collaboration with the science and technology sections of Chinese embassies in other BRICS countries and STI think tanks in other BRICS countries Based on the latest available data, the Report of the BRICS STI cooperation, and presents country and thematic studies on the STI development of BRICS countries.
The Report consists of four parts, with a total of 12 sub⁃reports Part I two general sub⁃reports: an analysis report which evaluates and forecasts the national innovation competitiveness of BRICS countries and their STI cooperation and strategic priorities; and a research report on the priority areas BRICS STI cooperation for win⁃win results This part evaluates the comprehensive national innovative competitiveness of the BRICS countries since 2001 and forecast their innovative competitiveness in the next five years It also assesses the current status and progress of China's STI cooperation with other BRICS countries, and identifies priority areas of BRICS STI cooperation, support for BRICS countries to strengthen their national innovation competitiveness Part Ⅱ presents six country reports, which evaluate, analyze and forecast of the national innovation competitiveness of the BRICS countries and studies of their STI cooperation within the BRICS framework Part Ⅲ presents four thematic reports, which focus on the four thematic areas to STI, including digital economy, inclusive finance, energy, and agriculture, elaborate the STI development and potential of the individual BRICS countries in those areas, and provide valuable inputs for the BRICS countries' national innovation competitiveness Part IV contains appendixes, including an introduction to the related indicator system BRICS STI cooperation.
We use data on economic, management and political science journals to produce quantitative estimates of (in)consistency of evaluations based on seven popular bibliometric indicators (impact factor, 5-year impact factor, immediacy index, article influence score, h-index, SNIP and SJR). We propose a new approach to aggregating journal rankings: since rank aggregation is a multicriteria decision problem, ordinal ranking methods from social choice theory may solve it. We apply either a direct ranking method based on majority rule (the Copeland rule, the Markovian method) or a sorting procedure based on a tournament solution, such as the uncovered set and the minimal externally stable set. We demonstrate that aggregate rankings reduce the number of contradictions and represent the set of single-indicator-based rankings better than any of the seven rankings themselves.
This is the second volume in a series of five books bringing together the results of intensive research on the national systems of innovation (NSI) in the BRICS countries – Brazil, Russia, India, China, and South Africa. This book analyses the co-evolution of inequality and NSI across the BRICS economies. Inequality and Development Challenges argues that inequalities (assets, access to basic services, infrastructure, knowledge, race, gender, ethnicity, and geographic location) that go beyond the aspects of income, must be factored into development strategies since the benefits of innovation are not distributed equally. It combines original and detailed data, making this book an invaluable resource for researchers and scholars in economics, development studies and political science, as well as policymakers and development practitioners interested in the BRICS countries.
This chapter examines bibliometric indicators related to citedness of journals, authors, research groups, institutions and whole countries. The introductory section deals with the basics of bibliometric analysis and features of citation databases. The author discusses the usage of various bibliometric indicators: the impact factor, average citedness, share of uncited papers, Eigenfactor and Article Infl uence Scores, Hirsch and Hirsch-type indices, and others. A special section investigates indicators of chronological distribution of references. Particular attention is paid to normalized indicators, including indicators normalized by research disciplines, as well as by publication sources. The fi nal section emphasizes the importance of informed and reasonable use of bibliometric indicators in research policy-making, funding allocation, and faculty and research personnel recruitment.
This paper discusses the possibilities and limitations of the use of publication databases such as Web of Science and Scopus to determine the research capabilities and prospective areas of research and development of universities. It also analyses major problems related with the analysis of universities’ publication activities in Scopus and Web of Science databases such as author surname variations, identification of author profile among authors with the same surname, author and organization profile merging, identification of author affiliation etc. This paper proposes a list of bibliometric indicators for the analysis of publication activities of individual researchers, university departments and universities as a whole. Furthermore, it describes the methodological approaches for interpreting these indicators. Finally, the paper reviews the possibilities of VOSviewer software for analysis of different aspect of publication activities at individual and department level such as international collaboration networks, detection of the hot topics of research activity and co-citation networks.
We reviewed the output of research and innovation cooperation between Russia and the US, including publications and patents, in the four prospective areas of energy efficiency and renewable energy during 2007-2011. Joint US-Russia research groups appear to focus primarily on hydrogen energy (fuel cells), followed by solar photovoltaics. The upcoming areas of smart grid and biofuels were left out entirely both from research and innovation collaboration. Russian patents in green energy technologies registered in the US are very low in comparison to those from Japan, Korea, and China.
This paper is devoted to the challenges of measuring, analyzing and visualizing research capacity of university. We identify the related methodological issues, propose solutions and apply these solutions to a complex analysis of the research potential of three departments of a Russian university. First, we briefly review the current literature on different aspects of an analysis of research capacity of university. The next step is a discussion on the key challenges faced when analyzing the publication activity of a university. Further, we discuss the opportunities offered by and limitations of using the Web of Science and Scopus databases to determine the research capabilities of universities. In the empirical section of the paper, we analyse the research capacity of university departments and individual employees using simple yet illustrative tools of bibliometric analysis. We also make recommendations for university administrative personnel, which can be derived from our analysis.
This illuminating book combines theory and practice to analyze the experiences and impacts of foresight activities in various European countries. It includes case studies with a focus on different societal issues including national development, science and technology, and sustainable development.