Index – Prometheus Assessed?

Index

A

A journals, 62
‘A’ status, 199
academic impact, 300
academic research, 149
academics, 110, 112, 153–4
Academie Royale des Science, 48
Accademia del Cimento, 48
accountability, 2
achievement, 52
Administrative Reform Council (ARC), 235
agencification, 235–6
allocational mechanism, 97–8
American Political Science Review, 65, 113
American Psychological Association, 92
Anamata, 220
Annals of Physics and Classical and Quantum Gravity, 107
anonymity, 99–103
policy, 102
rejection decisions, 102
article acceptance, 111–12
author influence and organisational reputation, 104
article citation, 67
article publication, 56, 211, 217
article rejection, 111–12
assessment implementation, 249–52
assessors assessment, 205–9
limitations, 208–9
Association of University Staff, 220
attribution problem, 262–3
Australia, 58–9, 62
Australian Political Studies Association, 62
Australian Research Council, 60
author reputation, 104

B

‘B’ category, 225
B journals, 62
Baldi, S., 72
bandwagoners, 112
‘barbarians at the gates’ 112
Basic Law of Independent Administrative Institutions, 235
bias, 110–13
bibliometric assessment, 281–5, 298–9
bibliometric study, 192
bibliometrics, 3–4, 158–62
research assessment, 45–83
Biological Sciences Panel, 213
panel member article publications, 215
Biomed Direct, 119
blind reviewing vs. non-blind, 102, 103, 118
book editing, 198
book manuscripts, 86, 100
reference, 197
book publication, 54–7
books, 86
bureaucratic control, 295–8
‘bureaucratic personalities’ 296
Business and Economics Panel, 216–17
panel member article publications, 217
business journals, 222
Business Process Re-engineering, 306

C

C journals, 62
Cambridge, 125
Campanario, J. M., 111
Canadian Medical Association Journal, 114
candidate reputation, 194
‘census date’ 139
‘certified evaluation and accreditation’ system, 237
citable items, 80
citation, 162
limitations, 70–82
role, 68–70
citation analysis, 3, 5, 7, 39, 66–70, 281–5, 294–5
limitation, 81–2
research assessment, 45–83
types, 67
citation behaviour, 75, 78, 79
citation models, 6–7
Columbia University, 276
commercial interests, 113–15
commercial publishers, 59
commercial success, 197
Committee for Policy Evaluation and Evaluating the Independent Administrative Institutions, 244
commodification, 153
Community College Research Centre, 276
competitive higher education model, 173
conference papers, 59–60
conformity, 110–13
conservatism, 110–13
Conservative Government, 125
Conservatives, 307
constructivism, 33
contribution to research environment (CRE), 195–8
important assessment factors, 196
corruption, 287
cost, 144–6
cost centres, 132
Council for Higher Education Accreditation, 278
Council for University Chartering and School Corporation, 234
Council of Public Finance, 263
creationism, 96
Creative and Performing Arts Panel, 206
Critical Success Factors, 277
critical work, 32
critique/criticism, 26
cronyism, 172
Crown Research Institutes, 77
‘dangerous enthusiasms’ 292-3
Database of Accredited Postsecondary Institutions and Programs, 278
Dearing Report, 126
decentralised models, 5–6
delusional paradigms, 71
Democratic Party Japan (DPJ), 266
disciplinary device, 98–9
disciplinary ‘gatekeepers’ 112
disciplinary mechanism, 93
disciplinary power, 294–5
‘distinguished performance’ 249
double-blind 90, 100, 102–3
‘double-weighting’ 162
Durham, 125

E

‘economic and societal’ impacts, 300
edited books, 55
editorial comments, 86
Education Act (1988), 126
Education Act (1989), 52
Education Council, 230
education discipline, 217–18
Education Panel, 217–18
panel member article publications, 218
education system, 51
Einsteinian physics, 17
emancipatory interest, 32
empirical work, 33
Engineering Technology and Architecture Panel, 209–10
panel member article publications, 209
error detection, 107–10
Evaluation Committee for Teaching and Research of National Universities, 247
Excellence in Research Australia, 4, 45, 54, 60–1, 78, 82
Excellence in Research for Australia Initiative, 60

F

fallibilism, 10–14
falsifiability, 10, 12–13
Feyerabend, P., 21–3
Financial Times, 52
Foucault, M., 23–6
Foucauldian, 24–5
Foucauldian sense, 294
fraud
detection, 107–10
misbehaviour, 109
Freudian psychoanalysis, 10–11
Fundamental Code of Education, 233
fundamentalism, 71
funding
allocation, 142–4
rating point, 142
Further and Higher Education Act, 126

G

gestalts, 16
Gibson, J., 224–5
Google Scholar, 46, 70, 78, 223, 283–4, 285
government research, 149
governmentality, 23
grey literature, 57, 162
group think, 292
Guilt culture, 88

H

H-index, 68
halo effect, 76
Health Panel, 210–11
panel member article publications, 210
hermeneutics, 31–3
Hicks, D., 278–9
Higher Education Funding Council for England (HEFCE), 38, 141, 143, 145, 147, 149, 160, 164, 299–302
Higher Education Funding Councils, 127
higher education institutions (HEIs), 142, 145–6
higher education policy, 173
higher education system, 125
‘historical’ impact, 301
history, 212
‘holistic’ judgement, 200–1
House of Commons, 141, 144, 149, 150
humanities, 26–30
database, 212
Humanities and Law Panel, 211–12
panel member article publications, 211
Hwang, W. S., 108
hyper-positivist models, 31
hypothesis testing, 191

I

impact factor, 61, 65, 208–9, 252
limitations, 79–81
impact indicator, 163
impact measure, 304, 306
impact statement, 163
Independent Administrative Institutions (IAIs), 235
individual research, 226
international excellence, 141
international journal publication, 206
international networking, 195, 198
International Psychoanalytic Association, 306
ISI citation indexes, 61, 78, 80
ISI database, 207

J

Japan, 87–8
Japan Institution for Higher Education Evaluation, 237
Japan research evaluation, 313–14
analysis and hypotheses, 252–9
data and methods, 255–9
distinguished performance character, 251
distinguished research performance, 250
mid-term goals and plan samples, 240–1
National University Corporations, 229–67
NUC evaluation research achievement, 248
NUC research level evaluations, 249
resources for academic activities, 262
self-evaluation vs NIAD-UE’s evaluation, 259
teaching and research achievements and levels, 256
Japan University Accreditation Association, 237
Japanese government, 95
Japanese university system, 233–6
agencification, 235–6
Jarret Report, 125–6
journal, 86
Journal of Accounting and Economics, 50
Journal of Ayn Rand Studies, 97
Journal of Business Ethics, 65
journal rankings, 3, 7, 207–8
impact factor and professional association listings, 61–6
journeymen, 110

K

Keynes, J. M., 54
knowledge, 23–4
knowledge creation, 2
Kuhn, T., 16–20, 283, 291

L

Labour Coalition Government, 173
Lacan, J., 306
Lakatos, I., 12, 14–16, 37
law journals, 212
‘level’ evaluation, 260
Liberals, 307
Likert scale, 185, 187, 255
Linnean Society, 48
Llewellyn, C., 109
‘lower quality’ research, 303

M

main panel, 136, 138, 139–40, 141–2
management development fads, 306
Maori Knowledge and Development Panel, 206, 213
market-favouring, 156
Marxism, 15
massification, 234
Mathematical and Information Sciences and Technology Panel, 213
panel member article publications, 214
media commentary, 197
Medicine and Public Health Panel, 213
panel member article publications, 214
Mendel, G., 49, 302
merit pool, 280
metaphysical research programs, 14
methodological anarchism, 21–3
Middlesex University, 153
Ministry of Education, 234
Ministry of Education, Sports, Science and Technology (MEXT), 230, 236, 237, 241, 246, 260–1, 311
Ministry of Finance (MOF), 246, 267
Ministry of Internal Affairs and Communications (MIC), 244
misbehaviour, 109
Money, J., 305
Morrieson, R., 302

N

‘naive knowledges’ 25
naive positivism, 28
naive realism, 9
National Academy of Sciences, 273
National Committee of Inquiry into Higher Education, 126
National Endowment for the Humanities, 272
National Institution for Academic Degrees and University Evaluation (NIAD-UE), 237, 241–2, 247–9, 261, 263
National Research Council, 272–3, 280
National Science Foundation, 279
National Science Funding, 272
National University Corporation Evaluation Committee (NUCEC), 237, 241, 263
National University Corporation Law, 236, 246, 263–4
National University Corporations, 229–67
analysis and hypotheses, 252–9
discussion, 259–64
evaluation system, 264
implementation and results, 246–52
Japan research, 231–3
Japanese university system, 233–6
national universities performance evaluation, 239–46
performance evaluation, 239–46
performance evaluation framework, 244–6
performance evaluation process, 242–4
research assessment, 237–9
Nature category, 208, 211
Nature Neuroscience, 92
nature of knowledge, 8
Nature or Cell, 50
Nature or Science, 63
neoclassical economics, 15, 34
New and World Report, 273
New England Journal of Medicine, 114
New Public Management (NPM), 2, 230
New Scientist and The Scientific American, 19
New York Times, 106
New Zealand, 52, 58, 60, 66, 77, 122, 127, 155
New Zealand Performance Based Research Fund (PBRF), 169–227, 312–13
‘A’ researcher characteristics, 199–200
assessment process, 200–2
assessors assessment, 205–9
contribution to research environment (CRE), 195–8
design and operation, 174–5
effect on research, 221–3
evaluation, 219–23
higher education policy framework, 173
limitations and suggested improvements, 223–7
panel members evaluation, 183–5
panel process, 181–3
peer esteem assessment, 193–5
possible improvements, 204–5
problems with PBRF, 203–4
quality assessment approach, 176–81
research outputs assessment, 185–92
respondent’s view of scheme, 202–5
results, 209–19
Newtonian physics, 17
non-blind reviewing, 90, 100
vs. blind, 102, 103, 118
‘non-Western’ science, 21, 22
non-standard forms, 162
normal scientists, 110
North America, 66

O

Office of Postsecondary Education, 278
organisational reputation, 104
originality, 189
over-all excellence profile, 162
Oxford, 125

P

panel appointment processes, 286–7
panel assessment, 122
panel assessment model, 295–7
panel member bias, 287–9
panel member website, 207
panel members, 136, 226
panel members evaluation, 183–5
respondent characteristics, 184–5
panel review, 86–7, 285–91, 294–5
decision-making problems, 289–91
panel member bias, 287–9
panel review models, 4–5
panel workloads, 136–7
panels, 136–7
appointment process, 139–40
assessment, 122
members, 136
submission and assessment, 137–8
workloads, 136–7
paradigms, 16–20
patents, 46, 52, 57
peer esteem, 7, 122
peer esteem assessment, 193–5
important factors, 193
peer esteem factors, 137
peer review, 5–6, 6–7, 39, 47, 285–91
2006 PBRF panels, 182
decision-making problems, 289–91
New Zealand Performance Based Research Fund (PBRF), 169–227
peer definition and panel appointment processes, 286–7
questions and problems, 99–115
refereeing and discontents, 85–120
review process, 89–92
value, 92–9
peers, 4
Performance Based Research Fund (PBRF), 5, 41, 58
periphery effects, 77
Phelan, T. J., 73–5, 76
Physical Science Panel, 213
panel member article publications, 215
‘physics envy’ 28
plagiarism, 108–9
‘pockets of excellence’ 147
policing mechanism, 93
political correctness, 111–12
Political Theory, 101
Polytechnics, 125–6, 135
Popper, K., 10–14
positivism, 31–4
post-modernism, 20–1
post-structuralism, 20–1
power, 23–6
practical interests, 32
‘problematic papers’ 111
Proceedings of the Linnean Society, 49–50
process development, 87–9
authors by status, 89
process study, 38–42
Provisional Evaluation, 248
public university systems, 6
publication
centrality to research assessment, 47–53
journal rankings, 61–6
measurement, 58–61
outlet research, 53–8
research assessment, 45–83
publication counts, 3, 5, 7

Q

quality assessment approach, 176–81
funding, 180–1
scoring and weightings, 178–80
quality control, 93–5
quantitative measures, 4, 7 See also bibliometrics

R

radical empiricism, 9
‘refereed proceedings’ 56
refereed publication, 53
refereeing
disciplinary device, 98–9
peer review and discontents, 85–120
process development, 87–9
review process, 89–92
referees, 93, 94, 99, 100, 101, 102, 105
Reimer, D., 305
reputational mechanism, 97–8
research
teaching link and balance, 150–2
types and forms, 155–7
United Kingdom, 123–4
research active staff, 137
research assessment, 1–42, 237–9, 269–315
bibliometric and panel assessment dichotomy, 298–9
bibliometric measures, 3–4
bibliometrics, 45–83
bureaucratic control, 295–8
citation analysis, 45–83
decentralised models, 5–6
disciplinary power, panel review and citation analysis, 294–5
group decision-making problems, 291–4
impact, relevance and new directions, 299–309
issues investigation, 281–5
key issues, 8–9
models overlap, 6–7
panel review models, 4–5
peer and panel review, 285–91
policy implications, 310–15
process study, 38–42
process study and research assessment, 38–42
publication, 45–83
science philosophy, 7–9, 7–34, 34–8
science philosophy and research assessment, 7–34, 34–8
types, 238
US research and university evaluation, 272–81
Research Assessment Exercise (RAE), 5, 40, 121–66, 127–8, 312
Evaluation of RAE, 146–56
Evolution of RAE, 128–44, 129–31
Process and Cost of RAE, 144–6
RAE rating scale, 133–4
Research and Science in United Kingdom, 123–4
Research Excellence Framework, 157–65
United Kingdom University System, 124–7
research environment, 122
Research Excellence Framework, 5, 40, 121–66, 157–65, 271, 309, 314
bibliometrics, 158–62
Evaluation of RAE, 146–56
Evolution of RAE, 128–44
Evolution of REF, 129–31
impact measurement, 163–5
Process and Cost of RAE, 144–6
RAE rating scale, 133–4
Research and Science in United Kingdom, 123
Research Assessment Exercise, 127–8
United Kingdom University System, 124–7
research inactive, 178, 204
research monographs, 86
research output, 78, 128, 252
research outputs assessment, 185–92
importance of, 186
important factors, 190
outputs assessment, 187–8
quality indicators, 187
research and methods types, 189–91
research outputs outside nominated four, 191–2
Research Quality Framework (RQF), 60, 62
research quantum, 59
research ranking, 140–2
Research Selectivity Exercise, 132
‘research top-up’ 173
research users, 140
resources allocation mechanism, 93
resubmit rates, 91
review panels, 4–5
review process, 89–92
failure, 103
reviewers, 87, 88, 90, 91, 93, 94, 100, 102
consensus and competence, 105–7
revise rates, 91
revolutionary change, 16–20
Royal Society, 48
Royal Society of Edinburgh, 87
Royal Society of New Zealand (RSNZ), 193–4
Russian government, 95

S

scholars, 95–7
scholarship, 32
Schon, J. H., 108
science, technology, engineering and mathematics (STEM), 153
Science category, 208, 211
science philosophy
research assessment, 7–9, 7–34, 34–8
social science and humanities, 26–30
‘Science Wars’, 106
scientific attitude, 28
‘scientific’ method, 190
scientific research programmes, 14–16
Scopus, 46, 70
self-citation, 76–7
self-evaluation report, 257
self-governing community, 95–7
Shame culture, 88
single blind, 90, 103
social activism, 189
social science, 26–30
Social Sciences and Other Cultural/Social Studies Panel, 216
social scientific models, 30
sociology, 34–5
software development fads, 306
Sokal, A., 106–7
Sokal hoax, 21
specialist advisers, 136
Starbuck, W., 290
sub-panels, 136, 141–2
subjugated knowledges, 25–6
submission periods, 138–9, 144

T

teaching/research relationship, 151
technical interest, 32
Tertiary Education Commission (TEC), 38, 173, 182
tertiary sector, 220
Thalidomide, 307
The Origin of Species, 49
Times Higher Education, 52, 66, 124, 273, 301

U

UK Association of Business Schools, 63
UK Idol, 306
UK Research Councils, 300, 301
UK researchers, 64
under review, 90
United Kingdom, 46, 60, 121
research and science, 123–4
university system, 124–7
United States, 6
accreditation processes, 277–8
government agency measures, 278–9
NRC measures for doctoral programmes, 274
ranking research at national level, 273–6
research and university evaluation, 272–81
research assessment, 279–81
state-based evaluation measures, 276–7
universities, 6, 273
University Establishment Standard, 234
University Funding Council, 125, 126, 133, 135
University Grants Committee (UGC), 125, 132, 135
University of Auckland, 219
University of London, 125
University of Otago, 219, 223
University of Tokyo, 233, 246
university ranking, 52
university research, 133
university sector, 171
University System
central control and commodification, 153–4
higher education system, 125
United Kingdom, 124–7

V

Valuazione triennale della ricerca (VTR), 244
value, 92–9

W

Wallace, A., 49
Wananga, 176
web-based survey, 183
Web of Science, 46, 70, 78, 206–7, 210, 221, 271, 285
‘Western’ science, 21, 22
‘who is a peer’ issue, 286
World War II, 233