References – Prometheus Assessed?


Adams, J.Strategic Review of the Performance-Based Research Fund. The assessment process. Leeds, UK: Evidence Ltd, 2008.

Adcock, R., Bevir, M. The History of Political Science. Political Studies Review. 3(1–16), 2005.

Aguillo, I.F., Bar-Ilan, J., Levene, M., Ortega, J.L. Comparing University Rankings. Scientometrics. 2010; 85(1):243–256.

Akrami, M. The Difficulties with Popper’s Nontraditional Conception of Metaphysics. In: Parusnikova Zuzana, Cohen Robert S., eds. Rethinking Popper. Boston, Mass: Springer, 2009.

Alexander, F.K. The Changing Face of Accountability: Monitoring and assessing institutional performance in Higher Education. Journal of Higher Education. 2000; 71(4):411–431.

Amano, I. Continuity and Change in the Structure of Higher Education. In: Cummings W.K., Amano I., Kitammura K., eds. Changes in the Japanese University. A Comparative Perspective. New York: Preager Publishers, 1979.

Andersson, G. Critical Rationalism and the Principle of Sufficient Reason. In: Parusnikova Z., Cohen R.S., eds. Rethinking Popper. Boston, Mass: Springer, 2009.

Arimoto, A., Daizen, T., Kofukuto, K., Hasegawa, Y., Ehara, T. Research on Transition of Academic Profession (2): A comparative study between Japan, United States and Germany. Proceedings of the 60th Meeting of the Japan Society of Educational Sociology. 2008:43–48. [(in Japanese).].

Arnold, I.J.M. Course Level and the Relationship between Research Productivity and Teaching Effectiveness. Journal of Economic Education. 2008; 39(4):307–321.

Associated Press. Japanese Flock to First-Ever Open Budget Debate. see, 2009.

Atkinson, D.R., Furlong, M.J., Wampold, B.E. Statistical Significance, Reviewer Evaluations, and the Scientific Process – Is There a (Statistically) Significant Relationship. Journal of Counseling Psychology. 1982; 29(2):189–194.

Australian Government and Australian Research Council. Excellence in Research for Australia. Canberra: Australian Government; 2009.

Baert, P. Towards a Pragmatist-Inspired Philosophy of Social Science. Acta Sociologica. 2005; 48(3):191–203.

Baggs, J.G., Broome, M.E., Dougherty, M.C., Freda, M.C., Kearney, M.H. Blinding in Peer Review: The preferences of reviewers for nursing journals. Journal of Advanced Nursing. 2008; 64(2):131–138.

Baker, G.P. Incentive Contracts and Performance Measurement. Journal of Political Economy. 1992; 100:598–614.

Bakker L., Boston J., Campbell L., Smyth R., eds. Evaluating the Performance-Based Research Fund. Wellington: Institute of Policy Studies, 2006.

Baldi, S. Normative Versus Social Constructivist Processes in the Allocation of Citations: A network-analytic model. American Sociological Review. 1998; 63(December):829–846.

Banta, T.W., Rudolph, L.B., Van Dyke, J., Fisher, H.S. Performance Funding Comes of Age in Tennessee. Journal of Higher Education. 1996; 67(1):23–45.

Bardi, J.S. The Calculus Wars – Newton, Leibniz and the Greatest Mathematical Clash of All Time. New York: Basic Books; 2006.

Barnett, R. Linking Teaching and Research: A critical inquiry. Journal of Higher Education. 1992; 63(6):619–636.

Baxt, W.G., Waeckerle, J.F., Berlin, J.A., Callaham, M.L. Who Reviews the Reviewers? Feasibility of using a fictitious manuscript to evaluate peer reviewer performance. Annals of Emergency Medicine. 1998; 32(3):310–317.

Bence, V., Oppenheim, C. The Evolution of the UK’s Research Assessment Exercise: Publications, performance and perceptions. Journal of Educational Administration & History. 2005; 37(2):137–155.

Benedict, R. The Chrysanthemum and the Sword: Patterns of Japanese Culture. Boston Mass: Houghton Mifflin; 1946.

Benos, D.J., Bashari, E., Chaves, J.M., Gaggar, A., Kapoor, N., LaFrance, M., Mans, R., Mayhew, D., McGowan, S., Polter, A., Qadri, Y., Sarfare, S., Schultz, K., Splittgerber, R., Stephenson, J., Tower, C., Walton, R.G., Zotov, A. The Ups and Downs of Peer Review. Advances in Physiology Education. 2007; 31(2):145–152.

Benton, T., Craib, I., Philosophy of Social Science. The Philosophical Foundations of Social Thought 2nd edn. Palgrave MacMillan, Houndsmill, Basingstoke, 2011.

Bergstrom, T.C. Free Labor for Costly Journals? Journal of Economic Perspectives. 2001; 15(4):183–198.

Besancenot, D., Faria, J.R., Vranceanu, R. Why Business Schools Do So Much Research: A signalling explanation. Research Policy. 2009; 38(7):1093–1101.

Bevir, M. Foucault, Power, and Institutions. Political Studies. 1999; 47(2):345–359.

Biology Direct. Biology Direct. BioMed Central Ltd, 2009.

Blank, R.M. The Effects of Double-Blind Versus Single-Blind Reviewing – Experimental – Evidence from The American Economic Review. American Economic Review. 1991; 81(5):1041–1067.

Bornmann, L., Daniel, H.D. Selection of Research Fellowship Recipients by Committee Peer Review. Reliability, fairness and predictive validity of Board of Trustees’ decisions. Scientometrics. 2005; 63(2):297–320.

Bornmann, L., Nast, I., Daniel, H.D. Do Editors and Referees Look for Signs of Scientific Misconduct When Reviewing Manuscripts? A quantitative content analysis of studies that examined review criteria and reasons for accepting and rejecting manuscripts for publication. Scientometrics. 2008; 77(3):415–432.

Boston, J. Rationale for the Performance-Based Research Fund: Personal reflections. In: Bakker L., Boston J., Campbell L., Smyth R., eds. Evaluating the Performance-Based Research Fund. Wellington: Institute of Policy Studies, 2006.

Boston, J., Mischewski, B., Smyth, R. Performance-Based Research Fund - Implications for research in the social sciences and social policy. Social Policy Journal of New Zealand. 2005; 24:55–84.

Bourke, P., Butler, L. Publication Types, Citation Rates and Evaluation. Scientometrics. 1996; 37(3):473–494.

Brante, T. Explanatory and Non-Explanatory Goals in the Social Sciences: A reply to Reiss. Philosophy of the Social Sciences. 2008; 38(2):271–278.

Brinn, T., Jones, M.J., Pendlebury, M. The Impact of Research Assessment Exercises on UK Accounting and Finance Faculty. The British Accounting Review. 2001; 33(3):333–355.

Brody, J.R., Kern, S.E. Stagnation and Herd Mentality in the Biomedical Sciences. Cancer Biology & Therapy. 2004; 3(9):903–910.

Browman, H.I., Stergiou, K.I. Factors and Indices Are One Thing, Deciding Who Is Scholarly, Why They Are Scholarly, and the Relative Value of their Scholarship Is Something Else Entirely. Ethics in Science and Environmental Politics. 2008; 8:1–3.

Bruijn, H.D. Managing Performance in the Public Sector. London: Routledge; 2001.

Buchanan, R.A. Accuracy of Cited References: The role of citation databases. College & Research Libraries July:. 292–303, 2006.

Budden, A.E., Tregenza, T., Aarssen, L.W., Koricheva, J., Leimu, R., Lortie, C.J. Double-Blind Review Favours Increased Representation of Female Authors. Trends in Ecology & Evolution. 2008; 23(1):4–6.

Burgstahler, D. Inference from Empirical-Research. Accounting Review. 1987; 62(1):203–214.

Burke, L.A., Rau, B. The Research-Teaching Gap in Management. Academy of Management Learning & Education. 2010; 9(1):132–143.

Burnham, J.C. The Evolution of Editorial PeerReview. 1st International Congress on Peer Review in Biomedical Publication. American Medical Association, Chicago, Il, 1989:1323–1329.

Butler, D. Theses Spark Twin Dilemma for Physicists. Nature. 420(6911), 2002.

Cameron, B.D. Trends in the Usage of ISI Bibliometric Data: Uses, abuses, and implications. Portal-Libraries and the Academy. 2005; 5(1):105–125.

Campanario, J.M. Consolation for the Scientist – Sometimes it is hard to publish papers that are later highly-cited. Social Studies of Science. 1993; 23(2):342–362.

Campanario, J.M. The Parallelism Between Scientists’ and Students’ Resistance to New Scientific Ideas. International Journal of Science Education. 2002; 24(10):1095–1110.

Campbell, P. Escape from the Impact Factor. Ethics in Science and Environmental Politics. 2008; 8:5–7.

Cave, M., Hanney, S., Kogan, M. The Use of Performance Indicators in Higher Education. A Critical Analysis of Developing Practices. London: Jessica Kingsley; 1991.

Centre for Science and Technology Studies. Scoping Study on the Use of Bibliometric Analysis to Measure the Quality of Research in UK Higher Education Institutions. Report to HEFCE by Centre for Science and Technology Studies, Leiden University. Centre for Science and Technology Studies, Leiden University, Leiden, 2007.

Charlton, B.G., Andras, P. Evaluating Universities Using Simple Scientometric Research-Output Metrics: Total citation counts per university for a retrospective seven-year rolling sample. Science and Public Policy. 2007; 34(8):555–563.

Chatterji, M., Seaman, P. Research Assessment Exercise Results and Research Funding in the United Kingdom: A regional “territorial analysis”. Education Economics. 2007; 15(1):15–30.

Cheung, C.K. Possible Biases in Educational Journals: a retrospective study of five leading journals. Asia-Pacific Education Researcher. 2009; 18(1):129–138.

Cicchetti, D.V. The Reliability of Peer-Review for Manuscript and Grant Submissions - A cross-disciplinary investigation. Behavioral and Brain Sciences. 1991; 14(1):119–134.

Codd, J.A. The Performance-Based Research Fund and the Production and Commodification of Knowledge’. In: Bakker L., Boston J., Campbell L., Smyth R., eds. Evaluating the Performance-Based Research Fund. Wellington: Institute of Policy Studies, 2006.

Cohen, J., Hansel, C.E.M., May, E.F. Natural History of Learned and Scientific Societies. Nature. 1954; 173(4399):328–333.

Cohen, M.D., March, J.G., Olsen, J.P. A Garbage Can Model of Organizational Choice. Administrative Science Quarterly. 1972; 17:1–25.

Cole, S., Cole, J., Simon, G. Chance and Consensus in Peer Review. Science. 214(881–6), 1981.

Collini, S. Postscript: Disciplines, Canons, and Publics: The History of “The History of Political Thought in Comparative Perspective”. In: Castiglione D., Hampsher-Monk I., eds. History of Political Thought. Cambridge: Cambridge University Press; 2001:280–302.

Collins, R. Conflict Sociology: Towards an Explanatory Science. New York: Academic Press; 1975.

Conlon, D.E., Morgeson, F.P., McNamara, G., Wiseman, R.M., Skilton, P.F. Examining the Impact and Role of Special Issue and Regular Journal Articles in the Field of Management. Academy of Management Journal. 2006; 49(5):857–872.

Connor, J.T. Positive Reasons for Publishing Negative Findings. American Journal of Gastroenterology. 2008; 103(9):2181–2183.

Connor, S. Nobel Laureates: Don’t put money before science. New rules will stifle groundbreaking research, Government told. The Independent, Thursday, 7. (January):2010.

Corbyn, Z. RAE 2008: The results’, Times Higher Education. December:. 2008; 18:1–19.

Council for Higher Education Accreditation, Important Questions about Accreditation. Degree Mills and Accreditation Mills, 2011.

Couzin-Frankel, J., Grom, J. Plagiarism Sleuths. Science. 2009; 324(5930):1004–1007.

Creation Research Society (CRS), Inc. Creation Research Society (CRS), Inc. History and Aims. Available at, 2009.

Cret, B. Accreditations as Local Management Tools. Higher Education. 2011; 61(4):415–429.

Cronin, B. The Citation Process: The Role and Significance of Citations in Scientific Communication. London: Taylor Graham; 1984.

Curtis, B. Academic Life: Commodification, continuity, collegiality, confusion and the Performance Based Research Fund. New Zealand Journal of Employment Relations. 2007; 32(2):1–16.

Curtis, B., Phibbs, S. Body Politic within the Academy: Gender and the Performance-Based Research Fun. In: Bakker L., Boston J., Campbell L., Smyth R., eds. Evaluating the Performance-Based Research Fund. Wellington: Institute of Policy Studies, 2006.

Cutting G., ed. Continental Philosophy of Science. malden, Mass: Oxford and Carlton Victoria, 2005.

Cyranoski, D. Japanese Science Faces Deep Cuts. Nature. 2009; 462:258–259.

Cyranoski, D., Gilbert, N., Ledford, H., Nayar, A., Yahia, M. The World is Producing More PhDs Than Ever Before. Is it time to stop? Nature. 2011; 472:276–279.

Czarnitzki, D., Glanzel, W., Hussinger, K. Heterogeneity of Patenting Activity and Its Implications for Scientific Research. Research Policy. 2009; 38(1):26–34.

Dale, T., Goldfinch, S. Article Citation Rates and Productivity of Australasian Political Science Units 19952002. Australian Journal of Political Science. 2005; 40(3):425–434.

Davies R., Ikeno O., eds. The Japanese Mind. Boston, Mass: Ruland; and Tokyo: Tuttle Publishing, 2002.

Davis, S. The Decline of Job Loss and Why It Matters. American Economic Review. 2008; 98(2):263–267.

Department for Business Innovation and Skills. Higher Ambitions: The future of universities in a knowledge economy. London: Department for Business Innovation and Skills; 2009.

Department for Business Innovation and Skills. International Comparative Performance of the UK Research Base. London: Department for Business Innovation and Skills; 2009.

Department of Education and Science2007 HERDC Specifications for Collection of 2006 data. Canberra: Department of Education, Science and Training, 2006.

Department of Innovation Research Science. 2009 HERDC Specifications for Collection of 2008 data. Available at, 2009.

Dixon, H., Suckling, J. Outcome Measures in Higher Education. In: Smith P., ed. Measuring Outcome in the Public Sector. Bristol: Taylor & Francis, 1996.

Djulbegovic, B., Lacevic, M., Cantor, A., Fields, K.K., Bennett, C.L., Adams, J.R., Kuderer, N.M., Lyman, G.H. The Uncertainty Principle and Industry- Sponsored Research. In: VII Cochrane Colloquium. Rome: Lancet Ltd; 1999:635–638.

Dollery, B. The Influence of Economic Theories of Government Failure on Public Management Reform. In: Goldfinch S., Wallis J., eds. International Handbook of Public Sector Reform. London: Edward Elgar, 2009.

Donovan, C. The Governance of Social Science and Everyday Epistemology. Public Administration. 2005; 83(3):597–615.

Donovan, C. The Hidden Perils of Citation Counting for Australasian Political Science. Australian Journal of Political Science. 2007; 42(4):665–678.

Donovan, C. The Qualitative Future of Research Evaluation. Science and Public Policy. 2007; 34(8):585–597.

Donovan, C. Gradgrinding the Social Sciences: The politics of metrics of political science. Political Studies Review. 2009; 7(1):73–83.

Dora, M., Peter, N. The Changing Research Funding Regime in Australia and Academic Productivity. Mathemadtics and Computers in Simulation. 2008; 78(2–3):283–291.

Doucet, M., Sismondo, S. Evaluating Solutions to Sponsorship Bias. Journal of Medical Ethics. 2008; 34(8):4.

Dougherty, K., Natow, R., The Demise of Higher Education Performance Funding Systems in Three States CCRC Working Paper No. 17. Community College Research Centre, Columbia University, New York, 2009.

Dougherty, K., Natow, R. The Politics of Performance Funding in Eight States. Final Report to the Lumina Foundation for Education. New York: Community College Research Centre, Columbia University; 2011.

Eberley, S., Keith Warner, W. Fields or Subfields of Knowledge: Rejection rates and agreement in peer review. The American Sociologist. 1990; 21(3):217–231.

Epstein, W.M. Confirmational Response Bias and the quality of the Editorial Processes Among American Social Work Journals. Research on Social Work Practice. 2004; 14(6):450–458.

Epstein, W.M. The Lighter Side of Deception Research in the Social Sciences. Journal of Information Ethics. 2006; 15(1):11–26.

Erren, T.C. The Long and Thorny Road to Publication in Quality Journals. PLoS Computational Biology. 2007; 3(12):e251.

Evans, A.T., McNutt, R.A., Fletcher, R.H., Fletcher, S.W. The Effects of Blinding on the Quality of Peer Review – A randomized trial. Clinical Research. 1990; 38(2):A738.

Evans, A.T., McNutt, R.A., Fletcher, R.H., Fletcher, S.W. The Characteristics of Peer Reviewers Who Produce Good–Quality Reviews. Journal of General Internal Medicine. 1993; 8(8):422–428.

Evans, J.A. Industry Collaboration, Scientific Sharing, and the Dissemination of Knowledge. Social Studies of Science. 2010; 40(5):757–791.

Evidence Ltd and HEFCE. Pilot Study of Bibliometric Indicators of Research Quality: Development of a bibliographic database. Report to the UK HE funding bodies by Evidence. HEFCE Leeds: Evidence Ltd; 2009.

Feyerabend, P. Against Method, 4th edn. London: Verso; 2010.

Flynn, J.R. How to Defend Humane Ideals: Substitutes for Objectivity. Lincoln Neb: University of Nebraska Press; 2000.

Foucault, M. Power/Knowledge. New York: Pantheon Books; 1980.

Foucault, M. Two Lectures. In: Gordon C., ed. Power/Knowledge: Selected Interviews and Other Writings, 1972–1977. Cambridge, Mass: MIT Press, 1980. [pp. viii, 413.].

Foucault, M. Governmentality. In: Burchell G., Gordon C., Miller P., eds. The Foucault Effect: Studies in Governmentality. Chicago, Ill: University of Chicago Press, 1991.

Friedman, M. Essays in Positive Economics. Chicago Ill: University of Chicago Press; 1953.

Fujimura-Faneslow, K. Japan. In: Postiglione G.A., Mak O.C.L., eds. Asian Higher Education. An International Handbook and Reference Guide. Westport, Conn: Greenwood Press, 1997.

Gao, X., Guan, J.C. Networks of Scientific Journals: An exploration of Chinese patent data. Scientometrics. 2009; 80(1):283–302.

Garfield, E. The History and Meaning of the Journal Impact Factor. Journal of the American Medical Association. 2006; 295(1):90–93.

Gauld, R., Goldfinch, S. Dangerous Enthusiasms: E-Government, Computer Failure and Information System Development. Dunedin, NZ: Otago University Press; 2006.

Gibbons, R., Murphy, K.J. Relative Performance Evaluation for Chief Executive Officers. Industrial and Labor Relations Review. 1990; 43:30–51.

Gibson, J., Tressler, J., Anderson, D.L., Do Research Assessment Exercises Raise the Returns to Publication Quality? Evidence from the New Zealand Market for academic economists Working Paper in Economics 11/08: Department of Economics, University of Waikato, 2008.

Godlee, F. Making Reviewers Visible – Openness, accountability, and credit. Journal of the American Medical Association. 2002; 287(21):2762–2765.

Godlee, F., Gale, C.R., Martyn, C.N., Effect on the Quality of Peer Review of Blinding Reviewers and Asking Them to Sign Their Reports – A randomized controlled trial. 3rd International Congress on Peer Review in Biomedical Publication. American Medical Association, Prague Czech Republic, 1997:237–240.

Goldberg, R.E.A. Reference Accuracy in the Emergency Medicine Literature. Annals Emergency Medicine. 1993; 22:1450–1454.

Golder, S., Loke, Y.K. Is There Evidence for Biased Reporting of Published Adverse Effects Data in Pharmaceutical Industry-Funded Studies? British Journal of Clinical Pharmacology. 2008; 66(6):767–773.

Goldfinch, S. Remaking New Zealand and Australian Economic Policy. Ideas, institutions, and policy communities. Wellington and Washington, DC: Victoria University Press and Georgetown University Press; 2000.

Goldfinch, S. Paradigms, Economic Ideas and Institutions in Economic Restructuring: The case of New Zealand. Political Science. 2000; 52(1):1–21.

Goldfinch, S., Bringing Theories of the State Back. Governmentality, the State and Neoliberal Policy Change. Paper delivered to the NZPSA Conference, Palmerston North, NZ, 7–9 December, 2001.

Goldfinch, S. Investing in excellence? The performance-based research fund and its implications for political science departments in New Zealand. Political Science. 2003; 55(1):39–53.

Goldfinch, S. Examining the National University Corporation Plan and University Reform in Japan: Lessons from Higher Education Reform in New Zealand. The Journal of Finance and Management in Colleges and Universities. 2004; 1:231–261.

Goldfinch, S. Rituals of Reform, Policy Transfer, and the National University Corporation Reforms of Japan. Governance – An International Journal of Policy and Administration. 2006; 19(4):585–604.

Goldfinch, S. Measuring Research and Citation Analyses in Australasian Political Science Units: A reply to Donovan. Australian Journal of Political Science. 2007; 42(4):679–681.

Goldfinch, S. Pessimism, Computer Failure, and Information Systems Development in the Public Sector. Public Administration Review. 2007; 67(5):917–929.

Goldfinch, S. Dangerous Enthusiasms and Information Systems Development in the Public Sector. In: Golfinch S., Wallis J., eds. International Handbook of Public Sector Reform. London: Edward Elgar, 2009.

Goldfinch, S. Introduction. In: Golfinch S., Wallis J., eds. International Handbook of Public Sector Reform. London: Edward Elgar, 2009.

Goldfinch, S., Bellamy, P. The Performance of Public Science: A research assessment of the Crown Research Institutes of New Zealand using bibliometric measures. Journal of Contemporary Issues in Business and Government. 2001; 7(1):5–20.

Goldfinch, S., Malpass, D., The Polish Shipyard: Myth, economic history and economic policy reform in New Zealand. Paper delivered to APSA. September, Dunedin, NZ, 2005.

Goldfinch, S., Malpass, D. The Polish Shipyard: Myth, economic history and economic policy reform in New Zealand. Australian Journal of Politics and History. 2007; 53(1):118–137.

Goldfinch, S., Wallis, J. Two Myths of Convergence in Public Management Reform. Public Administration. 2010; 88(4):1099–1115.

Goldfinch, S., Dale, T., DeRouen, K. Science from the Periphery: Collaboration, networks and “Periphery Effects” in the citation of New Zealand Crown Research Institutes articles, 1995–2000. Scientometrics. 2003; 57(3):321–337.

Goodman, S.N., Berlin, J., Fletcher, S.W., Fletcher, R.H. Manuscript Quality Before and after Peer-Review and Editing at Annals of Internal-Medicine. Annals of Internal Medicine. 1994; 121(1):11–21.

Gordon, M. Peer Review in Physics. Physics Bulletin. 1979; 30:112–113.

Gorton, W. Karl Popper and the Social Sciences. Albany, NY: State University of New York Press; 2006.

Gray, R., Guthrie, J., Parker, L. Rites of Passage and the Self-immolation of Academic Accounting Labour: An essay exploring exclusivity versus mutuality in accounting scholarship. Accounting Forum. 2002; 26(1):1.

Habermas, J., Weber Nicholsen, S. The New Conservatism: Cultural Criticism and the Historians’ Debate. Cambridge: Polity; 1989.

Hacking, I. The Social Construction of What?. Cambridge, Mass: Harvard University Press; 1999.

Hall, C., Morris Mathews, K. Impact of the Performance-Based Research Fund on Teaching and the Research-Teaching Balance: A survey of a New Zealand University. In: Bakker L., Boston J., Campbell L., Smyth R., eds. Evaluating the Performance-Based Research Fund. Wellington: Institute of Policy Studies, 2006.

Hamilton, M., and Rhodes, R.A.W. (undated) Australian Political Science: Journal and Publisher Rankings. Prepared for the Australian Political Studies Association.

Hanada, K., Miyata, M., Insiders View: Japanese research culture September 12, see. Science 2003;. http://www.sciencecareers.sciencemag.orh/career_development/previous_issues/articles/2590/insiders_view_japanese_research_culture

Hargens, L., Schuman, H. Citation Counts and Social Comparisons: Scientists’ use and evaluation of citation index data. Social Science Research. 1990; 19:205–221.

Hart, P., Bureau, Austin. Emails Tell Different Story About Perry Links to Higher Ed “Reforms”. Chron. Com. Sunday, 17 April 2011. Available at, 2011.

Hattie, J., Marsh, H.W. The relationship between research and teaching: A meta–analysis. Review of Educational Research. 1996; 66(4):507–542.

Hausman, D. The Inexact and Separate Science of Economics. Cambridge and New York: Cambridge University Press; 1992.

Hazledine, T., Kurniawan, C. Impact and Implications of the 2003 Performance Based Research Fund. In: Bakker L., Boston J., Campbell L., Smyth R., eds. Evaluating the Performance-Based Research Fund. Wellington: Institute of Policy Studies, 2006.

Heeks, R., Bailur, S. Analyzing e-government research: Perspective, philosophies, theories, methods, and practice. Government Information Quarterly. 2007; 24(2):243–265.

HEFCEM6/97 The impact of the 1992 Research Assessment Exercise on higher education institutions in England. London: Higher Education Funding Council for England (HEFCE), 1997.

HEFCEResearch Excellence Framework. Consultation on the assessment and funding of research post–2008. London: Higher Education Funding Council for England (HEFCE), 2007.

HEFCEResearch Excellence Framework. Second consultation on the assessment and funding of research. London: Higher Education Funding Council for England (HEFCE), 2009.

HEFCE. Research Excellence Framework: Second consultation on the assessment and funding of research. Summary of responses. March 2010 Available at, 2010.

Henrickson, L., McKelvey, B., Foundations of “New” Social Science: Institutional legitimacy from philosophy, complexity science, postmodernism, and agent-based modelling. Proceedings of the National Academy of Sciences of the United States of America. 2002:7288–7295. [99].

Hergovich, A., Schott, R., Burger, C. Biased Evaluation of Abstracts Depending on Topic and Conclusion: Further evidence of a confirmation bias within scientific psychology. Current Psychology. 2010; 29(3):188–209.

Hicks, D., Tomizawa, H., Saitoh, Y., Kobayashi, S. Bibliometric Techniques in the Evaluation of Federally Funded Research in the United States. Research Evaluation. 2004; 13(2):78–86.

Hillman, A.J., Rynes, S.L. The future of doubleblind review in management. Journal of Management Studies. 2007; 44(4):622–627.

Hindu, The, Concern over corruption in universities. The Hindu, 2011. [9 may].

Hirsch, J.E., An index to quantify an individual’s scientific research output. 2005. [arXiv: physics/0508025v5 [physics.soc–ph], 29 Sept.].

Hodder, A., Hodder, C. Research culture and New Zealand’s Performance-Based Research Fund: Some insights from bibliographic compilations of research outputs. Scientometrics. 2010; 84(3):887–901.

Hojat, M., Gonnella, J.S., Caelleigh, A.S. Impartial judgment by the “gatekeepers” of science: Fallibility and accountability in the peer review process. Advances in Health Sciences Education. 2003; 8(1):75–96.

Holmstrom, B., Milgrom, P. Multitask Principal-Agent Analyses: Incentive Contracts, Asset Ownership, and Job Design. Journal of Law Economics and Organization. 1991; 7(S):24–52.

House of Commons Science and Technology Committee. The Research Assessment Exercise. Second Report of Session 2001–2. London: House of Commons; 2002.

Hoy, D. Jaques Derrida. In: Skinner Q., ed. The Return of Grand Theory in the Social Sciences. Cambridge and New York: Cambridge University Press, 1985.

Hubbard, R., Vetter, D.E. An Empirical Comparison of Published Replication Research in Accounting, Economics, Finance, Management, and Marketing. Journal of Business Research. 1996; 35(2):153–164.

Hubbard, R., Vetter, D.E., Little, E.L. Replication in Strategic Management: Scientific testing for validity, generalizability, and usefulness. Strategic Management Journal. 1998; 19(3):243–254.

Hutchinson, P., Read, R., Sharrock, W. There Is No Such Thing As a Social Science: In Defence of Peter Winch. London: Ashgate; 2008.

Intemann, K. Feminist Standpoint Empiricism: Rethinking the Terrain in Feminist Philosophy of Science. In: Magnus P.D., Busch J., eds. New Waves in Philosophy of Science. Basingstoke: Palgrave Macmillan, 2010.

Irvin, R.S. The Role of Conflict of Interest in Reporting of Scientific Information. Chest. 2009; 136(1):253–259.

Isenberg, S.J., Sanchez, E., Zafran, K.C. The Effect of Masking Manuscripts for the Peer-review Process of an Ophthalmic Journal. British Journal of Ophthalmology. 2009; 93(7):881–884.

ISI Web of Knowledge. Essential Science Indicators. Available at, 2009.

Jackson, N., Lund, H. Benchmarking for Higher Education. Buckingham: SRHE and Open University Press; 2000.

Jagsi, R., Sheets, N., Jankovic, A., Motomura, A.R., Amarnath, S., Ubel, P.A. Frequency, Nature, Effects, and Correlates of Conflicts of Interest in Published Clinical Cancer Research. Cancer. 2009; 115(12):2783–2791.

Janis, I. Groupthink, 2. Boston, Mass: Houghton Mifflin, 1982.

Jenkins, A. The Relationship Between Teaching and Research: Where does geography stand and deliver? Journal of Geography in Higher Education. 2000; 24(3):325–351.

Johnes, J., Taylor, J. Performance Indicators in Higher Education. Buckingham: SRHE and Open University Press; 1990.

Jump, P. Claims of “Widespread Corruption” in Spain. Times Higher Education, 12. (May):2011.

Justice, A.C., Cho, M.K., Winker, M.A., Berlin, J.A., Rennie, D., Does Masking Author Identity Improve Peer Review Quality? – A randomized controlled trial. 3rd International Congress on Peer Review in Biomedical Publication. Amer Medical Assoc, Prague Czech Republic, 1997:7–14.

Kanazawa, S. Intelligence and Physical Attractiveness. Intelligence. 2011; 39(1):7–14.

Katz, D.S., Proto, A.V., Olmsted, W.W., Incidence and Nature of Unblinding by Authors: Our experience at two radiology journals with double–blinded peer review policies. 4th International Congress on Peer Review in Biomedical Publication. Amer Roentgen Ray Soc, Barcelona, Spain, 2001:1415–1417.

Kelly, A., Morris, H., Rowlinson, M., Harvey, C. The Association of Business Schools. Academic Journal Quality Guide, 2009. [Version 3].

Kilwein, J.H. Biases in Medical Literature. Journal of Clinical Pharmacy and Therapeutics. 1999; 24(6):393–396.

Koertge N., ed. House Built on Sand: Exposing Postmodernist Myths about Science. Cary, NC: Oxford University Press, 1998.

Koh, C.E. IS Journal Review Process: A survey on IS research practices and journal review issues. Information and Management. 2003; 40(8):743–756.

Krammer, S.M.S. Drivers of National Innovation in Transition: Evidence from a panel of Eastern European countries. Research Policy. 2009; 38(5):845–860.

Kuhn, T.S. The Structure of Scientific Revolutions, 3rd edn. Chicago Ill: University of Chicago Press, 1996.

Kukla, A. Social Constructivism and the Philosophy of Science. London and New York: Routledge; 2000.

Lakatos, I.Lakatos Imre, Feyerabend Paul, Motterlini Matteo, eds. For and Against Method: Including Lakatos’s Lectures on Scientific Method and the Lakatos-Feyerabend Correspondence. Chicago and London: University of Chicago Press, 1999.

Lansley, P. Making Sense of Research Quality Assessment. Engineering Construction and Architectural Management. 2007; 14(1):7–25.

Larivière, V., Gingras, Y., Archambault, É. The Decline in the Concentration of Citations, 1900–2007. Journal of the American Society for Information Science and Technology. 2009; 60(4):858–862.

Larsson, K.S. The Dissemination of False Data Through Inadequate Citation. Journal of Internal Medicine. 1995; 238(5):445–450.

Latour, B. Science in Action: How to Follow Scientists and Engineers through Society. Cambridge Mass: Harvard University Press; 1987.

Latour, B.Pandora’s Hope. Essays on the Reality of Science Studies. Cambridge, Mass: Harvard University Press, 1999.

Lazear, E.P. Economic Imperialism. Quarterly Journal of Economics. 2000; 115:99–146.

Lee, F.S. The Research Assessment Exercise, the state and the dominance of mainstream economics in British universities. Cambridge Journal of Economics. 2007; 31(2):309–325.

Lee, F.S., Harley, S. Peer Review, the Research Assessment Exercise and the Demise of Non-Mainstream Economics. Capital & Class. 1998; 66:23. [12].

Lee, K.P., Schotland, M., Bacchetti, P., Bero, L.A., Association of Journal Quality Indicators with Methodological Quality of Clinical Research Articles. 4th International Congress on Peer Review in Biomedical Publication. Amer Medical Assoc, Barcelona, Spain, 2001:2805–2808.

Lehmann, S., Jackson, A.D., Lautrup, B.E. Measures for Measures. Nature. 2006; 444(7122):1003–1004.

Leisyte, L., Enders, J., de Boer, H. The Balance Between Teaching and Research in Dutch and English Universities in the Context of University Governance Reforms. Higher Education. 2009; 58(5):619–635.

Lerner, E. Fraud Shows Peer–review Flaw. The Industrialist Physicist. 2003; 8(6):12–17.

Liebeskind, J., Oliver, A., Zucker, L., Brewer, M. Social Networks, Learning and Flexibility: Sourcing scientic knowledge in new biotechnology firms. Organization Science. 1996; 7(4):428–443.

Liesegang, T.J. The Distortion of Commercial Research Reported in the Peer-reviewed Literature. Canadian Journal of Ophthalmology – Journal Canadien d’Ophtalmologie. 2009; 44(4):367–368.

Lindsay, R., Breen, R., Jenkins, A. Academic Research and Teaching Quality: The views of undergraduate and postgraduate students. Studies in Higher Education. 2002; 27(3):309–327.

Lindsey, D.The Scientific Publication System in Social Science: A Study of the Operation of Leading Professional Journals in Pschology, Sociology, and Social Work. San Franciso, Calif): Jossey-Bass, 1978.

Lock, S.A Difficult Balance: Editorial Peer Review in Medicine. Philadelphia, Pa: ISI Press, 1985.

Lynn, F.B., Podolny, J.M., Tao, L. A Sociological (De)Construction of the Relationship between Status and Quality. American Journal of Sociology. 2009; 115(3):755–804.

MacRoberts, M.H., MacRoberts, B.R. Problems of Citation Analysis: A critical review. Journal of the American Society for Information Science. 1989; 40:342–349.

Mahoney, M. Publication Prejudices: An experimental study of the confirmatory bias in peer review system. Cognitive Therapy and Research. 1(161–75), 1977.

Manas, M. International Accreditation of Universities. Politicka Ekonomie. 2002; 50(4):598–599.

March, J.G., Olsen, J.P. Institutional Perspectives on Political Institutions. Governance. 1996; 9(3):247–264.

Marois, R., Ivanoff, J. Capacity Limits of Information Processing in the Brain. Trends in Cognitive Sciences. 2005; 9(6):296–305.

Marsh, H.W., Hattie, J. The Relation Between Research Productivity and Teaching Effectiveness – Complementary, antagonistic, or independent constructs? Journal of Higher Education. 2002; 73(5):603–641.

Marsh, H.W., Bonds, N.W., Jayasinghe, U.W. Peer Review Process: Assessments by applicant-nominated referees are biased, inflated, unreliable and invalid. Australian Psychologist. 2007; 42(1):33–38.

Marsh, H.W., Jayasinghe, U.W., Bond, N.W. Gender Differences in Peer Reviews of Grant Applications: A substantive-methodological synergy in support of the null hypothesis model. Journal of Informetrics. 2010; 5(1):167–180.

Martin, B.R. The Use of Multiple Indicators in the Assessment of Basic Research. Scientometrics. 1996; 36:343–362.

Martin, B. Dissent and Heresy in Medicine: models, methods, and strategies. Social Science & Medicine. 2004; 58(4):713–725.

Martin-Alcoff, L. Foucault’s Philosophy of Science: Structures of Truth/Structures of Power. In: Cutting G., ed. Continental Philosophy of Science. Malden Mass: Oxford and Carlton Victoria, 2005.

Marx, W., Bornmann, L. How Accurately Does Thomas Kuhn’s Model of Paradigm Change Describe the Transition from the Static View of the Universe to the Big Bang Theory in Cosmology? Scientometrics. 2010; 84(2):441–464.

McLean, M., Barker, H. Students Making Progress and the “Research-teaching Nexus” Debate. Teaching in Higher Education. 2004; 9(4):407–419.

McMillan, G.S., Hamilton, R.D. The Public Science Base of US Biotechnology: A citation-weighted approach. Scientometrics. 2007; 72(1):3–10.

McMillan, G., Hamilton, R., Deeds, D. Firm Management of Scientific Information: An empirical update. R&D Management. 2000; 30:177–182.

McVeigh, B. Japanese Higher Education as Myth. Armonk, London and New York: M. E. Sharpe; 2002.

Melero, R., Lopez-Santovena, F. Referees’ Attitudes Toward Open Peer Review and Electronic Transmission of Papers. Food Science and Technology International. 2001; 7(6):521–527.

MEXT. Statistical Abstract for 2009, 2009.

Meyer, M. What is Special About Patent Citations? Differences between scientific and patent citations. Scientometrics. 2000; 49(1):93–123.

Middleton, S. Researching Identities: Impact of the Performance-Based Research Fund on the subject(s) of education. In: Bakker L., Boston J., Campbell L., Smyth R., eds. Evaluating the Performance-Based Research Fund. Wellington: Institute of Policy Studies, 2006.

Miller, K., Roderick, P., Sawicka, T. Impact of the Performance-Based Research Fund on Research Management at Victoria University of Wellington. In: Bakker L., Boston J., Campbell L., Smyth R., eds. Evaluating the Performance-Based Research Fund. Wellington: Institute of Policy Studies, 2006.

Minelli, E., Rebora, G., Turri, M. The Structure and Significance of the Italian Research Assessment Exercice (VTR). In: Mazza C., Quattrrone P., Riccaboni A., eds. European Universities in Transition. Cheltenham: Edward Elgar, 2007.

Ministry of Education (NZ)How the PBRF Has Shifted Funding. Wellington: Ministry of Education, 2008.

Mirowski, P. The Scientific Dimensions of Social Knowledge and their Distant Echoes in 20th-Century American Philosophy of Science. Studies in History and Philosophy of Science. 2004; 35A(2):283–326.

Modelling Value Judgements and the Estimation of Uncertainty in Climate. Optimism and the Pessimistic Induction. In: Magnus P.D., Busch J., eds. New Waves in Philosophy of Science. Basingstoke: Palgrave Macmillan, 2010.

Moizer, P. Publishing in Accounting Journals: A fair game? Accounting Organizations and Society. 2009; 34(2):285–304.

Montaner, J.S.G., O’Shaughnessy, M.V., Schechter, M.T. Industry-sponsored Clinical Research: A double-edged sword. Lancet. 2001; 358(9296):1893–1895.

Morgan, R.P., Kruytbosch, C., Kannankutty, N. Patenting and Invention Activity of U.S. Scientists and Engineers in the Academic Sector: Comparisons with Industry. Journal of Technology Transfer. 2001; 26(1):173–183.

MORST. National Bibliometric Report 2001 to 2004 International Benchmarking of New Zealand Research. Wellington: MORST; 2006.

Morton, R. The Less Acceptable Face of Bias. The Lancet. 2000; 356(9234):959–960.

Motterlini, M. Reconstructing Lakatos: A reassessment of Lakatos’ epistemological project in the light of the Lakatos Archive. Studies in History and Philosophy of Science. 2002; 33A(3):487–509.

Mulligan, A. Is Peer Review in Crisis? Oral Oncology. 2005; 41(2):135–141.

Munro, G.D., Stansbury, J.A. The Dark Side of Self-Affirmation: Confirmation bias and illusory correlation in response to threatening information. Personality and Social Psychology Bulletin. 2009; 35(9):1143–1153.

Musgrave, A. Common Sense, Science, and Scepticism: a Historical Introduction to the Theory of Knowledge. Cambridge: Cambridge University Press; 1993.

Musgrave, A. Experience and Perceptual Belief. In: Parusnikova Z., Cohen R.S., eds. Rethinking Popper. Boston, Mass: Springer, 2009.

Narin, F., Olivastro, D. Linkage Between Patents and Papers: An interim EPO/US comparison. Scientometrics. 1998; 41(1–2):51–59.

National Committee of Inquiry into Higher Education. Report of the National Committee (Dearing Report). Available at, 1997.

National Research CouncilResearch-Doctorate Programs in the United States Continuity and Change. Washington, D.C: National Research Council. National Academy of Sciences, 1995.

National Research CouncilA Guide to the Methodology of the National Research Council of Doctorate Programs. Washington, D.C: The National Academies Press, 2009.

National Research CouncilA Data-Based Assessment of Research Doctorate Programs in the United States. Washington, D.C: National Research Council. National Academy of Sciences, 2010.

National Research Council Report in BriefA Data-Based Assessment of Research Doctorate Programs in the United States. Washington, D.C: National Research Council. National Academy of Sciences, 2010.

National Research CouncilA Revised Guide to the Methodology of the Data–Based Assessment of Research– Doctorate Programs in the United States Washington. Washington D.C: The National Academies Press, 2010.

Newcombe, N.S., Bouton, M.E. Masked Reviews Are Not Fairer Reviews. Perspectives on Psychological Science. 2009; 4(1):62–64.

Nickerson, R.S. Confirmation Bias: A ubiquitous phenomenon in many guises. Review of General Psychology. 1998; 2(2):175–220.

Nightingale, P., Scott, A. Peer Review and the Relevance Gap: Ten suggestions for policy-makers. Science and Public Policy. 2007; 34(8):543–553.

Nola, R. Postmodernism, a French Cultural Chernobyl, Foucault on Power/Knowledge. Filozofia. 1995; 50(3):172–191.

Nola, R. Knowledge, Discourse, Power and Genealogy in Foucault. Critical Review of International Social and Political Philosophy (CRISPP). 1998; 1(2):109–154.

Obrecht, M., Tibelius, K., D’Aloisio, G. Examining the Value Added by Committee Discussion in the Review of Applications for Research Awards. Research Evaluation. 2007; 16(2):79–91.

Oerlemans, L., Meeus, M., Boekema, F. Do Networks Matter for Innovation? The usefulness of the economic network approach in analyzing innovation. Tijdschrift voor Economische en Sociale Geografie. 1998; 89(3):298–309.

Olbrecht, M., Bornmann, L. Panel Peer Review of Grant Applications: What do we know from research in social psychology on judgment and decision-making in groups? Research Evaluation. 2010; 19(4):293–304.

Olson, C.M. Peer-Review of the Biomedical Literature. American Journal of Emergency Medicine. 1990; 8(4):356–358.

Ormerod, P. The Death of Economics. London and Boston Mass: Faber and Faber; 1994.

PA Consulting Group. RAE 2008 Accountability Review, 2008. [PA Consulting Group.].

Paisey, C., Paisey, N.J. The Research Assessment Exercise 2001: Insights and implications for accounting education research in the UK. Accounting Education. 2005; 14(4):411–426.

Parusniková, Z., Cohen, R.S.Rethinking Popper. Boston, Mass: Springer, 2009.

Paxton, P., Bollen, K.A. Perceived Quality and Methodology in Graduate Department Ratings: Sociology, political science, and economics. Sociology of Education. 2003; 76(1):71–88.

Peters, D.P., Ceci, S.J. Peer-review Practices of Psychological Journals: The fate of published articles, submitted again. Behav Brain Sci. 1982; 5:187–255.

Phelan, T.J. A Compendium of Issues for Citation Analysis. Scientometrics. 1999; 45:117–136.

Phillimore, A.J. University Research Performance Indicators in Practice: The University Grants Committee’s evaluation of British universities, 1985–86. Research Policy. 1989; 18(5):255–271.

Popper, K.The Logic of Scientific Discovery. London and New York: Routledge Classics, 2002.

Potter, G.The Philosophy of Social Science. Harlow: Pearson, 2000.

Power, M.The Audit Society: Rituals of verification. Oxford: Oxford University Press, 1997.

Psillos, S. The Present State of the Scientific Realism Debatein. In: Clark P., Hawley K., eds. Philosophy of Science Today. Oxford and New York: Oxford University Press, 2000.

RAE2001. Overview of the Research Assessment Exercise. Available at, 2002.

RAE2008. Research Assessment Exercise 2008: the outcome, 2008. [RAE2008.].

RAE2008. RAE 2008 Manager’s Report – April 2009, 2009. [RAE2008.].

(undated) ‘The QR Funding Formula’. Available at

Ramsden, P., Moses, I. Associations between Research and Teaching in Australian Higher-Education. Higher Education. 1992; 23(3):273–295.

Rebora, G. La valutazione dei risultati nelle amministrazioni pubbliche. Proposte operative e di methodo. Milano: Guerini e Associati; 1999.

REF2014Decisions on Assessing Research Impact. Bristol: HEFCE, 2011.

Regehr, G., Bordage, G. To Blind or Not To Blind? What authors and reviewers prefer. Medical Education. 2006; 40(9):832–839.

Research Quality Framework. Research Quality Framework: Assessing the quality and impact of research in Australia 2005. RQF: Canberra; 2005.

Roa, T., Beggs, J.R., Williams, J., Moller, H. New Zealand’s Performance Based Research Funding (PBRF) Model Undermines Maori Research. Journal of the Royal Society of New Zealand. 2009; 39(4):1175–8899.

Roberts, G. Review of Research Assessment. Report by Sir Gareth Roberts to the UK funding bodies. Issued for consultation May 2003, 2003. [RA Review].

Rosell, C., Agrawal, A. Have University Knowledge Flows Narrowed? Evidence from patent data. Research Policy. 2009; 38(1):1–13.

Ross, S. The Economic Theory of Agency. The Principal’s Problem. American Economic Review. 1973; 63:134–139.

Rouse, J. Heidegger on Science and Naturalism. In: Cutting G., ed. Continental Philosophy of Science. Oxford and Carlton Victoria: Malden, Mass, 2005.

Roush, S. Optimism and the Pessimistic Induction. In: Magnus P.D., Busch J., eds. New Waves in Philosophy of Science. Basingstoke: Palgrave Macmillan, 2010.

Sadovnikov, S. Systemism, Social Laws, and the Limits of Social Theory: Themes out of Mario Bunge’s – The sociology-philosophy connection. Philosophy of the Social Sciences. 2004; 34(4):536–587.

Schnatz, P.F., Romegialli, A., Abrantes, J., Marakovits, K., Cunningham, D., O’Sullivan, D.M. The North American Menopause Society: From abstract to publication. Menopause – the Journal of the North American Menopause Society. 2008; 15(5):996–1001.

Schultz, D.M., Are Three Heads Better than Two? How the number of reviewers and editor behavior affect the rejection rate. Scientometrics DOI 2009;, doi: 10.1007/s11192-009-0084-0.

Schultz, D. Rejection Rates for Journals Publishing in the Atmospheric Sciences. Bulletin of the American Meteorological Society. 2010; 91(2):231–243.

Schulz-Hardt, S., Frey, D., Lüthgens, C., Moscovici, S. Biased Information Search in Group Decision Making. Journal of Personality and Social Psychology. 2000; 78(4):655–669.

Schwartz, C.A. The Rise and Fall of Uncitedness. College & Research Libraries. 1997; 58(1):19–29.

Serban, A.M., Burke, J.C. Meeting the Performance Funding Challenge: A nine–state comparative analysis. Public Productivity & Management Review. 1998; 22(2):157–176.

Sharman, J.C. Benchmarking Australian IR: Low impact, a bookish lot or a very British affair? Australian Journal of International Affairs. 2008; 62(4):529–540.

Sharman, J.C., Weller, P. Where is the Quality? Political science scholarship in Australia. Australian Journal of Political Science. 2009; 44(4):597–612.

Sherratt, Y. Continental Philosophy of Social Science. Cambridge and New York: Cambridge University Press; 2006.

Shin, J.C. Impacts of Performance-Based Accountability on Institutional Performance in the US. Higher Education. 2009; 60(1):47–68.

Shuchman, M., Redelmeier, D.A. Politics and Independence – The Collapse of the Canadian Medical Association Journal. N Engl J Med. 2006; 354(13):1337–1339.

Sibbald, B., Flegel, K. Integrity at the Croatian Medical Journal. CMAJ. 2008; 178(13):1637–1638.

Simkin, M.V., Roychowdhury, V.P. Read Before You Cite!. Complex Syst. 2003; 14:269–274.

Simons, M., Elen, J. The “Research–teaching Nexus” and “Education Through Research”: An exploration of ambivalences. Studies in Higher Education. 2007; 32(5):617–631.

Skolnik, M.L. How Academic Program Review Can Foster Intellectual Conformity and Stifle Diversity of Thought and Method. Journal of Higher Education. 1989; 60(6):619–643.

Smart, W. What Determines the Research Performance of Staff in New Zealand’s Tertiary Education Sector?: An analysis of the Performance-Based Research Fund Quality Evaluation. Wellington: Ministry of Education; 2005.

Smart, W. The Impact of the Performance-Based Research Fund on the Research Productivity of New Zealand Universities. Social Policy Journal of New Zealand. 2009; 34:136–151.

Smart, W.Making an Impact: Wellington Tertiary Sector Performance Analysis and Reporting. Ministry of Education: Strategy and System Performance’, 2009.

Smith, A. (undated) ‘Benchmarking Google Scholar with the New Zealand PBRF Research Assessment Exercise’. Wellington: School of Information Management, Victoria University of Wellington.

Smith, A., Ferguson, P.B. Impact of the Performance-Based Research Fund on Maori Research and Researchers. In: Bakker L., Boston J., Campbell L., Smyth R., eds. Evaluating the Performance-Based Research Fund. Wellington: Institute of Policy Studies, 2006.

Smith, P. On the Unintended Consequences of Publishing Performance Data in the Public Sector. International Journal of Public Administration. 1995; 18:277–310.

Smith, R. Peer Review: A flawed process at the heart of science and journals. Journal of the Royal Society of Medicine. 2006; 99(4):178–182.

Sokal, A.D. What the Social Text Affair Does and Does Not Prove. In: Koertge N., ed. A House Built on Sand: Exposing Postmodernist Myths about Science. Oxford: Oxford University Press, 1997.

Southern Association of Colleges and Schools Commission on Colleges. The Principles of Accreditation: Foundations for Quality Enhancement, 4th edn. Decatur, Georgia: Southern Association of Colleges and Schools Commission on Colleges; 2009.

Starbuck, W.H. How Much Better Are the Most-prestigious Journals? The statistics of academic publication. Organization Science. 2005; 16(2):180–200.

Starbuck, W.H. Personal communication. NUBS: Nottingham; 2009.

Statistics New Zealand. Research and Development Survey: 2008. Wellington: Statistics New Zealand; 2009.

Stephan, P.E. The Economics of Science. Journal of Economic Literature. 1996; XXXIV:1199–1235. [September].

Stiftel, B., Rukmana, D., Alam, B. Faculty Quality at US Graduate Planning Schools – A national research council-style study. Journal of Planning Education and Research. 2004; 24(1):6–22.

Stolz, I., Hendel, D.D., Horn, A.S. Ranking of Rankings: Benchmarking twenty-five higher education ranking systems in Europe. Higher Education. 2010; 60(5):507–528.

Suls, J., Martin, R. The Air We Breathe: A Critical Look at Practices and Alternatives in the Peer-Review Process. Perspectives on Psychological Science. 2009; 4(1):40–50.

Sutherland, J. Higher Education, the Graduate and the Labour Market: From Robbins to Dearing. Education & Training. 2008; 50(1):47–51.

Svensson, R. Commercialization of Patents and External Financing During the R&D Phase. Research Policy. 2007; 36(7):1052–1069.

Sweitzer, K., Volkwein, J.F. Prestige Among Graduate and Professional Schools: Comparing the US News’ Graduate School Reputation Ratings Between Disciplines. Research in Higher Education. 2009; 50(8):812–836.

Szava-Kovats, E. The False “Ortega Hypothesis”: A literature science case study. Journal of Information Science. 2004; 30(6):496–508.

’t Hart, P. Groupthink in Government: A Study of Small Groups and Policy Failure. Baltimore, MD: Johns Hopkins University Press; 1994.

Talbot, C., Johnson, C., Wiggan, J. Exploring Performance Regimes: A New Approach to Understanding Public Sector Performance. Report for the National Audit Office; 2005.

Tang, R. Citation Characteristics and Intellectual Acceptance of Scholarly Monographs. College & Research Libraries. 2008; 69(4):356–369.

Tanne, J.H. New Editor Takes Charge at New England Journal of Medicine. BMJ. 1999; 319(7211):662.

Tapper, T. The Governance of British Higher Education. Netherlands: Springer; 2007.

Tarohmaru, H. Complaints and Absence of Consensus on Peer Review of Submitted Papers (in Japanese). Sociology. 2010; 54(3):121–126.

Technopolis. Lessons Learned by Institutions Participating in the Research Excellence Framework (REF), bibliometrics pilot. Results of the Round One. Technopolis: Consultation – report to the HEFCE by Technopolis’; 2009.

Tertiary Education Commission (TEC)Performance-Based Research Fund Draft Guidelines for Assessing Evidence Portfolios that Include Pacific Research. Draft Guidelines for Assessing Evidence Portfolios that Include Pacific Research. Tertiary Education Commission (TEC): Wellington, 2003.

Tertiary Education Commission (TEC)Performance-Based Research Fund Evaluating Research Excellence: the 2003 Assessment. Tertiary Education Commission (TEC): Wellington, 2004.

Tertiary Education Commission (TEC)Performance-Based Research Fund. Annual Report 2008. Tertiary Education Commission (TEC): Wellington, 2009.

Tertiary Education Commission (TEC) and Sector Reference GroupPerformance-Based Research Fund Sector Reference Group Review: New and emerging researchers. Tertiary Education Commission (TEC): Wellington, 2008.

Tijssen, R.J.W. Science Dependence of Technologies: Evidence from inventions and their inventors. Research Policy. 2002; 31(4):509–526.

Tijssen, R.J.W., Buter, R.K., van Leeuwen, T.N. Technological Relevance of Science: An assessment of citation linkages between patents and research papers. Scientometrics. 2000; 47(2):389–412.

Times Higher Education. Court Probes Corruption in Universities. Time Higher Education, 27. 1995.

Times Higher Education, Staff at Risk in RAE Run–up. Times Higher Education. 2004. [20 may].

Times Higher Education, Editor Digs in Over Medical Hypotheses reform. Times Higher Education. 2010. [April].

Times Higher Education, Loss of Philosophy at Middlesex Raised Fears for Humanities. Times Higher Education. 2010. [6 May: 6.].

Times Higher Education, Impact Initiative Will “Corrupt Thinking”, Nobel Laureates Claim. Times Higher Education. 2011. [14 April: 38.].

Todd, P.A., Yeo, D.C.J., Li, D., Ladle, R.J. Citing Practices in Ecology: Can we believe our own words? Oikos. 2007; 116:1599–1601.

Traut-Mattausch, E., Jonas, E., Frey, D., Zanna, M.P. Are there “His” and “Her” Types of Decisions? Exploring gender differences in the confirmation bias. Sex Roles. 2011; 65(3–4):223–233.

Trow, M.The Transition from Elite to Mass Higher Education. Paris: OECD, 1974.

Tussen, R.J.W. Technological Relevance of Science. Scientometrics. 2000; 47(2):389–412.

Research Councils, U.K. What do Research Councils Mean by “Impact”? Available at, 2011.

UNIVERSITAS. Operational Review of the Research Assessment Exercise, 2001 Report to the Joint Funding Bodies’ Research Assessment Review,. Milton Keynes: UNIVERSITAS; 2001.

Universities Funding CouncilReport on the 1989 Research Assessment Exercise. Universities Funding Council, 1989.

Universities Funding CouncilResearch Assessment Exercise 1992: The Outcome. Universities Funding Council Circular 26/92. Universities Funding Council, 1992.

University Funding CouncilReport on the 1989 Research Assessment Exercise. University Funding Council, 1989.

US Department of Education. The Database of Accredited Post-Secondary Institutions and Programs. Available at, 2011.

van Dalen, H.P., Henkens, K. Signals in Science – On the importance of signaling in gaining attention in science. Scientometrics. 2005; 64(2):209–233.

van Rooyen, S., Godlee, F., Evans, S., Black, N., Smith, R. Effect of Open Peer Review on Quality of Reviews and on Reviewers’ Recommendations: A randomised trial. British Medical Journal. 1999; 318(7175):23–27.

Van Thiel, S., Leeuw, F.L. The Performance Paradox in the Public Sector. Public Performance and Management Review. 2002; 25(3):267–281.

Vance, P., Alexander, M., Sandhu, A. Phase Two PBRF Evaluation – Context for the analysis of data in Evidence Portfolios. Wellington: Tertiary Education Commission; 2007.

Visser-Wijnveen, G.J., Van Driel, J.H., Van der Rijst, R.M., Verloop, N., Visser, A. The Relationship Between Academics’ Conceptions of Knowledge, Research and Teaching – A metaphor study. Teaching in Higher Education. 2009; 14(6):673–686.

Wade, R. East Asia’s Economic Success. Conflicting perspectives, partial insights, shaky evidence. World Politics. 1992; 44(2):270–330.

Wampold, B.E., Furlong, M.J., Atkinson, D.R. Statistical Significance, Power, and Effect Size – A response to the reexamination of reviewer bias. Journal of Counseling Psychology. 1983; 30(3):459–463.

Watford, G.Restructuring Universities: Politics and Power in the Management of Change. London: Croom Helm, 1987.

Watson, D., Taylor, R.Lifelong Learning and the University. London: Palmer Press, 1998.

Weber, E. Social Mechanisms, Causal Inference, and the Policy Relevance of Social Science. Philosophy of the Social Sciences. 2007; 37(3):348–359.

Weber, M. The Crux of Crucial Experiments: Duhem’s problems and inference to the best explanation. British Journal for the Philosophy of Science. 2009; 60(1):19–49.

Weller, A.C. Editorial Peer Review: A comparison of authors publishing in two groups of US medical journals. Bulletin of the Medical Library Association. 1996; 84(3):359–366.

White, K.G. Impact of the Performance-Based Research Fund on Staffing in Tertiary Education Institutions. In: Bakker L., Boston J., Campbell L., Smyth R., eds. Evaluating the Performance-Based Research Fund. Wellington: Institute of Policy Studies, 2006.

Wilholt, T. Bias and Values in Scientific Research. Studies in History and Philosophy of Science. 2009; 40(1):92–101.

Willetts, D. David Willetts Delivers First Keynote Speech as Minister for Universities and Science. University of Birmingham, 2010 4 April. Available at

Williams, P., Stevenson, L., Nicholas, D., Watkinson, A., Rowlands, I. The Role and Future of the Monograph in Arts and Humanities Research. Aslib Proceedings. 2009; 61(1):67–82.

Willmott, H. Managing the Academics: Commodification and control in the development of university education in the U.K. Human Relations. 1995; 48(9):993–1027.

Wong, P., Singh, A. University Patenting Activities and their Link to the Quantity and Quality of Scientific Publications. Scientometrics. 2009; 83(1):271–294.

Woodside, A.G. Journal and Author Impact Metrics: An editorial. Journal of Business Research. 2009; 62:1–4.

Yamamoto, K. Agencification in Japan: Renaming or revolution. In: Pollitt C., Talbot C., eds. Unbundled Government. London: Routledge, 2004.

Yamamoto, K. Public Sector Management Reform in Japan. In: Goldfinch S., Wallis J.L., eds. International Handbook of Public Management Reform. Cheltenham: Edward Elgar, 2009.

Yokoyama, K. The Effect of the Research Assessment Exercise on Organisational Culture in English Universities: Collegiality versus managerialism. Tertiary Education Management. 2007; 12:311–322.

Yonezawa, A. Quality Assessment and Assurance in Japanese Universities: The plight of the social sciences. Social Science Japan Journal. 2008; 11(1):69–82.