Ordering Markets for Academic Knowledge Goes Global: Pablo Lerner on "Foreign Influences and 'Law Reviews Rankings' in Israeli Legal Scholarship "

The production of knowledge, especially among academics, is not simply offered for consumption.  Like all consumables, it is judged and valued in the market within which it is offered.  And again, like other consumables, the judging of academic knowledge products has become standardized, and its assessment routinized in ways that  both make it easier to understand what one is buying and also makes it more likely that all products will resemble each other. 

(Pix (c) Larry Catá Backer 2013)

One of the principle external consequences of this market for academic production, is that much of what is produced is predictably the same--and the market likes it that way.  Another is that producers are judged  (tenure and promotion) by their fidelity to the packaging (law review articles and books) and content standardization (its fidelity to the generally shared normative presumptions of the leading practitioners in the field) features of  knowledge commodities.  To that end, assessment structures are required not merely for the academic, but also for the secondary structures within which her status is itself assessed. (Cf. Backer, Larry Catá, Global Panopticism: States, Corporations and the Governance Effects of Monitoring Regimes. Indiana Journal of Global Legal Studies, Vol. 15, 2007). Within the law academy, principal among these are  systems for the review and ranking of law reviews--the publication spaces which serve as the market space within which legal academic knowledge is proffered for consumption. (Backer, Larry Catá, “Defining, Measuring and Judging Scholarly Productivity: Working Toward a Rigorous and Flexible Approach,” Journal of Legal Education 52(3):317-341 (2002)).

Pablo Lerner on the faculty at the College of Law and Business- Ramat Gan, School of Law will be presenting a very interesting paper, "Foreign Influences and "Law Reviews Rankings" in Israeli Legal Scholarship" at the Legal Education Mini Conference of the 10th Annual International Conference on Law 8-11 July 2013, Athens, Greece. Lerner suggests both the power of ranking in the construction of hierarchies of knowledge production and the disciplining of academics.  More importantly he suggests the way in which these systems are going global.  His suggestion for a "critical pluralist policy" is well worth considering carefully.

This post includes the Abstract, Background and Conclusion of that excellent paper. 

Foreign Influences and "Law Reviews Rankings" in Israeli Legal Scholarship

Pablo Lerner*

Abstract

This paper examines the influence of American law on legal scholarship and legal education in Israel in light of the ranking of law reviews published by the Hebrew University of Jerusalem, Faculty of Law. I argue that law review rankings should not be an exclusive parameter in evaluating academic quality. I critically review the method of ranking as well as show that the quality of an article is not linked to the rank of the review in which it was published —which is at odds with an open, critical, and pluralistic approach to scholarship.

The Jerusalem Ranking—Background

In April 2012, the Hebrew University of Jerusalem, Faculty of Law published a ranking of American, some international, and Israeli law reviews (both published in English and in Hebrew)[1] (Jerusalem Ranking).[2] This ranking was conducted by a commission of Hebrew University Law Faculty professors of Law of Jerusalem—who also consulted scholars from other universities including those outside of Israel. Law reviews are ranked according to four categories: Category A includes the most highly reputed periodicals; category B are good law reviews; category C are intermediate level journals; and category D is logically, those less reputed. The ranking intends to mainly serve as reference (or perhaps a compulsory guide) in which to determine the promotion and tenure of faculty members and candidates through the law reviews and journals in which the publications appear. Accordingly, the higher the standard of the law review in which the candidate publishes, the greater the probability of the candidate being promoted in the faculty. As a matter of principle, this ranking is to be used by the Hebrew University; all other Israeli law schools are free to follow it or not. 

  The Jerusalem Ranking has been criticized by many Israel scholars (most significantly Prof. Reich[1] and Prof. Sangero[2]). In this paper, I refer to these critics and add some new considerations, including some personal aspirations for Israel law scholarship.
About the use of rankings in general

General speaking, rankings have both benefits and shortcomings. As with other disciplines, as well as law, rankings ensure a certain degree of equality between tenure candidates. A candidate who has published in reviews of high standing will likely have greater chances of being admitted than a candidate who has published articles in less qualified reviews. The use of rankings may allow faculty admittance commissions to judge the candidate without investing time and resources in analyzing every article in detail—thus saving resources. Rankings may have also positive influence upon editors—who will do their best to attract better articles and improve their review’s standing.[3]

But rankings also involve a number of pitfalls. Relying only on rankings to choose staff can be limiting and counter-productive: A good scholar who has not succeed in publishing in top journals, may be hindered by the ranking of the law review; thus, her work is not judged by merit. Moreover, certain research subjects are considered "popular" and are more likely to be published in law reviews.[4] Less “sexy” subjects[5] may not be accepted by the top ranked reviews. Unlike the field of exact science in which international review rankings are acceptable, in the humanities, and certainly in law, the comparison between the level and quality of the review is troublesome since different trends, approaches, and legal traditions complicate an overhaul qualification.

Moreover, in the era of Google scholar, Westlaw, Lexis, SSRN,[6] researchers’ home pages, Google search, etc. the importance of law reviews ranking should be reevaluated, and the rejection parameter may have a different meaning due to an advance in less traditional publication opportunities.[7] Today, a meritorious article could be found, and consequently cited, regardless where it is published.[8] Top scholars can be tempted to publish in less elite law-reviews, correctly assuming that the article will be read—and judged—by all those researching the topic on the web. The multitude of alternative publication options allows authors to eschew the tortuous acceptance and publication processes of refereed journals.

Apart from this general question regarding rankings modern relevance, the "Jerusalem ranking" also brings up particular questions related to Israeli scholarship and more broadly—Israeli legal culture.

* * * * * *



A final observation

Rankings develop a culture of competition between scholars, which is not necessarily bad in and of itself.[1] Trying to publish in "high ranking" journals is likely to lead to more accomplished research. With that, rankings may develop a culture of conformism: In an attempt to conform and hence be accepted, the author may conform to the type of research, the subjects studied, the review’s ideology, its editorial patterns, its trends, and the subject the editors are interested in and in this manner loses all academic objectivity. Some scholars tend to develop a certain addiction to rankings. As other addictions this too can be dangerous. Scholars should not become slaves to rankings, especially when this does not necessarily serve good research and shows a wrong dependence on foreign legal culture.

Israeli Law researchers should not reject rankings; however, they only should be used as a point of reference for improving scholarship. In my view, reliance on rankings could be justified, but only so far that they do not become an obstacle to accepting that there are different approaches to law and recognizing there are different legal traditions.

---

* Academic Center of Law and Business- Ramat Gan.

Thanks to Michele Manspeizer for her help in the edition of this paper

[1]  See  Boaz Sangero, In Condemn of the Jerusalem Ranking, ( May 2012 ) [ in Hebrew], available at http://hatraklin.org/2012/05/29/%D7%91%D7%92%D7%A0%D7%95%D7%AA-%D7%9E%D7%93%D7%93-%D7%99%D7%A8%D7%95%D7%A9%D7%9C%D7%99%D7%9D-%D7%9C%D7%A1%D7%99%D7%95%D7%95%D7%92-%D7%9B%D7%AA%D7%91%D7%99D7%A2%D7%AA-%D7%91%D7%9E%D7%A9%D7%A4%D7%98/ .See also Alon Clement, Academic Quality and Competition Market, http://aklement.wordpress.com/2012/08/04/%D7%90%D7%99%D7%9B%D7%95%D7%AA-%D7%90%D7%A7%D7%93%D7%9E%D7%99%D7%AA-%D7%95%D7%A9%D7%95%D7%A7-%D7%AA%D7%97%D7%A8%D7%95%D7%AA%D7%99/ (last visit June 2013)  (Interestingly there was a site for the ranking, but at the time I wrote this paper, the site was not available to the public.)  The ranking was accompanied by a report by the editors, http://Israelaw.files.wordpress.com/2012/05/d79ed793d793-d799d7a8d795d7a9d79cd799d79d-d793d799d79f-d795d797d7a9d791d795d79f.pdf  (last visited  June 11 , 2013).

[2] The intertwining of international and Israeli reviews in the same ranking is questionable, but the editors preferred to do so in order to prevent creating a double scale of values.

[1]  Arieh Reich , The Intellectuals  Betrayal, 16 Horech Hadin 40 (2012) [ in Hebrew].

[2] See supra note 1.  Reich and Sangero are not professors at the Hebrew University. During the annual symposium of Israeli Association of Law and Society held at the Academic Center of Law and Business on Dec. 20, 2012, a panel was held on the ranking. Even among Hebrew University Law School staff there is discord. For example Professor Alon Harel expressed his opposition to this ranking.

[3] See Russell Korobkin, Ranking Journals: Some Thoughts on Theory and Methodology, 26 Florida State University Law Review 851, 858 ff (1999).

[4] This is mitigated in specialist journals. See id. at 874.

[5] For example those who write according to a feminist approach to law may find that ranking does not contribute to feminist scholarship. See Joana Grossman, Feminism Law Reviews and Ranking Conundrum, 12 Columbia Journal of Gender 523 (2003). This article was written ten years ago; thus, I am unsure if the same assessment is true today.

[6] See Cameron Stracher, Reading, Writing and Citing: in Praise of Law Reviews, 52 N.Y. U. L. Rev. 349, 351 (2008). See Bernard Black & Paul Caron, Ranking Law Schools: Using SSRN to Measure Scholarly Performance, 81 Indiana L. Rev. 83 (2006) (examining SSRN’s performance measure ability).

[7] See Carol Bast, Linda Samuels, Legal Studies Scholarship: Setting the Standard for Excellence, 26 J. Legal Stud. Edu. 263 (2009).

[8] See Denis Calhallan & Neal Devins, Law Reviews Article Placement: Benefit or Beauty Prize?, 56 J. Legal Stud. Edu. 375, 387 (2006).

* * * * *

 [1] In an interesting article, two professors scrutinized the American legal education system, arguing that it encourages competition and conformism.  I think that to a certain extent this analysis could be extended to review rankings. Susan Sturn & Lani Guinier, The Law School Matrix: Reforming Legal Education in a Culture of Competition and Conformity, 60  Vanderbilt L. Rev. 515 (2007).

Brian Leiter years ago reminded us that the assessment focus of disciplinary technique grounded on reputation were not very reliable but were better than nothing, noting "one would expect scholarly impact to be an imperfect measure of scholarly quality. But an imperfect measure may still be an adequate measure, and that is almost certainly true of citation rates as a proxy for impact as a proxy for reputation or quality. ". Brian Leiter, Top 35 Law Faculties Based on Scholarly Impact 2007, Brian Leiter's Law School Rankings, Sept. 1, 2007.  The need to discipline markets requires assessment, and the fact that assessment methodologies are crude, and indeed may produce perversity, is of small matter in the face of the need to ensure that academics are ranked and markets for the production of knowledge make it easier to assess product.  Ronen Perry has argued on the basis of his review of law journal ranking theory and methodology that citation based methods might work best (Ronen Perry, "The Relative Value of American Law Reviews: A Critical Appraisal of Ranking Methods," Virginia Journal of Law and Technology 11(2):  (2006) (with an excellent discussion of theory and method)).   Others have sought to use law journal ranking to help frame the institutions through which academic production is validated.  Roger Korobkin has argued that 

rankings can be used to create a status competition among journal editors that can, in turn, provide journal authors with an incentive to produce more valuable scholarly articles. Part III considers the merits of alternative ranking methodologies. It concludes that the desired incentives for editors and authors are best achieved if journals are ranked on the basis of expert evaluations of scholarly value where feasible, and otherwise on the basis of the frequency in which journals are cited in other publications. (Roger Korobkin, Ranking Journals:  Some Thoughts on Theory and Methodology, Florida State Law Review 26:851, 853 (1999)).

But Lerner adds critically to this discussion.  First, globalization substantially impacts assessment.  What was once relatively easy to control by the community of scholars who were subject to its strictures  has now crossed borders and become more autonomous of the communities into which it is applied.  Lerner nicely points to the consequences of globalization of knowledge production outlets for standardization and the discipline of local academic communities,.  The imbalance between global knowledge markets and local academic communities will require substantially more research.  Second, Lerner reminds us that standardization and systems of production reputation alone are no substitute for critical quality assessment of the work of individual academics.  Proxies are just that--a shorthand for gauging assessment that is at best crude.  It is hardly worth repeating that relying solely on those factors will likely produce little assessment of value in individual cases, especially int he context of promotion and tenure.  Lerner suggests these measures can produce presumptions, but that individuals ought to be able to overcome those presumptions through justificatory arguments. While there is much work to be done, and a more careful analysis of the market framework within which these discussions are increasingly undertaken (something that should give academics pause, but does not), Lerner has contributed a very useful approach well worth further study.