Research Ranking for the Economics Department at WVU

Methodology

I have again updated a ranking of research output based on the methodology of Scott and Mitias (Economic Inquiry, 1996, pp. 378-400).  Our department now would rank 30 nation-wide based on per-faculty-member pages published in leading journals in economics over the last five years.

Scott and Mitias rank economics departments in the U.S. based on pages published in the top 36 journals in economics. Adjustment for co-authorship is made simply by dividing the page count by the number of co-authors. They convert pages published in each of the 36 journals to "AER equivalent" pages by a multiplication factor that depends on the typical number of words per page relative to the typical number of words on a page in the American Economic Review (AER). Table I  lists the 36 journals used in the ranking and their AER equivalent page conversion factor.

Three ranking methods are provided by Scott and Mitias, based on: (1) flow of publications based on affiliation listed on the article; (2) stock of publications based on faculty listed as full-time faculty members in the economics department; (3) stock of publications per full-time faculty member in economics. These rankings rely on publications during the period 1984-1993.

Using ranking method (1), Scott and Mitias ranks WVU at number 102 over the 10-year period from 1984-1993 with a total of 304.5 AER equivalent pages. Under ranking methods (2) and (3), the Economics Department at WVU does not make the list of 80 universities provided by Scott and Mitias.

Given that majors in economics are offered at approximately 2000 universities and colleges in the U.S., a ranking of 102 does not seem so bad. However, from the perspective that WVU is one of only eighty-some Research I universities, and given that the economics department offers a Ph.D. degree, this ranking is disappointing and appears to under-represent the research qualifications of the current economics faculty.

The purpose of this note then is to extend the Scott and Mitias methodology to (a) provide a more up-to-date ranking starting in 1997; and (b) provide a ranking based on methods (2) and (3), which, instead of considering all economists (including those in finance and agricultural economics departments) is limited to economics department faculty only.

Results

The extension of the Scott and Mitias methodology is straightforward. Data on research output were taken from different years of the Department of Economics "Abbreviated Curriculum Vitae" book, which lists publications since 1990 for economics faculty members and adjuncts. These data were then cross-checked against the publications listed with WVU affiliation in ECONLIT (available on-line via the WVU library system). Results in terms of AER equivalent pages by WVU faculty are listed in Table II for all years since 1990.

To rank the department's output the following assumption is made: since 1988-89 (the midpoint of the Scott and Mitias ranking period) the group of top 36 economics journals hasn't changed and neither has the distribution across departments of pages published in these journals.

This assumption appears to be quite reasonable for our purposes. Possibly some newer journals could have broken into the top 36. However, if this were to be so, our ranking would probably improve rather than worsen since the department has published quite extensively in the journals that are pushing to enter the top 36 (such as the Journal of Health Economics, the Review of International Economics, the European Economic Review, etc.). Also, if the distribution of pages were to change so that certain departments would rise in the ranking, then other departments would fall. So our numerical ranking would not be far off.

Ranking method (1) yields at total of 254.8 pages for the 5-year period 1996-2000 as shown in Table II.  To properly compare with the 10-year ranking in Scott and Mitias double this number to yield 509.6 pages. The Scott and Mitias Article (their Table I) then shows that this number moves us in the rankings from 102 up to number 79.  As Table II shows, the departmental output in the early 1990s was very weak which has led to the relatively low ranking in Scott and Mitias. The Table however clearly shows a positive trend which leads to the 79 ranking over the last five years (not considered by Scott and Mitias).

Ranking method (2) is more appropriate for ranking the research output of economics faculty. Compared to method (1), it: (a) leaves out faculty that are no longer on the faculty (such as Yangru Wu, Eun-Soo Park, and Luc Anselin in our case); (b) leaves out Ph.D. students publishing before employment at another institution (such as Tom Garrett and Atilla Varga in our case); and (c) leaves out faculty at non-economics departments publishing in economics journals (such as Victor Chow and Peter Schaeffer in our case). The resulting ranking is based on the publications during the 1996-2000 period of tenure-track faculty full-time in the economics department in December 2000. It can be seen from Table II that excluding these groups leaves us with 203.1 AER equivalent pages. Again converting to make comparable with the 10-year ranking yields a total of 406.2 pages. This number is quite a bit lower than our number for method (1) but the same applies for other institutions as the publications by Finance and Ag. Econ faculty are ignored. The Scott and Mitias article (their Table III) shows that our ranking based on method (2) would be 63 (compared to us not being ranked at all with method (2) in the Scott and Mitias article). This ranking would put us ahead of such Research I schools as Kansas, Florida State, and Purdue.

Ranking method (3) provides the best measurement of average research achievement of individual econ faculty members. It corrects for the size of departments by dividing the number of pages from ranking method (2) by the number of full-time tenure-track faculty members in a department. For us that is 15 (in December 2001). Thus, dividing 406.2 pages by 15 gives us 27.1 pages per faculty member (that is 2.71 AER equivalent pages in the top 36 journals per econ faculty member per year). Based on the numbers in Scott and Mitias (their Table IV) that puts us at number 30 (shared with the Virginia Tech) in the ranking of the top 80 Ph.D. granting departments. This ranks us ahead of the likes of Berkeley, Cornell, Johns Hopkins, and Columbia!

Discussion

Thus, an objective outcomes assessment, which should be based on method (3), would rank the economics faculty as a whole highly in terms or research productivity over the 1997-2001 period.

To what extent is this assessment dependent on the ranking methodology chosen? The Scott and Mitias methodology was designed to rank all economics departments and listed 240 schools with publications in the 36 journals considered. For this purpose, a relatively large group of journals was useful. However, if a smaller number of top economics journals were considered, how would that affect our ranking? This would depend on the exact number. If we look only at the consensus top 3 journals (AER, Econometrica, and JPE) we have only the JPEs of (Garrett and) Sobel. If we look at a somewhat larger group of say between 8 and 12 journals, though, the department would fare much better in the rankings. For instance, at times it has seemed that WVU owned the IER which is listed as one of the top 8 journals by Dusansky and Vernon (Journal of Economic Perspectives, 1998, pp. 157-170) and has published in 3 other journals in this top 8 as well (RES, JPE, and REStat).

Other measures like the method currently used by the College of looking at refereed publications per capita have made the output of the economics faculty seem mediocre, even compared to the other, non-Ph.D. granting, departments in the college. It must be kept in mind however that there is a very large number of refereed journals, which has increased dramatically over the last 10 years or so. ECONLIT abstracts 685 journals in economics, which are almost all refereed, and does not account for all existing refereed journals. Thus, the 36 top journals which many of the economics faculty have targeted, easily belong to the top 5% of refereed journals in economics! As Fishe (Journal of Finance, 1998, p. 1077) states concerning the importance of adjusting for quality: "...the average number of publications and publications per year do not vary significantly between top 20 and lower-ranked schools." It seems therefore that an outcomes assessment of research productivity cannot just rely on numbers of publications and must be based on methods such as those presented here.

This is not to say that publication in refereed journals below the top 5% is unimportant. The research output in the econ department's C.V. book shows that almost half the economics faculty did not contribute pages in the top 36 journals over the period considered. However, considering other refereed publications as well provides a more balanced outcome which is crucial for a small department running a Ph.D program. Also, continued AACSB accreditation requires refereed publications by a broad group of the faculty, including tenured faculty. The experience with our Ph.D. students has shown that publishing in refereed journals is no longer as daunting as it used to be and should be a reasonable goal for a faculty member who is serious about research.

Table I: Top 36 Economics Journals from Scott and Mitias

       Journal (AER equivalent pages)
       ------------------------------------------------------------------


Table II: Ranking of Economics Department Productivity Based on Scott and Mitias Methodology. Pages Adjusted to AER Equivalent Size.
1990 1991 1992 1993 1994 1995 1996 1997 1998 1999
2000
 2001
  Total
                       
D. Adams     6.61                 6.61
R. Balvers * 14.24   4.81 8.43 7.68   22.63     23.75  81.53
S. Bandyopadhyay     9.19 9.14 7.28 6.86  
32.47
S.C. Bandyopadhyay       9.19     6.86    16.04
V. Chow ****   3.06   5.65 2.20 3.90          14.81
B. Cushing           3.90        
 6.98
10.88
S. Douglas           5.93   23.04 12.82    41.78
K. Denning ****     5.65              
5.65
D. Mitchell   6.21 5.69   2.64     1.98   12.47  28.99
W. Reece                   3.34  
3.34
P. Schaeffer ****  11.86  11.86
R. Sobel *     10.61 9.63 17.66 18.95  24.01 3.34
 3.79
  87.99
W. Trumbull     5.69   7.57            13.26
J. Vilasuso
7.32
7.32
L.Anselin ** ' ****           9.32 6.37      
15.69
H. Wall ** ' ****   2.64   2.70            
5.34
Y. Wu ** ' *** ' ****           21.31 19.31 3.30    
43.92
E. Park ****                    5.95     5.95
T. Garrett ****             2.63    10.98  
13.61
O. Smirnov ****           9.32        
9.32
A.Varga *** ' ****             3.19      
3.19
                       
Total (All listed)                  
404.96
                       
Total (WVU affiliation listed; five years) 0.00 9.27 25.42 19.74 22.79 43.96 58.71 104.08 48.78 40.94  42.90
18.09
 254.79
                       
Total (By current Econ faculty; five years) 14.24 6.21 22.79 8.43 17.89 38.81 18.77 72.59 45.48 24.01 42.90 18.09 203.08
                       
* Publications with other affiliations listed not counted for ranking method (1)
** Only publications with WVU affiliation listed are included
*** Publications with two affiliations listed are counted for 50%
**** Not considered for ranking methods (2) and (3)

Last Updated: February 12, 2002
 

Up to Top of Page