Stony Brook Graduate School Dean, Lawrence Martin, Develops New Method for Ranking Doctoral Programs

STONY BROOK, N.Y. - Academic Analytics, LLC (AA LLC), recently announced America 's most productive research universities based on an algorithm for ranking doctoral programs developed by Dr. Lawrence Martin, Dean of the Graduate School at Stony Brook University. Partly financed by the Stony Brook Foundation and produced by Academic Analytics, the new algorithm known as the Faculty Productivity Index (FSP IndexT) is the first quantitative method for ranking doctoral programs at research universities.

The FSP IndexT rates programs based on journal publications by faculty members, book publications, journal articles, grants, honors and awards. In the most recent 2005 index, nearly 178,000 faculty members and 7,300 Ph.D. programs at 354 universities in the United States were rated.

For several years, the gold standard for rating the quality of graduate programs has been by the National Research Council (NRC). These rankings are updated every 13 years and based on reputation alone. As a result, the council's rankings have not been considered as a very accurate measurement. Some have deemed the FSP IndexT as being more accurate of the two. However, Dr. Martin was quick to dispel this assumption.

"I wouldn't say it is more accurate than the National Research Council. It's that this is the first completely objectively based assessment because it's only measuring one thing - the faculty scholarly productivity of faculty associated with graduate programs," said Dr. Martin. "How you interpret a program having high productivity is up to you."

Dr. Martin is the Associate Provost for Analysis and Planning, and serves as Co-PI on second NSF-Alliance for Graduate Education and the Professoriate grant. He is a Professor at the Stony Brook Department of Anthropology where he studies the evolution of apes and the origin of humans. He has edited five books, published 35 journal articles, and book chapters, making him a respected academic in his field. Martin served as Director of International Programs at Stony Brook from 1996 to 2003, as Director of Undergraduate Studies, and Director of the Interdepartmental Doctoral Program in Anthropological Sciences.

Martin also sits on the Scholarly Advisory Board for the Association of American Universities (AAU) Assessment of Quality of University Education and Research project. He is a past President of the Northeastern Association of Graduate Schools, a member of the Executive Committee of the Council on Research Policy and Graduate Education, Chair of the CGS Advisory Task Force on Research and Information Services, a member of the AAU Task Force on Graduate Education, the AAU Task Force on International Education, and a member of the Executive Committee of AAU's Association of Graduate Schools.

When Dr. Martin took on his role as the Dean of the Stony Brook Graduate School in 1993, he was met with the task of coming up with an assessment of the quality of Stony Brook programs to help make funding and management decisions. He took notice that the NRC rankings were not comprehensive and timely enough to achieve the objectives of the assessment. As a result, he became interested in the utilization of other measures of evaluating the quality of graduate programs.

Martin then proceeded to analyze data from the 1995 NRC study, "Research-Doctorate Programs in the United States : Continuity and Change," which rates the quality, effectiveness, and reputation of more than 3,600 doctoral programs at 274 universities. Looking at the newly developed electronic version of the NRC rankings, he was able to ascertain which measures were significantly correlated to program quality, and which programs changed in ranking when the measures were used.

In doing so, he noticed that the rankings of numerous programs changed and alternatively found more indices better predicting faculty productivity. Dr. Martin was able to determine 11 indices that could be used for per capita measurement of faculty productivity versus the 4 used by NRC. He then engaged a graduate student in Anthropological Sciences, Anthony Olejniczak, to help aggregate, construct, and perform statistical analysis of higher education databases. With his help, Dr. Martin created an algorithm to predict Stony Brook's reputation in a national context, and the new ranking index was born.

As a former member of the methodology committee for the council's 1995 study, Mr. Martin was keen to have an annual index to complement the NRC data. With this goal being beyond the scope of the NRC, the new index was developed by Academic Analytics, a for-profit company.

Founded in 2005, Academic Analytics is the result of collaboration between Dr. Martin and Mark Shay, founder and President of Educational Directories Unlimited (EDU). Martin serves as the chief scientific consultant for the company, while Shay acts as the Interim-CEO, bringing in over 22 years of experience running a number of businesses in graduate education including gradschools.com and study abroad.com. Dr. Martin on the other hand brings his wealth of experience as a graduate dean for 13 years, serving on several academic committees, and being a researcher himself.

Despite Academic Analytics' for-profit status, the index's emphasis on quantitative methods gives it credence in the academic world. The FSP IndexT has made it possible for creating standards for the measurement of academic quality, and has set in motion a series of discussions and debates around the needs to assess quality and validate the perceptions of academic excellence. It is a tool intended more for internal review and institutional development than competitive comparisons with other universities.

"We don't want people to try and compete with one another. We just want to show how their performance compares with peer institutions around the country," Dr. Martin clarified.

According to the 2005 index, Stony Brook ranked as the 19th most productive research university in the nation and 6 th among public research universities. In addition, seven of Stony Brook's Doctoral programs ranked in the top 10 among similar programs.

"It was a great pleasure to me to see a verification of what I knew to be true; which is that, this is a remarkable, extraordinary research faculty," said Dr. Shirley Strum Kenny, President of Stony Brook University. "The most important thing about it is for our faculty, our students and the outside world to know how highly we rank and it is very useful for us and any other universities in that regard."

Faculty productivity is judged depending on the most important variables in the each discipline and thus accounts for differences across each program. The top Stony Brook programs and their ranks were: Marine biology and biological oceanography (7), Comparative literature (6), Philosophy (9), Spanish (8), Physical oceanography (3), Anthropology (3), and Political science (4). The programs with the highest productivity indexes were Anthropology (1.78) and Political science (1.81).

The objectivity of the FSP IndexT gives university officials more facts to guide the decision-making process in funding and management, and monitor the activity level of a program over time. Using the index, universities can learn valuable things such as: tracking and predicting departmental productivity, faculty, or identifying areas of need.

Picture: Dr. Lawrence Martin, developer of the new index