What is the value of the research we do? The typical way we have to evaluate the impact of research is to count citations, and possibly weigh them in some way, in Economics and any other sciences (except maybe where patents are relevant). But this only evaluates how the research output is viewed within a narrowly defined scientific community. The contribution to social welfare is an entirely different beast to evaluate.
Robert Hofmeister tries to give research some value. The approach is to consider the scientific process through cohorts, where each wave provides fundamental research as well as end-applications based on previous fundamental research. A particular research results thus can have a return over many generations. It is an interesting way to properly attribute the intellectual source of a new product or process, but the exercise is of little value if it is not possible to quantify the social value of the end-application. Indeed, Hofmeister goes back to using citations in Economics for a data application, which is equivalent to evaluate research only within the scientific community. In terms of the stated goal of the paper, we are back to square one. In terms of getting a better measure of citation impact, this is an interesting application of an old idea. And the resulting rankings of journals and articles look very much like those that are already available.
Showing posts with label research. Show all posts
Showing posts with label research. Show all posts
Friday, June 10, 2011
Wednesday, May 11, 2011
Are PhD dissertations lagging the research frontier?
A doctoral or PhD dissertation is supposed to be work that pushes the research frontier further. Obviously, not all dissertations are created equal and it is to be expected that some will push more that others. But they are all supposed to push. Well, do they? This is something that is quite difficult to measure as one needs to know where the research frontier lies and what the contribution of a dissertation is. For each dissertation, only few people can do this, and it is thus impossible to have an aggregate picture, unless you use a clever trick.
Sheng Guo and Jungmin Lee take publications in top Economics journals as the research frontier and look in which JEL categories they fall. They compare this to the JEL codes for US Economics dissertations and find a strong correlation controlling for the number of jobs available in the field. If you lag the publications by two years, the regression is just as good, which hints that dissertations react to the research frontier rather than the opposite (which is unfortunately undocumented), especially when you consider that with the long publication delays in Economics, the journals are in fact a few years behind the research frontier.
My interpretation here is not quite that of the authors, who really want to understand how students choose their field of study, given that they want to be on the job market with research on a hot topic. But when they start working on it, they do not know yet what will be hot. I am not sure this is quite such a conundrum, as seminars and conferences already give quite a good picture, and working papers as well. But it still looks like dissertations follow the trends instead of creating them.
Sheng Guo and Jungmin Lee take publications in top Economics journals as the research frontier and look in which JEL categories they fall. They compare this to the JEL codes for US Economics dissertations and find a strong correlation controlling for the number of jobs available in the field. If you lag the publications by two years, the regression is just as good, which hints that dissertations react to the research frontier rather than the opposite (which is unfortunately undocumented), especially when you consider that with the long publication delays in Economics, the journals are in fact a few years behind the research frontier.
My interpretation here is not quite that of the authors, who really want to understand how students choose their field of study, given that they want to be on the job market with research on a hot topic. But when they start working on it, they do not know yet what will be hot. I am not sure this is quite such a conundrum, as seminars and conferences already give quite a good picture, and working papers as well. But it still looks like dissertations follow the trends instead of creating them.
Saturday, April 30, 2011
On the ethics of research cloning
Even though the Journal of Economic Perspectives recently went open access, a move the American Economic Association should be applauded for, I am still receiving physical copies. It is a nice journal to read while lounging in the garden or on a plane ride. The last issue has as usual a good set of interesting articles, including one I had reported on earlier when it was still a working paper. But while checking what I had said about it, I noticed something rather odd: the paper I discussed was ultimately published in the Journal of Economic Behavior and Organization. I had to investigate.
The two papers are by Bruno Frey, David Savage and Benno Torgler. They both report on the sinking of the Titanic and discuss the characteristics of the passengers who survived versus those who perished. Both papers come to the same conclusions. The texts are different, though, and the published regressions are slightly different, with no explanation why, because there is no reference to the other paper. One has therefore to read in much detail to understand what the contribution of each paper is, if there is any.
All this is very fishy. It really looks like the authors are playing games here, trying to get multiple publications out of the same work. They do not mention the other work to fool editors and referees into thinking these are original contributions, as required for any submission to those journals. They tweak the results and rewrite the text so that they cannot be accused of blatant self-plagiarism. This is unethical behavior, but it is not unheard of in the profession.
But like a late-night infomercial, there is a bonus. Looking at the author's CVs, I notice that they have a third publication with the same topic and results, in the Proceedings of the National Academy of Sciences. Bruno Frey has also published two short pieces in German in magazines prior to the academic publications: 1, 2, both pdf.
Now, who are the authors? David Savage is a PhD student at Queensland University of Technology. He must have been following orders of the more senior authors, either without realizing their unethical behavior or watching in horror and not being able to do something about it. Let us give him the benefit of the doubt. His adviser is Benno Torgler, who has already an impressive track record for someone whose first refereed publication was in 2002. His RePEc profile lists 105 working papers and 52 journal articles. Looking at the published works, it seems to like to revisit previous papers by adding new twists to them. Nothing wrong with that, but it may explain why there is no major hit in the publications. There is simply too much slicing and no single slice is a major contribution worth a good publication. But early in his career, he published a series of articles on tax morale using the World Values Survey. Using the same data and the same methodology, he managed to publish several articles whose distinguishing feature is only that they look at a different set of countries: Asia, transition countries, Canada, Latin America, and possibly more. While I must confess that I have not read the papers in detail, there is simply too much material, and Benno Torgler may be innocent, I still find these patterns very disturbing.
It took me some time to figure out where Benno Torgler earned his doctorate. It is at the University of Basel, under the supervision of René Frey (Basel) and Bruno Frey (Zurich), who are brothers, after undergraduate studies at the University of Zurich. Which bring us to Bruno Frey. He is a researcher of international recognition, mostly for his work on welfare economics, happiness research, and critiques of fundamental assumptions in economic models. He credits himself with over 600 published articles and books, an astounding number in Economics. Of course, if this number comes about by slicing papers or republishing known results as described above, this number is less surprising. Looking at his list of major articles, one can surely suspect something is not quite right. I do not have the time (or the will) to go all of this, but there is indeed a lot of rehashing the same themes, which is OK when one uses new data sets or new approaches. But seeing those quantities, that seem unlikely.
Another aspect that I find disturbing in Bruno Frey's record is that his recent work has been railing against the tendency of academics (and especially their administrators and grant makers) to look for quantifiable evidence of their productivity, what he calls "evaluitis." He writes against the pressure to publish and the prominence of rankings of research output. I have reported about some of this writing myself (1, 2, 3). But again he seems to be repeating himself a lot, even in published articles, essentially criticizing a game that he seems to be excelling at. Either he is sarcastic or hypocritical, I cannot decide.
I realize the accusations I am making here can have severe consequences. But I am only accusing, not condemning. I leave the reader the opportunity to make her own opinion, as I have linked to plenty of evidence. I hope to be proven wrong, that these three individuals are indeed extremely innovative and productive. But from what I have seen so far, my prejudice is strongly negative in this regard.
Update (Sunday): I have been alerted that there is a fourth publication about the same Titanic study, in Rationality and Study.
Further update: A follow-up post.
The two papers are by Bruno Frey, David Savage and Benno Torgler. They both report on the sinking of the Titanic and discuss the characteristics of the passengers who survived versus those who perished. Both papers come to the same conclusions. The texts are different, though, and the published regressions are slightly different, with no explanation why, because there is no reference to the other paper. One has therefore to read in much detail to understand what the contribution of each paper is, if there is any.
All this is very fishy. It really looks like the authors are playing games here, trying to get multiple publications out of the same work. They do not mention the other work to fool editors and referees into thinking these are original contributions, as required for any submission to those journals. They tweak the results and rewrite the text so that they cannot be accused of blatant self-plagiarism. This is unethical behavior, but it is not unheard of in the profession.
But like a late-night infomercial, there is a bonus. Looking at the author's CVs, I notice that they have a third publication with the same topic and results, in the Proceedings of the National Academy of Sciences. Bruno Frey has also published two short pieces in German in magazines prior to the academic publications: 1, 2, both pdf.
Now, who are the authors? David Savage is a PhD student at Queensland University of Technology. He must have been following orders of the more senior authors, either without realizing their unethical behavior or watching in horror and not being able to do something about it. Let us give him the benefit of the doubt. His adviser is Benno Torgler, who has already an impressive track record for someone whose first refereed publication was in 2002. His RePEc profile lists 105 working papers and 52 journal articles. Looking at the published works, it seems to like to revisit previous papers by adding new twists to them. Nothing wrong with that, but it may explain why there is no major hit in the publications. There is simply too much slicing and no single slice is a major contribution worth a good publication. But early in his career, he published a series of articles on tax morale using the World Values Survey. Using the same data and the same methodology, he managed to publish several articles whose distinguishing feature is only that they look at a different set of countries: Asia, transition countries, Canada, Latin America, and possibly more. While I must confess that I have not read the papers in detail, there is simply too much material, and Benno Torgler may be innocent, I still find these patterns very disturbing.
It took me some time to figure out where Benno Torgler earned his doctorate. It is at the University of Basel, under the supervision of René Frey (Basel) and Bruno Frey (Zurich), who are brothers, after undergraduate studies at the University of Zurich. Which bring us to Bruno Frey. He is a researcher of international recognition, mostly for his work on welfare economics, happiness research, and critiques of fundamental assumptions in economic models. He credits himself with over 600 published articles and books, an astounding number in Economics. Of course, if this number comes about by slicing papers or republishing known results as described above, this number is less surprising. Looking at his list of major articles, one can surely suspect something is not quite right. I do not have the time (or the will) to go all of this, but there is indeed a lot of rehashing the same themes, which is OK when one uses new data sets or new approaches. But seeing those quantities, that seem unlikely.
Another aspect that I find disturbing in Bruno Frey's record is that his recent work has been railing against the tendency of academics (and especially their administrators and grant makers) to look for quantifiable evidence of their productivity, what he calls "evaluitis." He writes against the pressure to publish and the prominence of rankings of research output. I have reported about some of this writing myself (1, 2, 3). But again he seems to be repeating himself a lot, even in published articles, essentially criticizing a game that he seems to be excelling at. Either he is sarcastic or hypocritical, I cannot decide.
I realize the accusations I am making here can have severe consequences. But I am only accusing, not condemning. I leave the reader the opportunity to make her own opinion, as I have linked to plenty of evidence. I hope to be proven wrong, that these three individuals are indeed extremely innovative and productive. But from what I have seen so far, my prejudice is strongly negative in this regard.
Update (Sunday): I have been alerted that there is a fourth publication about the same Titanic study, in Rationality and Study.
Further update: A follow-up post.
Wednesday, April 20, 2011
Should universities focus on teaching or research?
The UK is going through a rather traumatic reform of the finances of its higher education system. There is much debate, both public and among academics, on how much of the burden should be carried by the students and how. If we take as given that universities are funded by public grants, how to allocated them is still complex, in part due to the dual goals of universities: teaching and research. Research grants are now quality driven, while teaching grants are mostly quantity driven. Given that their is a budget constraint, either at the government or at the university level, any economist would tell you there is a trade-off and choices need to be made.
John Beath, Joanna Poyago-Theotoky and David Ulph discuss how universities choose whether to focus on research or teaching. Universities are funded according to a formula depending on the number of students taught, the number of academics and their research quality, which depends itself on the academic/student ratio, Every academic gets the same pay, and universities maximize an objective based on the quality-weighted volume of research and the quality adjusted number of students, where the quality of teaching is a consequence of the academic/student ratio.
It is clear there is not going to be a free lunch. A university cannot be good at both teaching and research. As one plays around with the funding formula, all sorts of equilibria can emerge. For example, if research is well rewarded and a minimum quality of research is required for it to be funded, a bifurcation occurs where a few universities concentrate on research and others on teaching. Which funding formula is best for society remains open, though.
John Beath, Joanna Poyago-Theotoky and David Ulph discuss how universities choose whether to focus on research or teaching. Universities are funded according to a formula depending on the number of students taught, the number of academics and their research quality, which depends itself on the academic/student ratio, Every academic gets the same pay, and universities maximize an objective based on the quality-weighted volume of research and the quality adjusted number of students, where the quality of teaching is a consequence of the academic/student ratio.
It is clear there is not going to be a free lunch. A university cannot be good at both teaching and research. As one plays around with the funding formula, all sorts of equilibria can emerge. For example, if research is well rewarded and a minimum quality of research is required for it to be funded, a bifurcation occurs where a few universities concentrate on research and others on teaching. Which funding formula is best for society remains open, though.
Tuesday, March 29, 2011
Which are the most efficient universities?
University rankings are not particularly useful because they only measure the output and hardly the input. And if they measure the input, having more inputs is better. This means that rankings reflect size and resources, not how well resources are used, and how much students improve from when they started their studies. This implies that any university that does not have a medical school or an engineering school starts with a disadvantage. Internationally, university rankings have become very important, to the point that, for example, France is now reversing the splitting of its large universities in field specific institutions. The new monster universities, now again covering all fields, will rank much better thanks mostly to their sheer size.
To counteract all this, you need to measure the efficiency of universities. Thomas Bolli does this for 273 universities across the world by estimating a production possibilities frontier. Unfortunately, the sole measured input it full-time equivalents of staff, while possible outputs are FTE of undergraduate and graduate students, and citation numbers. But it is a start. Universities in Switzerland and Israel appear to be very efficient (and indeed they are small are generate a good amount of research) while those in the UK seem particularly inefficient. That should fan some flames in the debate on university financing there.
To counteract all this, you need to measure the efficiency of universities. Thomas Bolli does this for 273 universities across the world by estimating a production possibilities frontier. Unfortunately, the sole measured input it full-time equivalents of staff, while possible outputs are FTE of undergraduate and graduate students, and citation numbers. But it is a start. Universities in Switzerland and Israel appear to be very efficient (and indeed they are small are generate a good amount of research) while those in the UK seem particularly inefficient. That should fan some flames in the debate on university financing there.
Friday, February 25, 2011
Are hot teachers better teachers?
It is well known that beautiful and tall people have better lives and are better paid. This is especially thought to be true in activities where skills are relatively unimportant. What about economics professors?
Anindya Sen, Marcel Voia and Frances Woolley use the hotness indicators from student evaluations at ratemyprofessor.com in Ontario and find hot economics university professors are paid a whooping 10% more than their less attractive counterparts. Not only is this a large number, it also runs counter to previous results that such effects are limited to unskilled professions. This effect is especially strong for men and not present for women, Indeed women who negotiate hard are not deemed attractive.
In addition, hotter teachers also get better student evaluations, even after controlling for all what the authors could put their hands on. For other indicators of professor productivity, it turns out that hotness affects positively women for citations, although this could be due to a few highly cited women (citation counts are always very skewed). But neither men nor women publish significantly more when hot, but they tend to attract more co-authors. I really need to be careful with my appearance.
Anindya Sen, Marcel Voia and Frances Woolley use the hotness indicators from student evaluations at ratemyprofessor.com in Ontario and find hot economics university professors are paid a whooping 10% more than their less attractive counterparts. Not only is this a large number, it also runs counter to previous results that such effects are limited to unskilled professions. This effect is especially strong for men and not present for women, Indeed women who negotiate hard are not deemed attractive.
In addition, hotter teachers also get better student evaluations, even after controlling for all what the authors could put their hands on. For other indicators of professor productivity, it turns out that hotness affects positively women for citations, although this could be due to a few highly cited women (citation counts are always very skewed). But neither men nor women publish significantly more when hot, but they tend to attract more co-authors. I really need to be careful with my appearance.
Sunday, February 6, 2011
Why should I write grant applications?
My administrators insist I should go for grants. They say it raises prestige and my research would benefit from it. I have not applied for a significant grant for quite some time for a reason: it is a horrible waste of time. I do not need grants. I do my research very well without the need for support money. All I need is a pencil, paper and a computer. I can even do without a printer. I do not need to pay for data, software and subscription, as all this is available for free (thanks to open source and open access). I do not need a research assistant as I do that much faster and better myself. And I do not need summer money as I am already well paid. In other words, I am doing just fine without grants, why should I put the time and effort into maybe getting a little money I do need, that comes with all sorts of strings attached?
My administrators do not care about the impact on my research, or my welfare for that matter. They want the overhead. They are begging for money to justify their existence. I already bring lots of money to the college by teaching many, many tuition paying and public funding attracting undergraduates. In fact, from a back of the envelope calculation, my pay should double just for that. I am already subsidizing the administrators, why would they need grant overhead? They need to feed a machinery that deals with those grants. The office of research, which manages the grants, is twenty people strong. And if I hire a research assistant among the graduate students, I have to pay his or her full tuition before anything can be assigned. I cannot hire outside the university. So why would I want to hire anyone?
In some way, the administration wants me to pay for my salary through grants, a salary I have already more than earned with teaching to overflowing classrooms. To be honest, if I were successful in obtaining grants, I would leave the university and keep everything for myself. I would then be able to concentrate on research instead of putting up with all the red tape. But most funding agencies do not accept submissions from independent researchers, so I continue doing my research without grants and try to ignore these administrators. Let them show their self-importance elsewhere.
My administrators do not care about the impact on my research, or my welfare for that matter. They want the overhead. They are begging for money to justify their existence. I already bring lots of money to the college by teaching many, many tuition paying and public funding attracting undergraduates. In fact, from a back of the envelope calculation, my pay should double just for that. I am already subsidizing the administrators, why would they need grant overhead? They need to feed a machinery that deals with those grants. The office of research, which manages the grants, is twenty people strong. And if I hire a research assistant among the graduate students, I have to pay his or her full tuition before anything can be assigned. I cannot hire outside the university. So why would I want to hire anyone?
In some way, the administration wants me to pay for my salary through grants, a salary I have already more than earned with teaching to overflowing classrooms. To be honest, if I were successful in obtaining grants, I would leave the university and keep everything for myself. I would then be able to concentrate on research instead of putting up with all the red tape. But most funding agencies do not accept submissions from independent researchers, so I continue doing my research without grants and try to ignore these administrators. Let them show their self-importance elsewhere.
Monday, January 24, 2011
How not to distribute research funds
Citation counts are often used to proxy for the quality of an article, researcher or journal. They are not a perfect measure, everybody agrees on that, but they have proven to be a useful starting point for evaluation. Sometimes they are taken very seriously, too seriously, for the distribution of funds and pay. But at least this is done within a field, as it is obvious that citing conventions and in particular frequencies differ from field to field.
Javier Ruiz-Castillo goes further in trying to infer how budget priorities should allocated across research fields by using citations counts. Of course, for this one first needs to have a good understanding of how citations are distributed. Roughly, citations are distributed following power laws with fields and subfields. This means that few articles garner a lot of citations, while many go empty (especially in Business, Management and Political Science). And if I understand the paper right, one can apply readily a multiplier to compare the citation frequencies across fields. And these multipliers then make it possible to compare researchers or research units across fields within, say, a country, as long as one assumes that an adjusted citation is equally worth citing. For example, is political science worth the same support as biomedical engineering after using these multipliers, to take two randoms fields? And the "size" of the field is important as well. Here the author makes an attempt at some definitions of size which I frankly did not understand.
That said, I wonder why I forced myself in reading this paper. First is it indigestible because it is poorly written and uses very bad analogies. Second, because trying to compare fields and use citations for the allocation of funds or prizes across then is impossible because you have no identification: in statistical speak, the fixed effects capture all the variance. You can only compare how well a field does in a country compared to the rest of the world, but this cannot measure how important the field is. You need more information than just citations.
Javier Ruiz-Castillo goes further in trying to infer how budget priorities should allocated across research fields by using citations counts. Of course, for this one first needs to have a good understanding of how citations are distributed. Roughly, citations are distributed following power laws with fields and subfields. This means that few articles garner a lot of citations, while many go empty (especially in Business, Management and Political Science). And if I understand the paper right, one can apply readily a multiplier to compare the citation frequencies across fields. And these multipliers then make it possible to compare researchers or research units across fields within, say, a country, as long as one assumes that an adjusted citation is equally worth citing. For example, is political science worth the same support as biomedical engineering after using these multipliers, to take two randoms fields? And the "size" of the field is important as well. Here the author makes an attempt at some definitions of size which I frankly did not understand.
That said, I wonder why I forced myself in reading this paper. First is it indigestible because it is poorly written and uses very bad analogies. Second, because trying to compare fields and use citations for the allocation of funds or prizes across then is impossible because you have no identification: in statistical speak, the fixed effects capture all the variance. You can only compare how well a field does in a country compared to the rest of the world, but this cannot measure how important the field is. You need more information than just citations.
Thursday, January 13, 2011
Journal editors are poor selectors of best papers
Journal editors are thought to be exceptional scholars who are capable of identifying the best papers and in particular those that will have the largest impact, typically measured by citation counts. But editors have considerable help: first peer reviewers make reports that should be informative, second authors to some extend self-select in the submission process, they would not send a paper where it has no chance of getting published. But it is difficult to evaluate whether an editor is doing a good job. However, when editors choose to put as lead paper the one they consider the best in the issue, one can in retrospect check whether these papers are cited the most. As I reported earlier, editors turn out not to be particularly good.
What about those papers that get the "best article of the year" award? Tom Coupé does a similar exercise and finds that they are more cited than the median article, and slightly more than the runner-up articles, but they are rarely the most cited in a year. You just cannot trust editors, even though they are even supported by a committee for such awards.
What about those papers that get the "best article of the year" award? Tom Coupé does a similar exercise and finds that they are more cited than the median article, and slightly more than the runner-up articles, but they are rarely the most cited in a year. You just cannot trust editors, even though they are even supported by a committee for such awards.
Tuesday, December 21, 2010
The international flow of doctorates
With globalization, the trade of goods has considerably increased. But the substitution to international trade, international migration has also increased. While the migration of low-skilled workers draws headlines, the most "globalized" are the high-skilled ones. In particular, those holding doctorates are very mobile and in particular they move frequently.
Laudeline Auriol has analyzed for the OECD the flow of doctorates across its members countries. For examples, across European countries, 15 to 30% of all doctorate holders has moved across a border over the last 10 years. This is a remarkable transformation for Europe, where language and cultural barriers were much much higher a few decades ago.
This high mobility reflects the particular labor market for doctorates. Positions and candidates are very specialized, thus often need to move far to fond a match. This is compounded by the large increase in new doctorates across the OECD: from 140'000 in 1998 to 200'000 in 2006. But this growth has been very uneven, low in Germany, France and the US, very high for some of the poorer OECD countries, and for women. Currently, the US has the most doctorates (340'000) with Germany not far behind, which explains why a professorship there requires a second doctorate (habilitation).
Unemployment rates are low, 2-3%, but it usually takes several years after graduation until a doctorate finds a stable job, and this after graduating at a much higher age than other workers. While this sort of indicates a healthy labor market, it hides considerable heterogeneity. Women and doctorates in humanities face much higher unemployment rates. The latter are much less mobile because their research (and teaching) topic is much more closely related to the local cultural context, and can thus not take advantage of international opportunities like, say. natural scientists. It is then not surprising that in some countries a fifth hold jobs that are not related to their doctorates. There is also considerable uncertainty about job security. For example, post-docs (which include temporary visiting positions) now outnumber full-time faculty at US academic institutions.
Coming back to international mobility, it is remarkable how in most counties over a fifth of doctorate holders are foreign born, over half in Canada. Half of the foreign born doctorates in the US are from Asia, and two-thirds of the graduate students as well. And in the countries that have lower proportions of foreign born considerable share has stayed abroad recently. A truly international workforce.
Laudeline Auriol has analyzed for the OECD the flow of doctorates across its members countries. For examples, across European countries, 15 to 30% of all doctorate holders has moved across a border over the last 10 years. This is a remarkable transformation for Europe, where language and cultural barriers were much much higher a few decades ago.
This high mobility reflects the particular labor market for doctorates. Positions and candidates are very specialized, thus often need to move far to fond a match. This is compounded by the large increase in new doctorates across the OECD: from 140'000 in 1998 to 200'000 in 2006. But this growth has been very uneven, low in Germany, France and the US, very high for some of the poorer OECD countries, and for women. Currently, the US has the most doctorates (340'000) with Germany not far behind, which explains why a professorship there requires a second doctorate (habilitation).
Unemployment rates are low, 2-3%, but it usually takes several years after graduation until a doctorate finds a stable job, and this after graduating at a much higher age than other workers. While this sort of indicates a healthy labor market, it hides considerable heterogeneity. Women and doctorates in humanities face much higher unemployment rates. The latter are much less mobile because their research (and teaching) topic is much more closely related to the local cultural context, and can thus not take advantage of international opportunities like, say. natural scientists. It is then not surprising that in some countries a fifth hold jobs that are not related to their doctorates. There is also considerable uncertainty about job security. For example, post-docs (which include temporary visiting positions) now outnumber full-time faculty at US academic institutions.
Coming back to international mobility, it is remarkable how in most counties over a fifth of doctorate holders are foreign born, over half in Canada. Half of the foreign born doctorates in the US are from Asia, and two-thirds of the graduate students as well. And in the countries that have lower proportions of foreign born considerable share has stayed abroad recently. A truly international workforce.
Thursday, November 11, 2010
Academia vs. university
I have recently opined on the status of academia, and I am more pessimistic about the survival of the current model in the United States than in Europe, in particular with respect to their research and teaching missions. I find it interesting to see that Bruno Frey seems to share some of this pessimism. But for somewhat different reasons.
Frey longs for the old model where the university is an untouchable and well-funded institution where academics spend their day thinking about research with little regard to what happens around them, While I think some ultra-talented researchers should have such privileges, this does not apply to the very vast majority of current university faculty whose research contributions are very marginal at best. While it important that they carry out research to get their teaching current, research should not be a resource draining focus.
Frey is afraid that his utopia of university is falling apart not because of market forces and costs, because of internal governance. He complains that research is nowadays too specialized. I am afraid this is unavoidable at the frontier, yet in Economics it is probably the least so as many publish in very different fields (and even outside of Economics). I agree with him that there is too much pressure to publish in mid-range universities where teaching should be the focus. He also claims that academia is rife with fraud, something I cannot testify to, but maybe I am naive. He finds also that academia is withering because universities admit too many students (I fully agree) and because people do not need to network within a physical university location to conduct research.
But the main reason for which academia is falling apart is, according to Frey, the emphasis ("mania") on rankings. While I agree that rankings are abused, they have a major justification: holding the university and its members accountable. Every university can (and does) claim it has the best teachers and researchers, but rankings put some realism in this. But what surprises me most in Frey's claim is that he is himself ranking obsessed to the point of venturing in unethical behavior. Indeed, he is managing editor (with his brother) of Kyklos, an old journal that has considerably lost in reputation. To improve its impact factor, he requires that accepted papers cite other articles from Kyklos...
Frey longs for the old model where the university is an untouchable and well-funded institution where academics spend their day thinking about research with little regard to what happens around them, While I think some ultra-talented researchers should have such privileges, this does not apply to the very vast majority of current university faculty whose research contributions are very marginal at best. While it important that they carry out research to get their teaching current, research should not be a resource draining focus.
Frey is afraid that his utopia of university is falling apart not because of market forces and costs, because of internal governance. He complains that research is nowadays too specialized. I am afraid this is unavoidable at the frontier, yet in Economics it is probably the least so as many publish in very different fields (and even outside of Economics). I agree with him that there is too much pressure to publish in mid-range universities where teaching should be the focus. He also claims that academia is rife with fraud, something I cannot testify to, but maybe I am naive. He finds also that academia is withering because universities admit too many students (I fully agree) and because people do not need to network within a physical university location to conduct research.
But the main reason for which academia is falling apart is, according to Frey, the emphasis ("mania") on rankings. While I agree that rankings are abused, they have a major justification: holding the university and its members accountable. Every university can (and does) claim it has the best teachers and researchers, but rankings put some realism in this. But what surprises me most in Frey's claim is that he is himself ranking obsessed to the point of venturing in unethical behavior. Indeed, he is managing editor (with his brother) of Kyklos, an old journal that has considerably lost in reputation. To improve its impact factor, he requires that accepted papers cite other articles from Kyklos...
Saturday, November 6, 2010
In the pretense of protecting me, Emerald stiffles my research
I received a rather unsettling message from Emerald Publishers the other day:
I find this very disturbing. This message is telling me that this publisher is trying to enforce my copyright while in truth it is the publisher's copyright. And it tells me that I better preemptively alert the publisher where I apply the fair-use provisions of copyright before I get automatically accused of violating copyrights on my own work.
Now looking at Emerald's Author Charter, I find another few gems:
Note that Emerald may publish your article in another journal, if it thinks it increases its dissemination (or increases the impact factor). Nothing is said about the author agreeing to it. But Emerald is also fine if you try to publish your article elsewhere, although the condition of "for your own career development" is open to interpretation.
That said, all this business with copyright on academic research is really sad. These commercial publishers try to tell us that they do their possible to disseminate research while all they is the exact opposite: they gate the research and chase down ungated versions. Let's all move to open access. Much simpler, much less costly, and much better dissemination!
As an Emerald author, you will know that Emerald is dedicated to protecting the copyright of your work. For this reason, we use the Attributor service. Attributor automatically searches cyberlockers for unauthorized copies of works or illegal hosting and then issues legally-binding takedown notices. We are increasing Attributor's searches to the full breadth of the internet, to ensure maximum copyright protection.
For this to run as smoothly and efficiently, we are asking that you provide us with (if applicable):
1. your personal website address
2. your institutional website address
3. the website address of your company
This is so we can exclude these sites from the Attributor searches, whilst protecting your copyright. Upon provision of this information, we will of course ensure full data protection.
We look forwards to hearing from you.
I find this very disturbing. This message is telling me that this publisher is trying to enforce my copyright while in truth it is the publisher's copyright. And it tells me that I better preemptively alert the publisher where I apply the fair-use provisions of copyright before I get automatically accused of violating copyrights on my own work.
Now looking at Emerald's Author Charter, I find another few gems:
Assigning copyright of your work to Emerald allows us to act on your behalf to:
* promote your rights
* facilitate dissemination of your work by granting permissions for educational use or republication
* target other Emerald journals whose readership would benefit from access to your work
* endeavour to protect your work from any infringement of your rights which are brought to our attention.
It does NOT, in any way, restrict your right or academic freedom to contribute to the wider distribution and readership of your work. This includes the right to:
1. Distribute photocopies of your own version of your article to students and colleagues for teaching/educational purposes within your university or externally. Please note, this does not refer to the Emerald branded, published version.
2. Reproduce your own version of your article, including peer review/editorial changes, in another journal, as content in a book of which you are the author, in a thesis, dissertation or in any other record of study, in print or electronic format as required by your university or for your own career development.
3. Deposit an electronic copy of your own final version of your article, pre- or post-print, on your own or institutional website. The electronic copy cannot be deposited at the stage of acceptance by the Editor.
Note that Emerald may publish your article in another journal, if it thinks it increases its dissemination (or increases the impact factor). Nothing is said about the author agreeing to it. But Emerald is also fine if you try to publish your article elsewhere, although the condition of "for your own career development" is open to interpretation.
That said, all this business with copyright on academic research is really sad. These commercial publishers try to tell us that they do their possible to disseminate research while all they is the exact opposite: they gate the research and chase down ungated versions. Let's all move to open access. Much simpler, much less costly, and much better dissemination!
Wednesday, October 20, 2010
Copyright and the lack of competition in academic publishing
The official story is that copyright encourages creators by giving them temporary monopoly rights. The unofficial story is that copyright prevents the diffusion of art and knowledge, and nowhere is it as frustrating as with academic publishing. Commercial publishers sell the research others paid for, and can extract substantial rents because researchers have to publish in established outlets for reputation, tenure and promotion.
Giovanni Ramello remarks that there is another unfortunate consequence of copyright in academic publishing: having been granted some market power, the monopolist will seek to extend this market power through acquisitions and thereby obtain even more dominance. The obvious example is Elsevier, which has reached now a market share that should trigger anti-trust investigations along with profit margin in the order of 30%. The situation is quite bad in Economics, as scholarly societies have done little to prevent Elsevier taking hold of the major field journals, thereby making it essential to any tenure file. And given this, research libraries have no choice but subscribe to those journals, falling in the trap of the monopolist.
In other sciences, I hear the situation is not much better. And I have reported previously about horror stories that still seem to have little impact (1, 2).
In any case, journals are dead to me, for reasons cited above and also because the publishing process is broken, starting with refereeing.
Giovanni Ramello remarks that there is another unfortunate consequence of copyright in academic publishing: having been granted some market power, the monopolist will seek to extend this market power through acquisitions and thereby obtain even more dominance. The obvious example is Elsevier, which has reached now a market share that should trigger anti-trust investigations along with profit margin in the order of 30%. The situation is quite bad in Economics, as scholarly societies have done little to prevent Elsevier taking hold of the major field journals, thereby making it essential to any tenure file. And given this, research libraries have no choice but subscribe to those journals, falling in the trap of the monopolist.
In other sciences, I hear the situation is not much better. And I have reported previously about horror stories that still seem to have little impact (1, 2).
In any case, journals are dead to me, for reasons cited above and also because the publishing process is broken, starting with refereeing.
Thursday, September 2, 2010
Why so few drug innovations?
Research and development has an inherent tendency to have a decreasing growth rate. As the pool of things to discover continuously shrinks, it becomes harder to innovate. But we a groundbreaking discovery is made, this opens a lot of new opportunities and one should see a lot of new innovation. But with molecular biology and genomics, the pharmaceutical industry should have seen a burst of innovation, and in particular a jump in innovation productivity. Yet the contrary happened. One argument could be similar to the one that has been made about the productivity slowdown of the seventies, that an groundbreaking innovation like information technology needs time and resources to be understood.
Fabio Pammolli, Massimo Riccaboni and Laura Magazzini claim that this effect is very important. They observe that all the low hanging fruit have been picked in pharmacology and that first have shifted their investment portfolio towards more difficult problems. They suggest that one particular reason to do so is that improving current drugs is not profitable as generics are close substitutes and little rents can be extracted. Thus new classes of molecules are sought.
I would add another development in the field of R&D in general. It has become increasingly difficult and costly to file patents, as the field is littered with "predators" who file vague patents to prevent other from innovating, or to claim royalties. Not only does this increase the cost of innovating, it also increases its uncertainty, as any discovery can be subject to litigation even if it was a genuine discovery. This also encourages laboratories to find new molecules that are much different from existing ones.
Fabio Pammolli, Massimo Riccaboni and Laura Magazzini claim that this effect is very important. They observe that all the low hanging fruit have been picked in pharmacology and that first have shifted their investment portfolio towards more difficult problems. They suggest that one particular reason to do so is that improving current drugs is not profitable as generics are close substitutes and little rents can be extracted. Thus new classes of molecules are sought.
I would add another development in the field of R&D in general. It has become increasingly difficult and costly to file patents, as the field is littered with "predators" who file vague patents to prevent other from innovating, or to claim royalties. Not only does this increase the cost of innovating, it also increases its uncertainty, as any discovery can be subject to litigation even if it was a genuine discovery. This also encourages laboratories to find new molecules that are much different from existing ones.
Saturday, August 21, 2010
The AEA is missing a golden opportunity
The American Economic Association is asking its membership to approve a drastic restructuring of its dues. There are two reasons for this. One, the AEA is swimming in money (despite last year's fiasco with the Economists calendars) and would have difficulties maintaining its non-profit status with fiscal authorities. Two, by default members get hard copies of the journals and need to opt out to reduce their membership fee. By making the default membership without journals, the AEA hopes to save on printing costs and thus can lower the average membership fee even more.
I will vote against the change not because I dislike a decrease in the fee, but because I believe the AEA has missed here a tremendous opportunity of putting its journals in open access. This is a society with a sound financial basis that could set an example for the rest of the publishers by showing that good research should not be gated. Would this be a money losing proposition? I do not think so, first because the AEA will always have good income from its meeting registrations, and second because it would not need to maintain anymore a whole infrastructure to keep outsiders away from its journals. The AEA could probably cancel membership fees altogether and still make it work.
NB: I realize that the Journal of Economic Perspectives was recently partially put in open access. This shows that the AEA is open to the concept.
I will vote against the change not because I dislike a decrease in the fee, but because I believe the AEA has missed here a tremendous opportunity of putting its journals in open access. This is a society with a sound financial basis that could set an example for the rest of the publishers by showing that good research should not be gated. Would this be a money losing proposition? I do not think so, first because the AEA will always have good income from its meeting registrations, and second because it would not need to maintain anymore a whole infrastructure to keep outsiders away from its journals. The AEA could probably cancel membership fees altogether and still make it work.
NB: I realize that the Journal of Economic Perspectives was recently partially put in open access. This shows that the AEA is open to the concept.
Friday, August 20, 2010
The strange dynamics of faculty merit pay
In many US universities, faculty performance is rewarded with merit bonuses or increases, the initial idea being to prevent other universities from poaching the best performers. But seeing how this is approached in a very heterogeneous way across institutions, one sometimes wonders whether the merit process is done optimally.
Finn Christensen, James Manley and Louise Laurence had access to much relevant data in a large public university. There, merit pay is distributed in each department by a committee of tenure faculty. The outcome is that about two thirds of all faculty reach the highest merit scale, three fourths among tenured faculty. Now this outcome could be justifiable with additional data, which the authors have for a particular college in this university in the form of various output measures that should matter for evaluation. It turns out that untenured faculty is as productive, but still gets less merit pay. Even worse, it appears that the output measures explain about 10% of the variation in merit.
What is going on? Christensen, Manley and Laurence show theoretically that given the institutional structure, one should not be surprised by these results. I can believe that, and this is probably compounded by compression: as new faculty commands higher pay than old, the old compensate with higher merit. The authors find little evidence of this. Internal politics also certainly play a role, as you want to avoid offending someone who may be determining your merit later. But what is clear is that there is, at least at this place, very little transparency in merit attribution.
Finn Christensen, James Manley and Louise Laurence had access to much relevant data in a large public university. There, merit pay is distributed in each department by a committee of tenure faculty. The outcome is that about two thirds of all faculty reach the highest merit scale, three fourths among tenured faculty. Now this outcome could be justifiable with additional data, which the authors have for a particular college in this university in the form of various output measures that should matter for evaluation. It turns out that untenured faculty is as productive, but still gets less merit pay. Even worse, it appears that the output measures explain about 10% of the variation in merit.
What is going on? Christensen, Manley and Laurence show theoretically that given the institutional structure, one should not be surprised by these results. I can believe that, and this is probably compounded by compression: as new faculty commands higher pay than old, the old compensate with higher merit. The authors find little evidence of this. Internal politics also certainly play a role, as you want to avoid offending someone who may be determining your merit later. But what is clear is that there is, at least at this place, very little transparency in merit attribution.
Friday, August 13, 2010
What makes a great journal in Economics?
If you are editor of an Economics journal that seeks to improve and come across a paper entitled "What Makes a Great Journal Great in Economics?", you read it expecting some great recommendations, for example on how to attract great papers, generate good readership that will cite the articles and how to get established authors interested in submitting.
I supposed this is what Chia-Lin Chang, Michael McAleer, and Les Oxley were out to do, and what a disappointment it is. They take Thomson Reuters ISI Web of Science citation data, torture it in all sorts of innovative ways and come to the conclusion that for the top 40 journals in Economics, looking at the 2-year impact factor gives a distorted picture. First, how does this answer the question in the title? Second, it is already well known that these impact factors are flawed and only appreciated by publishers. Indeed, they encourage self-citations by journals, whose editors strategically require from authors to cite other articles. The two year window is ridiculously short for Economics. And the choice of journals that are indexed is difficult to follow.
I supposed this is what Chia-Lin Chang, Michael McAleer, and Les Oxley were out to do, and what a disappointment it is. They take Thomson Reuters ISI Web of Science citation data, torture it in all sorts of innovative ways and come to the conclusion that for the top 40 journals in Economics, looking at the 2-year impact factor gives a distorted picture. First, how does this answer the question in the title? Second, it is already well known that these impact factors are flawed and only appreciated by publishers. Indeed, they encourage self-citations by journals, whose editors strategically require from authors to cite other articles. The two year window is ridiculously short for Economics. And the choice of journals that are indexed is difficult to follow.
Tuesday, August 3, 2010
Business research is as out of focus as Economics
There is quite a bit of soul searching in Economics, as the current crisis has highlighted that some research areas had been covered less than optimally. Political science also has its crisis, following severe cutbacks in US federal research grants. It turns out business research is also in introspection mode.
Maggie Geuens claims that business research is not useful anymore. Specifically, the author tries to make five points:
This could also have been written for Economics. I really struggle with this obsession about getting prestige within the profession over making useful contributions, especially among the younger generation. And unfortunately, this is more pronounced in the top departments, and these are the ones that will give us the next generations. We really need a wake-up call in the profession that economic research should primarily be about improving global welfare, not a pissing contest among academicians.
Maggie Geuens claims that business research is not useful anymore. Specifically, the author tries to make five points:
- The focus of research is other academicians, and not businesses.
- Publication is all about impact (within academics) and citations.
- Negative results never make it out.
- Having to craft a well polished paper is a crass misuse of a researcher's time.
- There is too much reliance on impact factors and citations.
This could also have been written for Economics. I really struggle with this obsession about getting prestige within the profession over making useful contributions, especially among the younger generation. And unfortunately, this is more pronounced in the top departments, and these are the ones that will give us the next generations. We really need a wake-up call in the profession that economic research should primarily be about improving global welfare, not a pissing contest among academicians.
Friday, July 9, 2010
Are academics still dysfunctional in Italy?
Academics, at least in Economics, have been largely dysfunctional in Italy for a long time. With faculty positions centrally administrated by the Ministry of Education, competitions rigged against people coming from abroad, and newcomers have first to pay their time in the purgatory (the South or the islands), there have been very few incentives for quality people to stay or return to the country. Hence a very large diaspora of Italian economists throughout the world who even have organized themselves into a separate society to try to shake things up. Things have moved, a little bit, as universities have now more autonomy in hiring, and one can wonder whether this has done any good.
Adriano Birolo and Annalisa Rosselli provide an assessment of the quality of new hires into entry-level positions around 1985, 1995 and 2005. While the number of hires seems to have substantially increased, their quality does not seem to have. While they may publish more, they do so with substantially more co-authors. where there seems to be some change is in the topics of research. Italy has always had a very strong tradition in the history of economic thought, indeed a sixth of all publications were devoted to this in the 1980s, and this emphasis seems to dwindle to the advantage of Microeconomics, which was virtually absent. So at least in terms of research topics, Italy is becoming less of an oddball.
Adriano Birolo and Annalisa Rosselli provide an assessment of the quality of new hires into entry-level positions around 1985, 1995 and 2005. While the number of hires seems to have substantially increased, their quality does not seem to have. While they may publish more, they do so with substantially more co-authors. where there seems to be some change is in the topics of research. Italy has always had a very strong tradition in the history of economic thought, indeed a sixth of all publications were devoted to this in the 1980s, and this emphasis seems to dwindle to the advantage of Microeconomics, which was virtually absent. So at least in terms of research topics, Italy is becoming less of an oddball.
Monday, May 31, 2010
How to avoid academic deadwood
Everybody who has ever worked in an academic environment has complained about academic deadwood: tenured professors who do not contribute to research and often do not contribute much to anything else as well. The tenure system is often blamed, as it makes it very difficult to fire someone for underperforming. But even if there is an incentive structure, like merit increases in pay, they clearly have less bite for old faculty who will benefit from them for a shorter time.
I have always been on the lookout for a solution to this academic deadwood problem, thus when I stumbled upon this paper by Yu-Fu Chen and Gylfi Zoega, I was very hopeful. They draw a life-cycle model with unobservable research effort, while administration and teaching are observable. The paper comes to the conclusion that only the senior professors who enjoy research will do any, and thus heterogeneity in research keeps increasing with age. Nothing new here. Where a model becomes really useful is with policy prescriptions: how could the incentive structure be changed? Is is better not to have a tenure system? But none of that. How disappointing.
Here is my take at it, but without the benefit of having worked out a structural model. 1) Tenure needs to be weakened. While it was instituted to preserve academic freedom when research could not be evaluated, we have now plenty of metrics fro research performance. Doing poorly on those should be punishable. 2) Merit raises for good performance need to increase with age to counter the fact that older faculty benefit from them for a shorter time. 3) There needs to be much more flexibility and heterogeneity in teaching loads for senior faculty, so as to balance total effort.
I have always been on the lookout for a solution to this academic deadwood problem, thus when I stumbled upon this paper by Yu-Fu Chen and Gylfi Zoega, I was very hopeful. They draw a life-cycle model with unobservable research effort, while administration and teaching are observable. The paper comes to the conclusion that only the senior professors who enjoy research will do any, and thus heterogeneity in research keeps increasing with age. Nothing new here. Where a model becomes really useful is with policy prescriptions: how could the incentive structure be changed? Is is better not to have a tenure system? But none of that. How disappointing.
Here is my take at it, but without the benefit of having worked out a structural model. 1) Tenure needs to be weakened. While it was instituted to preserve academic freedom when research could not be evaluated, we have now plenty of metrics fro research performance. Doing poorly on those should be punishable. 2) Merit raises for good performance need to increase with age to counter the fact that older faculty benefit from them for a shorter time. 3) There needs to be much more flexibility and heterogeneity in teaching loads for senior faculty, so as to balance total effort.
Subscribe to:
Posts (Atom)
