PLDI reviewing
I'm on the PDLI external reviewers committee this year. I've never published a paper at PLDI but I have attended. This is also my first time reviewing papers. I thought the quality of the papers I was given to review was quite high. They were also on a surprisingly broad and interesting range of topics. Unfortunately there was a inverse correlation between paper quality and how interesting I found the topic. In other words, the more radical and creative papers were in general much less well executed than the ones with more incremental results. Surprisingly, about 30% of the papers I read had no evaluation at all! I think that there are lots of valid kinds of evidence for the correctness of a result, including proofs, implementation, case studies, even subjective critique and commentary. But if the authors don't make any attempt to evaluate or critique their work, I don't see any way that a paper can be accepted today. Yes, I've published papers in the past that have little or no evaluation, but things have changed.
Subscribe to:
Post Comments (Atom)
Posts
3 comments:
Interest observation that "things have changed." Why are today's PC requiring more evaluation compared with past PCs?
A few of my papers in the early 90's, including "Interfaces for Strongly-Typed Object-Oriented Programming" and "F-Bounded Polymorphism for Object-Oriented Programming" were essentially descriptive. One of my more popular papers, "A Proposal for Making Eiffel Type-Safe" was a critique of an existing language and so you might argue it was all evaluation. I am not sure such papers could be published now, because the standards of the programming language community have evolved since then. But now we have Onward! and SPLASH Essays, which explicitly allow such papers. So not all is lost.
I believe the paper "N Degrees of Separation: Multi-Dimensional Separation of Concerns" is a highly influential and well-cited paper with absolutely no evaluation.
Post a Comment