Open Peer Review
April 16th, 2010 | kfitz
I’m hoping to spend some time talking about mechanisms for transforming peer review practices and understandings via open social publishing systems. In October 2009, I published a draft of my book, Planned Obsolescence, online for open review. The process has been extremely productive, and I’ve gotten a lot of great feedback, but it’s left me with three key questions about how to transform something like CommentPress into a viable mode of open peer review:
(1) How do we create the drive within communities of practice to participate in these reviews? I’m still amazed how many people contributed to mine, but it took a good bit of strategic planning (which is to say, begging and pleading) at the outset.
(2) How can we ensure that the reviews we’re getting through a system like CommentPress don’t lose the real strengths of conventional peer reviews — the ability of a reviewer to think synthetically about the text as a whole? There’s at least the potential in a fine-grained commenting system of losing the forest in the trees.
(3) What kinds of technological or social additions can we imagine to such a system that might help persuade review committees, publishers, provosts, etc., of the value of open review?
No doubt there are more issues as well. I’ll look forward to talking with you all there.
May 15th, 2010 at 10:51 pm
These are great questions, and look like they may be in a strategic sequence. I have seen some quite successful examples of communities of interest that meet regularly face-to-face to discuss work in progress, without mechanisms of credentialing as an immediate presence or motivation. That makes me wonder if some kind of real-time event, virtual or face-to-face, couldn’t help with (1) and (2). Particular communities of interest are already motivated enough sometimes to do things (like THATCamp, come to think of it) solely because of the value they find in trying to think synthetically in dialogue together. Some of these communities could conceivably over time develop a certain self-authorizing credibility that could start to help indirectly with (3).
May 19th, 2010 at 10:57 am
I’m quite interested in this topic, as I’m currently struggling with your step 1 (begging and pleading) for a project that is sort of similar to Planned Obsolescence.
I’m working to transform my dissertation into a book (or something book-shaped), and I’m trying to do that transformation in the public view. Phase 1 is to incrementally post the dissertation text, and invite commentary section by section by posting to twitter. It’s been an interesting process from my point of view, but hardly anyone’s reading it, as far as I can tell.
Could it be that my topic is too esoteric, or that I haven’t done enough or the right kind of begging and pleading?
In other words, I’m interested in that community aspect, more so than the validation of review committees et al., at least at this point in my career. I’m also a little more interested in doing something with value (to me if no one else) than I am in transforming academic publishing, although that would be nice too.
By the way, my project is on typography and textuality in videogames, and I’m doing it in a Drupal-powered website here: www.thevideogametext.com.