The Role of Information Technology in Reviewing and Publishing Manuscripts at MIS Quarterly

Information technology (IT) has been transforming the manuscript review and publication process at MIS Quarterly, and other journals as well. What are appropriate expectations of the change that information technology will, or ought to, bring to MIS Quarterly?

Consider that an IT as simple as electronic mail, allowing attached files, enables the delivery of a manuscript and review form to all of the manuscript's referees instantaneously, even if they are all on different continents. And the reviews, when completed, can likewise be returned instantaneously. Geography no longer makes a difference. But as recently as five years ago, it was still common for these documents to be delivered by snail mail, adding weeks or even months to the process. IT has also speeded up the publication of accepted manuscripts. Although MIS Quarterly does not (yet) publish its own electronic journal, MIS Quarterly now makes available (again, using simple IT, in the form of the web and Adobe Acrobat "pdf" files) electronic "pre-prints" of all manuscripts soon after they receive final acceptance, making them available to the world instantaneously, sometimes even months before their publication in hardcopy form. Also, electronic redistribution services, such as ABI-Inform, currently make available full-text and page-image versions of past MIS Quarterly articles.

In the age of IT-enabled reviewing and publishing, would four months be too long for an author to receive the decision (i.e., accept, reject, or revise and resubmit) in a review cycle? If so, then would a review cycle still be too long if it were to take three months, or two, or even one? Likewise, would the additional time required, from the manuscript's acceptance to its publication, be too long if it were to take another six months, or three months, or even one month?

A technology-driven or technocentric view, focusing on the logistical and measurable aspects of reviewing and publishing, would see the following as goals: minimizing the review-cycle time, the time to the final decision, and the overall time to the manuscript's publication. All three of these factors are important in the reviewing and publishing process; however, at MIS Quarterly, these factors do not receive consideration as goals, but play the role of constraints. The strategic goal motivating the "business process" of reviewing manuscripts at MIS Quarterly is, instead, to advance the state-of-the-art of information systems research by contributing to the development of the research of all authors who submit manuscripts to MIS Quarterly that fall under its editorial mission – even and especially those authors whose manuscripts MIS Quarterly eventually rejects. Considering that the Quarterly publishes only 10 to 15 percent of the manuscripts submitted to it, one could justifiably say that the Quarterly is not so much in the business of publishing research as it is in the business of providing constructive and collegial feedback to authors who submit manuscripts to it, especially the 85 to 90 percent whose manuscripts ends up not publishing. A reasonable review-cycle time, a reasonable time to the final decision, and a reasonable overall time to publication for accepted manuscripts are all important; however, they have importance as operational constraints to be satisfied, rather than as strategic goals to be optimized.

To illustrate what I mean by the strategic goal of advancing the state-of-the-art of information systems research by contributing to the development of the research of authors who submit manuscripts for consideration by MIS Quarterly, I offer an annotated chronology of the events leading up to the final acceptance of the manuscript, "Understanding Computer-Mediated Discussions: Positivist and Interpretive Analysis of Group Support System Use," whose authors are Eileen M. Trauth and Leonard M. Jessup. (I have scheduled this article to be published in a future issue of the Quarterly, but an electronic preprint is currently available on the Quarterly web site.) Jessup reconstructed a chronology of events for me, to which I have added (based on my e-mail archives) some more events and some more annotations.

• April 1994. Jessup conducts the intervention where he helps to organize and then run GSS sessions at the university in this study. These sessions provide the empirical material for his research.
• Late 1994 and early 1995. Jessup writes an initial, very short, descriptive case study and toys with the idea of trying to publish it in either the Chronicle of Higher Education or Academe as an applied piece. He even goes so far as to talk with the editors at each of these publications; however, he is not willing to write or edit the paper in the short and journalistic format that they require.
• October 1995. Jessup writes a full research paper, "Exploring Gender Equity with a Group Support System: GSS Use for a High Threat Task," and sends it to me (in my role as Senior Editor for MIS Quarterly's "Special Issue on Intensive Research in Information Systems: Using Qualitative, Interpretive, and Case Methods to Study Information Technology") for an informal screening. The deadline for submitting manuscripts to the Special Issue is March 30, 1997.
• November 1995. I examine the manuscript, which (unlike its finally accepted version) describes itself as an ethnography. In my judgment, it is certainly a form of qualitative, interpretive, and case research, but it is not an ethnography. For some editors, this disagreement could provide a sufficient justification to reject the manuscript (or, because the author asked for an informal screening, simply to tell him that the manuscript would receive no further consideration); however, I feel that I may not do this. In my own publications and several conference presentations, I have advocated for information systems researchers to accept, and to do, all forms of intensive research. Now, I see a leading quantitative and positivist researcher (Jessup) actually doing qualitative, interpretive, and case research, as I advocated. For me to rebuff this effort would, in this context, be sending out the wrong signal. I am in a quandary as to what to do.
• December 1995. I give priority to manuscripts that have formally entered the MIS Quarterly review process. Jessup's manuscript also joins my annual December gridlock of activities, consisting of the International Conference on Information Systems, the end-of-the-semester rituals (final exams, papers, grading, and the necessary preparations for the new semester), and Christmas. Jessup sends me a friendly reminder. I reread the manuscript.
• January 1996. I write a two-and-a-half page letter where I offer Jessup encouragement and specific directions. One suggestion is for Jessup to add material on ethnography, possibly with the help of a co-author. Another suggestion is for Jessup to grant me permission to share his manuscript with an associate editor (one who knows ethnography) for the latter's advice. Jessup agrees.
• February, March, and April 1996. Finding an associate editor is not easy. I secure an associate editor who agrees "to do a particularly thorough job of providing some helpful pointers," partly in an effort to make up for the delay.
• May 1996. The associate editor expresses concern to me about how to convey criticism on basic issues in a constructive way. I emphasize to the associate editor, "My philosophy behind the special issue is not only to publish exemplary articles, but also to promote qualitative/case/interpretive/intensive research in IS in general." The associate editor faxes to me his or her remarks for Jessup. I forward them to Jessup, who begins revising his manuscript to implement the associate editor's suggestions.
• January 1997. I receive an e-mail from Jessup. He says he would like to take up my earlier suggestion where he finds someone to co-author his paper with him. He asks me so suggest potential co-authors. I suggest one and I tell him about another suggestion that I receive from the associate editor.
• March 1997. Jessup reports that he and his new co-author, Eileen M. Trauth, "have made substantial progress in revising the manuscript." They formally submit it to MIS Quarterly by the deadline of March 30.
• May 1997. M. Lynne Markus (my co-senior editor for the Special Issue) and I receive a total of 50 manuscripts. I eventually examine the Jessup and Trauth manuscript and ask them for a revision before I send it out to reviewers.
• June 1997. I receive the revision.
• July 1997. I ask Jessup and Trauth for another revision (some "fine tuning") before I send it out to reviewers. This one is optional, but the authors agree to do it anyway.
• September 1997. Jessup and Trauth submit the latest revision for formal review. I ask the authors to nominate candidates for associate editor and for reviewers; the authors provide nominations.
• October 1997. I select the associate editor. The AE and I decide on four persons to approach about being reviewers. In general, getting an up-front agreement is better than simply sending out the manuscript; in this case, one of the candidates never replies to any of our e-mails, so we replace this candidate with another.
• December 1997. The AE and I receive two reviews while the third reviewer tells us that he or she never received the manuscript! Because the AE would not be able to write up the AE's report until mid-January, I extend the deadline for the third reviewer to mid-January. I send an e-mail to the fourth reviewer, gently reminding him or her to do the review. I send an e-mail to the authors where I update them on these events.
• February 1997. The third reviewer still has not provided the review, but takes the initiative to contact me, to provide an explanation for the delay, and to ask for an extension, which I grant.
• March 1997. I send e-mail reminders to the third and fourth reviewers, and I send an e-mail to the authors updating them on these events. Trauth then asks, "For purposes of scheduling my paper writing, could you give us a sense of a time frame about when changes would be needed if the paper is accepted?" I reply, "I would ask you to revise the paper in 2 to 3 months from the time you receive the review package." Within a week after receiving the reminder, the third reviewer sends in the review; also, this reviewer asks the AE for feedback on the quality of the review, and the AE responds with detailed comments. By the end of March, however, the AE drops out of the process. Facing the dilemma of getting a new AE involved and "up to speed" or doing all the work of an AE by myself, I choose the latter. The fourth reviewer, despite always saying "next week" in response to reminders for his or her review, never provides the review. I have waited because this person has the qualifications to do an excellent review. The delay is excessive so I give up on the fourth reviewer.
• April 1998. I write a formal editor's letter/e-mail to Jessup and Trauth where I ask them for a fairly serious "revise and resubmit." In the first two paragraphs of my letter, I say:
Of the three reviewers' reports, I received the last one on March 17. After re-reading your manuscript and reading the three reports, my decision is to ask you to write a completely new paper. The good news is that two of the three reviewers agree that there is a publishable paper somewhere in your research (a sentiment with which I agree), where this publishable paper is just waiting to emerge. In fact, this situation reminds me of a seminar I attended in graduate school, where someone said that a research paper is sometimes (not always, but sometimes) just waiting to emerge from all of the preceding research efforts, in much the same way that Michelangelo sculpted and "released" the figure of David that was just waiting to be freed from inside that block of marble. I believe, in the situation concerning your research, there is a "David" just waiting to be freed from somewhere in all the work you have done.

The bad news is that the three reviewers have identified numerous, major shortcomings in your manuscript -- so numerous and major, in fact, that an editor could readily justify a "reject" decision. However, because I believe that you would in any event, go ahead and write a new paper based on the three reviewers' comments, I might as well ask you to submit that paper to MIS Quarterly so that MISQ can potentially benefit (hopefully, in the form of a publishable paper) from all the work (yours and the reviewers') that has been expended so far. This paper would not be a revision of the existing paper. Going back to the sculpture analogy, this is a case where one cannot simply "resculpt" the prior work; often, and in this case, I am asking you to "sculpt" anew.

The three reviews are astonishingly detailed and helpful. I describe one of the reviews, to Jessup and Trauth, as equivalent in quality to an associate editor's report.
• August 1998. I receive the revision and screen it. The authors have done a great job of addressing the reviewers' requests. I ask Trauth (who is now the first author) to add an additional section that would explicitly address the Special Issue's editorial requirements for a manuscript to spell out the assessment criteria for its research method and to apply those criteria. Trauth and Jessup agree.
• September 1998. I receive the additional section and ask for a revision of it. Trauth and Jessup agree.
• October 1998. I receive the complete manuscript, including the new section. The MIS Quarterly review administrator, who works in the MIS Quarterly office in Minnesota, sends it out electronically to the reviewers.
• November 1998. A reviewer, in order to meet the review deadline, submits what he confesses to be (where "confess" is a word that he uses) the worst review he has ever written. He apologizes profusely. I give him more time to revise his review. He writes an excellent review. Another reviewer, when sent a reminder by the MIS Quarterly review administrator, reports that she never received the revision; the cause of the problem turns out to be an incorrect e-mail address. This reviewer cannot complete the review until January because of Thanksgiving, the International Conference on Information Systems, the end-of-the-semester rituals (final exams, grading, etc.), and Christmas.
• January 1999. I receive the last review. The three reviewers like the revision. I write a letter (e-mail) to inform Trauth and Jessup of my decision: "revise."
• March 1999. The authors submit the revision. The MIS Quarterly review administrator sends it to the reviewers.
• May 1999. A reviewer contacts me and asks for a telephone conversation to discuss serious issues about the manuscript. We talk and we resolve the issues. After receiving all three reviews, I tell Trauth and Jessup that I am conditionally accepting their manuscript for publication.
• June 1999. I have a long telephone call with Trauth in which we discuss how to accomplish the revision.
• July 1999. Jessup sends me the revision. As I promised in my May letter, I send the revision to only one of the reviewers.
• September 1999. The reviewer responds to me: "This final version lives up to all my hopes for this research. I am convinced it will be well cited for years to come. The authors have done a great job. Please see the attached. I've suggested a few minor changes that will take all of about 20 minutes to perform, but which I believe will improve the manuscript." I inform the authors that my decision is "final acceptance." I ask them to make the changes that the reviewer requests and to send the final version directly to the MIS Quarterly office.
• October 1999. I send a message to ISWorld, announcing to over 2,000 people the availability of an electronic pre-print of "Understanding Computer-Mediated Discussions: Positivist and Interpretive Analysis of Group Support System Use" on the MIS Quarterly web site.

The above chronology does not mention another important factor. My co-senior editor for the Special Issue, M. Lynne Markus, reviewed and commented on every letter to Trauth and Jessup in which I made a decision (such as revise or accept). We did this for each other's letters so as to ensure an editorial consistency for the Special Issue, as well as to improve the quality of these letters overall.

What are the lessons of the case history of the Trauth and Jessup manuscript?

First and foremost, the question is not whether a review cycle should take three months, two months, or one month; the true question is whether the time spent in a review cycle is commensurate with its value and productivity (the quality of the reviews and the quality of the revised paper). An appropriate length of time for a review cycle is not fixed and invariant across all review cycles and all manuscripts, but depends on the demands of the tasks to be accomplished and the unexpected problems to be solved (e.g., screening a first draft that is promising but problematic, answering reviewers' questions, dealing with the sudden departure of the associate editor in the middle of the process, suggesting and implementing the option of procuring a co-author, etc.). The value that an author finds in the resulting reviews and editor's letter can also justify a longer-than-average review cycle; indeed, in the case of the manuscript by Trauth and Jessup, the former recently mentioned to me, regarding the overall review process, "I don't think I've ever experienced such careful attention to editing." Certainly, IT can speed up communication and rush documents instantaneously from author to editor to reviewer, but IT but does not substitute for either the human labor of thinking about and reconceptualizing theory or the group work in critiquing research ideas. The lesson is that the quality of the review process is no less important a factor than the quantity of time that the review process requires.

Second, regardless of the role of IT, there were several points in the Trauth-Jessup review process where I could have shortened the review cycle dramatically, but this would have diminished the quality of the review process and, ultimately, the quality of the paper too. I could have rejected the paper at more than one point, rather than encourage the authors to develop their research further, but this would have defeated the purpose of advancing the state-of-the-art of information systems research and engaging the authors in this effort. I could have refrained from screening the paper prior to sending it out for review, but then, this could have further increased the chances for a negative reaction ("reject") from the reviewers. The reviewers and I could have also refrained from suggesting articles and entire books for the authors to look up, thereby reducing the overall time-to-publication, but this, too, would have been counterproductive to the Quarterly's strategic goal.

Third, the use of IT for publishing and distributing an electronic pre-print of the final Trauth and Jessup manuscript is an example of an appropriate role for IT – namely, automating a physical task. The lesson here is that not all tasks in the "business process" of crafting research are physical and subject to automation. Of course, IT is able not only to automate, but also to "informate"; for an example, Trauth mentioned to me: "Len [Jessup] and I didn't know each other [at the beginning of the collaboration] and were separated by several hundred miles. In fact, we didn't meet face to face until December 97 at ICIS, after the first version of the paper was submitted. This was my first virtual collaboration. IT makes it possible to get collaborators for reasons other than physical proximity." The manuscript review process remains open to additional possible innovations in which IT informates the review process.

Fourth, I now believe that the term, "the manuscript review process," is actually a misnomer. A more accurate term would be "the research development process," because the research actually undergoes significant development in the process. Jessup describes the overall experience for himself as follows:

I felt that I had reached the limits of the research approach that I had been using and needed to try a different approach in order to better understand interactions between people when they are mediated by computer-based technologies. Needless to say, it was risky for me for a variety of reasons to change approaches. When I began this project I literally had no idea what I was doing with respect to the new research approach and methodologies I was using. I had never been trained to think this way let alone use these new research methods. Frankly, I needed help, and I got it from Allen [Lee], the reviewers, and from Eileen [Trauth]. The process took a long time, but this was necessary. We needed time and effort in order to improve the quality of the manuscript, and I needed time, lots of hard work, and a great deal of learning, in order to personally get better at this. I was pushed out into new, useful literatures in terms of both content and methods, and I bought and studied several books on research methodology that were recommended to me by Allen and Eileen. As time went by, the manuscript did indeed get better and so did I. On some level I feel like I did a dissertation all over again (perhaps an entire doctoral program), but in the end I'm better for it and am proud of the work.

As for Trauth, she says:

This entire process was a learning experience as well. What I learned was how to better articulate my intuitive, interpretive acts. It was mentally painful sometimes to bring into consciousness the process of my own creativity. But having done so, the task will be so much easier in future writing! For me, this special issue achieved its goal because it helped to bring to light the interpretive process and make it more accessible to other researchers. If I had seen a paper like this 10 or 15 years ago, the direction of my own research would have been different because I would have felt I had 'permission' to embrace this type of research. How many IS people today have an interpretive researcher locked inside who will now be released because they can now see the possibilities?" In this light, the term, "the manuscript review process," fails completely to describe what was happening. Even the alternative term, "the research development process," would appear inadequate. Perhaps the most descriptive term would be, "the researcher development process," because the authors themselves undergo transformation in the process.

Finally, regarding the question, "What are appropriate expectations of the change that information technology will, or ought to, bring to MIS Quarterly?," one's answer should focus not on IT itself, but instead, on the overall business process by which research is produced, where IT plays a supporting and facilitative, but not determining, role.

Allen Lee
Editor-in-Chief

---

To order a copy of this article, click here.

---

This page is maintained by misq@csom.umn.edu. It was last updated on March 23, 2001.