National Academies Press: OpenBook

Academic Careers for Experimental Computer Scientists and Engineers (1994)

Chapter: Appendix B: Comparing Journal and Conference Publication

« Previous: Appendix A: Surveying the ECSE Community
Suggested Citation:"Appendix B: Comparing Journal and Conference Publication." National Research Council. 1994. Academic Careers for Experimental Computer Scientists and Engineers. Washington, DC: The National Academies Press. doi: 10.17226/2236.
×

Appendix B Comparing Journal and Conference Publication

This appendix compares several different quantitative measures of the acceptance and refereeing process (i.e., timeliness and selectivity) to determine how well the different publication media achieve the goals that experimental computer science and engineers desire, how these publication media might be improved, and what these publications tell us about the success of various academic researchers.

TIME TO PUBLICATION

As mentioned earlier, long delays to publication are a major hindrance to progress in experimental computer science and engineering (ECSE).1 Long publication delays also affect recognition and can be a problem in considering tenure and promotion cases. Journal editors try their best to minimize delay but are often stymied in their efforts by referees who do not return reviews in a timely fashion. The committee examined the time from submission to acceptance and publication for a variety of leading conferences and journals.

Table B.1 presents the times from submission to acceptance to publication for some major conferences; by comparison, the times for

1  

The circulation of preprints in ECSE is extensive (assisted in rapid distribution by connections to the Internet) and testifies to a problem with timeliness.

Suggested Citation:"Appendix B: Comparing Journal and Conference Publication." National Research Council. 1994. Academic Careers for Experimental Computer Scientists and Engineers. Washington, DC: The National Academies Press. doi: 10.17226/2236.
×

TABLE B.1 Time from Submission to Acceptance and Publication of Papers for Selected Conferences and Journals

 

Time to Acceptance (months)

Time to Publication (months)

Conferencea

PLDI

2.3

7.3

ISCA

2.6

6.2

ASPLOS

2.5

7.0

SOSP

3.0

7.8

OOPSLA

2.6

7.2

SIGGRAPH

2.3

6.8

Average

2.5

7.0

Journalb

TOPLAS

24.0

32.4

TOCS

17.0

21.3

IEEE TOC

N/A

32.2

IEEE TSE

N/A

29.2

a PLDI, conference on Programming Language Design and Implementation; ISCA, International Symposium on Computer Architecture; ASPLOS, conference on Architectural Support for Programming Languages and Operating Systems; SOSP, Symposium on Operating System Principles; OOPSLA, conference on Object-Oriented Programming Systems, Languages, and Applications; SIGGRAPH, conference of the ACM Special Interest Group on Graphics.

b TOPLAS, Transactions on Programming Languages and Systems (published by ACM); TOCS, Transactions on Computer Systems (published by ACM); TOC, Transactions on Computers (published by IEEE; data for 1988 to 1992, special issues omitted); and TSE, Transactions on Software Engineering (published by IEEE; data for 1988 to 1992, special issues omitted).

journal submission and publication are much longer. Table B.1 also indicates the comparable times for two of the primary Association of Computing Machinery (ACM) journals that publish ECSE research. Notice that the conferences listed, except for ASPLOS, are annual and the journals quarterly.

For Transactions on Programming Languages and Systems (TOPLAS), it takes approximately 10 times longer to acceptance and 5 times longer to publication than conferences require, whereas Transactions on Computer Systems (TOCS) requires about 6 times longer for acceptance and 2.7 times longer for publication than conferences. (For TOCS the average was significantly reduced by the use of special issues that offered fast-track publication to selected, top-rated conference papers.)

Multiple sequences of refereeing and revision account for a large

Suggested Citation:"Appendix B: Comparing Journal and Conference Publication." National Research Council. 1994. Academic Careers for Experimental Computer Scientists and Engineers. Washington, DC: The National Academies Press. doi: 10.17226/2236.
×

TABLE B.2 Publication Time and Manuscript Revision History for TOPLAS

Revisions Required

Fraction of Submissions (%)

Time to Acceptance (months)

Time to Publication (months)

1

50

18.0

26.4

2

45

30.0

37.2

3

5

37.2

46.8

TABLE B.3 Age of References in Journal and Conference Publications

Publicationa

Fraction of References to Papers Less Than Two Years Old (%)

Median Age of Reference (years)

ASPLOS (Conference)

32

3

ISCA (Conference)

35

3

PLDI (Conference)

26

4

TOCS (Journal)

20

4.5

TOPLAS (Journal)

12

5

a ASPLOS, conference on Architectural Support for Programming Languages and Operating Systems; ISCA, International Symposium on Computer Architecture; PLDI, conference on Programming Language Design and Implementation; TOCS, Transactions on Computer Systems (published by ACM); TOPLAS, Transactions on Programming Languages and Systems (published by ACM).

part of the delay in journal publication. For example, Table B.2 presents data from three years of TOPLAS. Even if only the 50 percent of submissions that require merely one revision are considered, journal acceptance requires about 6 times longer, and publication 3.7 times longer, than the average conference, which confirms that conferences offer a substantial advantage in timeliness.

With the rapid advances in experimental computer science, timeliness is critical. One way of seeing how important timeliness is and how it is affected by publication time is to look at the citations in published papers.

Table B.3 contains data computed from looking at all the references in an entire year for TOCS and TOPLAS plus all the references in one year's ASPLOS, ISCA, and PLDI conferences. From these data, it is clear that the longer selection and publication process of journals means that they cannot be as current as conference publica

Suggested Citation:"Appendix B: Comparing Journal and Conference Publication." National Research Council. 1994. Academic Careers for Experimental Computer Scientists and Engineers. Washington, DC: The National Academies Press. doi: 10.17226/2236.
×

tions. Rather than having about 30 percent of their references to recent papers (last two years), journals have on average only about 16 percent of their references to papers this recent. The average age of a reference is also older.

One of the reasons often given (by researchers as well as university administrators) for preferring journal over conference publication is the more critical reviewing and more permanent record offered by the former. If the journal papers were more critically reviewed and therefore of higher quality, it might be expected that they would be cited significantly more frequently than conference papers.

However, this is not borne out by the data. For example, among conferences the citations to other conference papers outnumber the citations to journals by 1.6 to 1 (45 percent versus 28 percent). Although the citations among journal papers are evenly divided, this appears to reflect the much older median age of a citation. For example, among the conference papers with citations older than four years, the citations to journal and conference papers are also evenly balanced. Thus, it appears that many researchers do not perceive a significant difference in quality or importance between conference and journal publication.

It is worth noting that a positive feedback loop may have occurred in ECSE. In particular, the need for rapid publication has driven many academic experimentalists away from journal publication, perhaps resulting in a reduction of the pressure on journals to publish.2 By contrast, other fast-moving disciplines make use of journals with rapid publication times; an informal survey of materials science and biotechnology journals indicates mean publication times on the order of several months, rather than the years that characterize CS&E journals.

SELECTIVITY

Among the smaller fraction of experimental computer science researchers who favored journal publication over conference publication (approximately one-quarter of the respondents to the survey in Appendix A), nearly half listed the more critical reviewing of journals as their motivation.

Although the journal reviewing process may be more thorough,

2  

This is not to say that journal editors are satisfied with the speed with which journal articles can be refereed, revised, and ultimately published—quite the contrary is true.

Suggested Citation:"Appendix B: Comparing Journal and Conference Publication." National Research Council. 1994. Academic Careers for Experimental Computer Scientists and Engineers. Washington, DC: The National Academies Press. doi: 10.17226/2236.
×

TABLE B.4 Acceptance Rates for Selected Conferences (%)

Conferencea

Year 1

Year 2

Year 3

Average

SIGGRAPH

21

19

20

20

ASPLOS

24

30

16

23

ISCA

14

19

23

19

PLDI

17

17

17

18

a SIGGRAPH, conference of the ACM Special Interest Group on Graphics; ASPLOS, conference on Architectural Support for Programming Languages and Operating Systems; ISCA, International Symposium on Computer Architecture; PLDI, conference on Programming Language Design and Implementation.

the prestigious conferences are highly selective. Table B.4 shows the acceptance rates for the last three occurrences of several top experimental conferences (ASPLOS is biennial, whereas the others are annual).

It is much harder to determine the acceptance rates for journals, because comparable statistics are not kept and the long reviewing process makes it difficult to determine the final outcome of a paper. The committee did examine the submission and publication rates for TOPLAS and found that the average acceptance rate over a five-year period was 27 percent of the papers submitted (this includes revised papers). An informal survey of editors of other major ACM and Institute of Electrical and Electronics Engineers (IEEE) journals found acceptance rates that vary from about 25 to 35 percent. The private journals tend to be in the same range, although at the higher end.

Suggested Citation:"Appendix B: Comparing Journal and Conference Publication." National Research Council. 1994. Academic Careers for Experimental Computer Scientists and Engineers. Washington, DC: The National Academies Press. doi: 10.17226/2236.
×
Page 135
Suggested Citation:"Appendix B: Comparing Journal and Conference Publication." National Research Council. 1994. Academic Careers for Experimental Computer Scientists and Engineers. Washington, DC: The National Academies Press. doi: 10.17226/2236.
×
Page 136
Suggested Citation:"Appendix B: Comparing Journal and Conference Publication." National Research Council. 1994. Academic Careers for Experimental Computer Scientists and Engineers. Washington, DC: The National Academies Press. doi: 10.17226/2236.
×
Page 137
Suggested Citation:"Appendix B: Comparing Journal and Conference Publication." National Research Council. 1994. Academic Careers for Experimental Computer Scientists and Engineers. Washington, DC: The National Academies Press. doi: 10.17226/2236.
×
Page 138
Suggested Citation:"Appendix B: Comparing Journal and Conference Publication." National Research Council. 1994. Academic Careers for Experimental Computer Scientists and Engineers. Washington, DC: The National Academies Press. doi: 10.17226/2236.
×
Page 139
Academic Careers for Experimental Computer Scientists and Engineers Get This Book
×
Buy Paperback | $44.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The information age has grown out of the work of experimental computer science, which is dedicated to the development of new hardware, software, graphics, interfaces, and other computer system technologies. While it is important to society in this larger sense, experimental computer science has found an awkward fit in university environments. This volume examines what is special about experimental computer science and what can be done to achieve a better fit for its practitioners in the academic context.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!