Questions on Scientific Publishing from the Immunology Retreat Faculty Panel
The following questions were submitted by students and post-docs for discussion by a faculty panel at the Immunology Retreat in 2016. Because of the rich discussion, the panel did not get to address all of these questions, so we have posed these questions to the Immunology faculty and compiled a list of responses below for inclusion on the website. Thanks to everyone for their participation and great questions and discussion!
1)Can you publish a high impact paper when you don’t work on something flashy or currently popular?
It is clear that there are some “fads” in science that help to motivate the acceptance of a paper in a high impact journal. Clearly, not all manuscripts will be of broad enough interest in the field to be published in journals like Nature, Science or Nature Immunology. However, it is also clear that you can have a high impact paper in your field that is not published in one of these journals. Dr. Moore gave the example that a highly cited primary research paper on her CV from her laboratory is a J. Immunology paper, even though she has other papers published in “higher impact” journals.
Some things to keep in mind regarding “high impact” journals: These papers often have a lot of complementary work in them from multiple labs that support the main conclusion (and therefore a lot of authors and equal contributions) or they utilize a rare resource or a novel technique. Another approach for a high impact paper is to address and answer a fundamental question in a definitive way that has not been addressed before.
2)As a reviewer, what is the most common error/problem that you see and comment on?
Major conclusions are not actually supported by the data/approach utilized
Authors have provided a collection of experimental data but haven’t framed it into a compelling story which addresses an important gap in knowledge
It is always embarrassing if you claim to be the first to show something, and the reviewer points out a previous paper that you missed! Do a literature search again right before you submit to be sure!
One thing that is more readily scrutinized today is the format of data presentation. Many journals want to see data presented as scatter plots rather than bar graphs which is the proper format, especially when there is significant spread in the data. Be sure you are using statistics correctly. For example..if you have 3 groups in the experiment, don’t just analyze 2 of them by a t-test if an ANOVA is more appropriate.
People don’t make it clear whether they have repeated the experiments multiple times and are showing a representative figure vs. showing a figure combining multiple experiments.
Technical replicates vs. biological replicates are not always clearly
indicated.
3)How do you respond to picky reviewers in a polite way?
This obviously depends on what they are being picky about. In general, it is good to ask yourself whether the experiments they are suggesting will fundamentally change the message of the paper, or whether they are “add ons”. For example, if you have used a pharmacologic approach to inhibit something, it may be fair for a reviewer to ask for a “knock-down” or “genetic” approach as confirmation due to potential off-target effects of the drug. On the other hand, if they ask you to do a series of experiments in B cells because it would be interesting, but your paper is focused on T cells…then it may be appropriate to claim that those experiments are very interesting, but beyond the scope. You can always use language like “we agree with reviewer #1 that those experiments are very interesting, and are subjects of our future studies, but we feel they are beyond the scope of the current manuscript.”
If they ask you to do an experiment that you cannot (e.g. because you do not have the KO mouse etc), then you should say that. A good rule of thumb for most editors is that if the reviewer is asking for experiments that will take more than 6-9 months to complete…it is better to reject and let the author try somewhere else, or resubmit the manuscript as a new submission when the work is completed.
A good editor will hopefully have provided some guidance on what aspects of the reviews are critical for a response vs. not. If they have not done that, and you are unsure..you should consider writing to the editor to ask for help in determining which aspects of the review they feel are critical to address! This is a good rule of thumb for many of the questions raised.
4)What is the first step in writing a good manuscript?
Most people feel that you should have a pretty good idea of what the main message of the paper is going to be. You should have the figures mocked up and organized in way to be sure that each figure addresses a discrete message. This will help you identify any “holes” that require additional experimentation. Hopefully you will have also done a thorough literature search to know how to frame your results in the context of previous work (see more below). Once you have a good idea of what results you have, or will be able to generate, it is time to start drafting.
A good rule of thumb for the introduction and discussion is to start with a general outline. The introduction should introduce the main area of study (e.g. introduce the disease, statistics about its incidence, introduce innate or adaptive immunity as important for xxx). Then the introduction should have discrete paragraphs that set the stage for what is currently known and what gap in knowledge your work addresses.
The discussion should generally (briefly) summarize your main results and then expand on them. A rule of thumb is to consider each figure in order and if the results of that figure suggest broader interpretations, contradict with previous literature ,etc, that should be discussed. Some folks like to devote a paragraph to potential limitations of the study (e.g. sample sizes that are small etc).
One common error is to spend too much space merely summarizing or recounting all the results, rather than focusing on discussing what they mean, or may suggest, and how they may change our understanding.
Some people write the results and then the intro, etc
Some people like to start at the beginning and do the intro first to help them organize the narrative better.
Each paper may have its own individual “creation story” and thus, no single rule may universally apply as to what works best in this regard.
5)How do you write an effective discussion for a paper?
See above (question 4)
6)How do you become an editorial board member or a reviewer of a journal?
Generally, you must develop experience as a journal reviewer. For most students or post-docs, your PI may let you “practice” this by allowing you to independently read and write a review on a manuscript they are asked to review. Then you can compare results and talk about the similarities and differences in your reviews. Your PI can even tell the editor that you helped with the review. Once you feel you are comfortable doing this on your own, generally as a senior post-doc, you can submit your CV to the editors or associate editors of the relevant journals, write them a letter stating your expertise areas and asking if they will consider you as a reviewer for relevant papers submitted to that journal.
Once you develop expertise as a reviewer, be aware that the journals keep stats about you…whether you return your reviews on time…whether the reviews are thorough and unbiased…are they fair? If you get consistently good marks for a few years, you may find that the journal solicits you to become an editor. Editors are generally picked from the pool of strong reviewers for a particular journal.
With the plethora of new journals, it is not uncommon to be “solicited” as a member of an editorial board. Be aware that some of these journals are “predatory”. Be cautious or ask for advice before agreeing to do this.
7)If two different reviewers have two different opinions about one experiment, how do you decide which one to consider?
This will most likely be dictated by how you decide to respond on aggregate, or which reviewer you agree with more. It is okay to respond to the journal and say….”reviewer #1 and #2 differed in their suggestions regarding figure 2. Thus, it was difficult for us to interpret the preferred course of action. After careful consideration, we have decided to address these concerns by doing…xyz…in line with the comments of reviewer #1.”
This is another situation where as an author you can contact the editor for advice/assistance.
8)Do editors use the reviewers that are suggested by authors or not? Do they exclude reviewers if you ask them too?
Most editors will use a suggested reviewer if they agree they are an expert in the field and have no reason to suspect bias.
Most journals will honor a request NOT to use a non-preferred reviewer, especially if the cover letter states something like a conflict of interest, or that you are working on identical questions and are competitors.
9)What are the merits of holding data in hopes of publishing one big paper versus publishing multiple shorter reports? What factors go into that decision?
This was discussed at the retreat. There are multiple considerations. A high impact paper is definitely a positive factor on your CV to make you competitive for jobs, grants etc. If it is reasonable to shoot for this type of paper, then it is a good idea to try.
Some faculty felt you should always aim to submit to a journal likely to reject it before moving down the list. You could benefit from the reviews to make it a better paper.
Timing is important…if a student needs to graduate to be able to move to a post-doc…if the ability to get needed mice or reagents is very long…it may not be reasonable.
Students may benefit from the experience of writing multiple papers vs. one.
This should be a topic of discussion with your mentoring committees!
There is no one rule that will always guide what to do; each paper must be individualized with consideration of what is likely to fly, the needs and urgency of both the trainee and the PI, whether other pieces of data COULD stand alone if held out for a separate paper, how feasible (and how long would it take) to wait for that big piece, etc.
10)What is the hardest you have fought to have a paper published and why and how did you do it?
This was also discussed at a retreat, and the answers varied. Dr. Lundy related a story about wishing he had fought harder to get a review on a novel topic published in a higher impact journal initially. Dr. Moore related a story about the challenges of combining data from 2 labs and 2 smaller papers to submit to JCI and how long it took to do all the requested revisions. Other people related stories of writing rebuttal letters to the editors when they felt the reviews were not objective or wrong.
Bottom line is that it may be worth a try IF you have a compelling reason – not merely a wounded ego.
11)How do you review a paper from someone you know without being biased?
As a reviewer, you check a box on the review form saying that your review is free of bias. If you cannot review a paper fairly, you should claim a conflict of interest and decline. Everyone is biased however (both positively and negatively). If you have not published with someone in ~3-5 years, you are generally considered “eligible” to review their work. Just ask yourself at the end of the review if your comments are fair and justified and if they make the paper substantially better if addressed.
12)What sequence do you write a paper in?
Personal preference...see question 4 above.
13)During the rebuttal process, how should one approach addressing labor-intensive follow-up experiments? Submit elsewhere or rebut with a response that would circumvent the need for them?
This will depend greatly on what is needed. Will the experiments fundamentally change the message of the paper? Are they required to solidify the conclusions? If so…you probably should consider them if you can afford the time and resources to do so. If the answer is that they don’t add much…rebut…if they are impossible to do, but important, you will have to consider another journal or finding a way to accomplish them.
14)How do authors determine which contributors deserve co-authorship vs. acknowledgements.
In general, people who generate experimental data that appears in the paper should be considered as authors. Someone who materially helps design the experiments or interpret the results, helps with drafting and editing should also be considered for authorship. Someone who provides a reagent (e.g. shares an antibody or a mouse) is more likely to deserve acknowledgement, rather than authorship…unless this is the first description of the reagent you are using. Summer students or undergraduates are a grey area. If they do a large amount of work, they should be included. If they are helping with just a few PCR reactions, acknowledgment is probably more appropriate. If they help with writing or editing…authorship is warranted.
15)What is the best way to handle a situation where your results differ from previously published literature?
Generally in the discussion, you should state clearly that your results differ from previous results. Offer your best insight into why that may be….strain differences, differences in methods, etc. If your data explain the difference, highlight that… if the results are discrepant and it is unclear why…it may be best just to state that.
16)How do you decide when to wrap up experiments and submit a paper?
What are the goals you are trying to achieve? High impact papers take a long time to generate data and to revise (~1 year at minimum on revisions is common). If you are limiting in doing more experiments due to time or funding…best to wrap up, assuming you have a logical and compelling story to tell with the data in hand.
17)Can you suggest ex-collaborators as potential reviewers? Is there a time limit after which this is ok?
Generally if you have not published with someone in 3-5 years, you can suggest them as a reviewer. However, that person may feel a conflict of interest still exists and they will have the chance to recuse themselves from a review of your work if they feel that way. You cannot suggest a reviewer from your same institution. Most people never feel comfortable suggesting their former mentor…hopefully because you will always have an on-going relationship with that person to some extent!
18)How do you handle a reviewer that is categorically incorrect?
We respectfully disagree with reviewer #1 regarding issue X because….there is published data that says he/she is wrong….
Alternatively, We are sorry for the confusion. It appears the reviewer is asking about x; however, our intention was to clarify y. We have revised the manuscript to be more clear on this point.