Review Analysis Suggestions
Having been through the review process many times ourselves, we know reviews on what one has struggled to create can be painful. Yet reviews are the spirit of research, as we all gain from another point of view. If others comment on our work, whether positive or negative, we must consider what they say. Hence please read the reviews carefully and do not dismiss them. Given a criticism, there are two options:
1.If the criticism is correct: Try to fix or address it. If it can’t be fixed accept and avoid it, e.g. reposition the paper to not attract that criticism.
2.If the criticism is incorrect: If an informed reviewer misunderstands, so may other readers. Change the chapter so this response is less likely
Even a reviewer "misunderstood" the chapter, there is still a problem to be addressed. Publishing is a process, so please follow the process: 1. Read the reviews carefully, 2. Improve the chapter accordingly.
The review form was based on common academic criteria from the research roadmap project (see Although you should address all issues raised by the reviewers, you can most improve your chapter by addressing its lowest rated scale, as follows:
1. Relevance is rated lowest? The chapter could better handle the reader concern: "Is this useful to me?" The reviewer did not understand the main reason why the chapter was written. What problem or issue does it address? Do a better job of outlining in the introduction section the main problem the chapter addresses. Answer this "So what?" question by stating the issue/problem and target audience initially, then conclude why this is useful.
2. Comprehensive is rated lowest? The chapter could better handle the reader concern: "How does this fit with other's work?" The reviewer felt you ignored important other perspectives. Do a better job of quoting relevant literature, having up to date references, and moving outside your own parochial perspective. Position your approach relative to other's work (how differs/is the same), and also scope it (when does it apply best/when not).
3. Validity is rated lowest?The chapter could better handle the reader concern: "Can I believe this?" The reviewer felt many statements had no substance, as simply to say something does not make it true. Validity is supported by a combination of valid research data (based on valid method), valid logic (based on valid assumptions), and/or valid past logic or data (based on valid references). Justify statements (by data, logic or past research), delete them, or turn them into suggestions (in the discussion section).
4. Well written is rated lowest? The chapter could better handle the reader concern: "Is reading this worth the effort?" The reviewer felt the chapter read poorly. If English is not your first language, engage (employ?) a native English speaker to copy-edit spelling and grammar errors. Like it or not, books are judged by appearances. Good writing also avoids unnecessary complexity: big words, long sentences and double barreled statements. So keep it simple (KIS), state points directly, consider one thing at a time, and use summary diagrams/tables help readers. Good writing improves reading efficiency by reducing reader effort.
5. Logical flow is rated lowest? The chapter could better handle the reader concern: "Does this go anywhere?" The reviewer felt the chapter was "everywhere and nowhere" A good idea sequence has a beginning, a middle and an end. A poor one rambles aimlessly and ends randomly. A chapter part should follow from the previous and lead to the next. Avoid "red herrings" (irrelevant side paths), focus the chapter on a clear goal, and match the final conclusion to the initial aim. The ideas of a chapter with a logical flow are like pearls on a string, a thing of beauty.
6. Interesting is rated low? The chapter could better handle the reader concern: "Is there anything new and exciting here?" The reviewer felt the chapter was boring and added little to knowledge of socio-technical design and social networking. Clarify what value the chapter adds. Make clear its implications (in the final section). While chapters must begin cautiously, they may conclude rambunctiously. The title, figures, and summary tables all help to interest readers.
7. Conforms to the submission guidelines and requirements is rated lowest? At this point authors should address the publishing guidelines for the book as a whole. We want a common style across chapters. See but note that biographical sketch and copyright are not required until the final (next) stage.
8. Overall contribution to handbook is rated lowest? The book is about how human/social concepts and theories can enlighten the design, use and evaluation of information systems. Does the chapter address how social and technical issues must be coordinated for success?
Finally, we as editors do not see a contradiction between work that is valid, logical and rigorous, and work that is relevant, interesting and well written. Hence we encourage authors to develop their chapters along all these dimensions, and particularly along those the reviews identified as weaker.
Editors comments
This is an interesting chapter, but as suggested in the reviews, you need to rewrite it to make it focus and flow better and create more synergy between the parts. Also you should include some more up to date references. Illustrate your points with material from a case study to demonstrate the validity of your approach.
Handbook of Research on Socio-Technical Design and Social Networking Systems
Handbook Evaluation Form
Chapter Number: STS8
Please review the enclosed chapter that has been submitted to the Handbook of Research on Socio-Technical Design and Social Networking Systemsand indicate your evaluation on this form. This is a double-blind review and neither your identity nor the author’s is revealed. It is necessary to return only the evaluation form.
Is the proposed chapter:
Relevant? (useful to others) (1-10)… 6
Valid? (research method or argument is sound) (1-10)… 8
Well written? (clear and understandable) (1-10)… 8
Comprehensive? (topic area covered, current references) (1-10)… 5
Logical flow? (good idea sequence) (1-10)… 8
Interesting? (interesting title/headings, summary diagrams/tables) (1-10)… 6
Conforms to the submission guidelines and requirements? (1-10)… ?
Overall contribution to handbook (1-10): 7
Overall Evaluation:
( ) outstanding
( ) definitely publishable
(X) probably publishable
( ) marginally publishable
( ) unpublishable
Editorial Decision
( ) accept
(X) accept after specified revision
( ) reject
COMMENTS TO THE AUTHOR:
Comments to the author are strongly encouraged. By providing constructive comments, even if the chapter is not, in your opinion, publishable, you are helping your colleagues.
The chapter doesn’t include an introduction or conclusion section. Please add an introduction which provides a bit of background information, states the significance of the topic and the original contribution to knowledge which the chapter seeks to make. It should then outline the structure of the chapter to prepare readers for what’s to come, rather than to delve right into it.
Legion of Doom... is that a US or international group?
Page 3: The first paragraph could potentially also discuss the notion of escapism, see:
Page 3, the 2nd paragraph could mention and discuss the Hacker Manifesto, see:
Page 4, Flow Theory, 2nd paragraph: Can you give examples to illustrate the two common characteristics?
Page 5, script kiddies, cyberpunk, penetration tester: add short definitions
Page 6, Acceptable Behavior Contracts and Orders: add definitions/explanations
Page 6, last paragraph ends with “a novel strategy”, but then page 7 goes on with more traditional solutions to hacking... where is the novel strategy?
Page 8, 1st paragraph: “A year ago”... when’s that?
Page 8, 2nd paragraph: “RPC”?
Page 9, last paragraph: “tools”... what tools?
At this stage, the chapter is in the form of an academic essay, there is no research data, methodology, analysis/discussion, etc. Depending on the advice of the book editors, this might not be a problem though.
The chapter should finish with a conclusion section summarising the main points, talking about limitations and constraints and hinting at future research.
The latest references in the reference list (apart from two 2007 URLs) are from 2004/2005. It looks a bit like this manuscript has been written in 2004/2005 and has just received a bit of a face-lift with the Second Life being added at the end. It requires a bit of an edit to bring the literature review up to date, and to deliver on its premise to identify: “A number of control variables ... which can be used to reduce the likelihood of people engaging in the hacking activity. Addressing the social network factors which motivate hacking provides an important early step in addressing cybercrime”
Handbook of Research on Socio-Technical Design and Social Networking Systems
Handbook Evaluation Form
Chapter Number: STS8 Cyber Security and Anti-Social Networking
Please review the enclosed chapter that has been submitted to the Handbook of Research on Socio-Technical Design and Social Networking Systemsand indicate your evaluation on this form. This is a double-blind review and neither your identity nor the author’s is revealed. It is necessary to return only the evaluation form.
Is the proposed chapter:
Relevant? (useful to others) (1-10) 5
Valid? (research method or argument is sound) (1-10) 5
Well written? (clear and understandable) (1-10) 5
Comprehensive? (topic area covered, current references) (1-10) 5
Logical flow? (good idea sequence) (1-10) 5
Interesting? (interesting title/headings, summary diagrams/tables) (1-10) 5
Conforms to the submission guidelines and requirements? (1-10) 5
Overall contribution to handbook (1-10): 5
Overall Evaluation:
( ) outstanding
( ) definitely publishable
(X) probably publishable
( ) marginally publishable
( ) unpublishable
Editorial Decision
( ) accept
(X ) accept after specified revision
( ) reject
COMMENTS TO THE AUTHOR:
Comments to the author are strongly encouraged. By providing constructive comments, even if the chapter is not, in your opinion, publishable, you are helping your colleagues.
The article is not deep enough and not really well –written. There is no introduction and no conclusion. Moreover, even if the subject is really interesting, there is no real original contribution except some state-of-the-art. I suggest the authors revise profoundly their article since at this state, it is too generic.
The authors should respond to scientific considerations in terms of methodology, models and concepts rather that writing and generic “encyclopedic” paper. They should propose their own solutions and ideas to the problem they want to handle. Even if the leads are interesting, they are in no way detailed enough to provide a concrete validation.
I suggest that the authors propose a case study and develop their own approach on that case study to validate and demonstrate it. The paper is a good introduction to the subject but nothing more. It then could be extended to a more prolific work considering the shortness of the paper.
The references should be complete.
Handbook of Research on Socio-Technical Design and Social Networking Systems
Handbook Evaluation Form
Chapter Number: STS8
Please review the enclosed chapter that has been submitted to the Handbook of Research on Socio-Technical Design and Social Networking Systemsand indicate your evaluation on this form. This is a double-blind review and neither your identity nor the author’s is revealed. It is necessary to return only the evaluation form.
Is the proposed chapter:
Relevant? (useful to others) (1-10)…9
Valid? (research method or argument is sound) (1-10)…3
Well written? (clear and understandable) (1-10)…8
Comprehensive? (topic area covered, current references) (1-10)…7
Logical flow? (good idea sequence) (1-10)…9
Interesting? (interesting title/headings, summary diagrams/tables) (1-10)…8
Conforms to the submission guidelines and requirements? (1-10)…7
Overall contribution to handbook (1-10): 7
Overall Evaluation:
( ) outstanding
( ) definitely publishable
(X) probably publishable
( ) marginally publishable
( ) unpublishable
Editorial Decision
( ) accept
(X) accept after specified revision
( ) reject
COMMENTS TO THE AUTHOR:
Comments to the author are strongly encouraged. By providing constructive comments, even if the chapter is not, in your opinion, publishable, you are helping your colleagues.
The paper aims at understanding the behavior of hackers by combining 2 theories from psychology. Based on that the authors suggest a number of measures to counteract hacking.
The authors motivate the use of the selected theories very well and put forward convincing explanations of hacking behavior in the light of these theories. From these insights they derive a number of measures against hacking that target the incentives for hacking rather than its consequences, i.e. put prevention before punishment.
At this point the paper ends abruptly and prematurely where it should have begun!
As it stands the paper is not acceptable because the suggested measures remain hypothetical. I suggest that you discuss the feasibility of these measures very thoroughly and provide some hard evidence that these measures a) can actually be put into practice and b) will have the desired effect. Otherwise the results are, although very interesting, barely more than speculation.
Chapter Number: STS8:
Please review the enclosed chapter that has been submitted to the
Handbook of Research on Socio-Technical Design and Social Networking
Systems and indicate your evaluation on this form. This is a
double-blind review and neither your identity nor the author's is
revealed. It is necessary to return only the evaluation form.
Is the proposed chapter:
Relevant? (useful to others)
(1-10) 3
Valid? (research method or argument is sound)
(1-10) 1
Well written? (clear and understandable)
(1-10) 5
Comprehensive? (topic area covered, current references) (1-10) 3
Logical flow? (good idea sequence)
(1-10) 5
Interesting? (interesting title/headings, summary diagrams/tables) (1-10) 2
Conforms to the submission guidelines and requirements? (1-10) 5
Overall contribution to handbook
(1-10) 2
Overall Evaluation:
( ) outstanding
( ) definitely publishable
( ) probably publishable
( ) marginally publishable
(x ) unpublishable
Editorial Decision
( ) accept
( ) accept after specified revision
( ) reject
COMMENTS TO THE AUTHOR:
While the topic covered by this chapter is an interesting and
important one, it's just not clear to me what the contribution is here
or where the authors claims and conclusions are coming from. There is
some synthesis of explanatory theories for hacking behavior, but no
new data are presented to justify the claims that are made. If the
intent is to provide more of a theoretical contribution, this chapter
does not go far enough in developing theory or demonstrating why the
speculation provided is (or even might be) valid.
For example, in applying Flow theory to hacking the authors say that
"the emotions reported by computer hackers are similar to those
reported by individuals experiencing flow." This is then taken as a
given, though no data or examples are provided to support the claim.
There is also a tension between positive and negative connotations of
hacking that is never resolved. The authors say early on that the word
has meant both, but they will focus on the negative. Later, though,
they use more positive examples (e.g. system testing).
In another example, the authors say in their conclusion that "flow
theory highlights the fact that a prerequisitve for the development of
hacking skills is the ease in which would-be hackers can enter the
community and gain access to the tools needed to start the flow
experience..." Again, though, it's just not clear why the authors
believe this is true. It might be, but I'm not at all convinced by
this chapter.
That Levy's classic 1984 book called "Hackers" is not cited also seemsvery odd.