Reviews from student survey paper

Thoughts on this?

27-Aug-2019

Dear Dr. Ogilvie,

Thank you for submitting your manuscript SGPE-05-2019-0050 entitled “Factors influencing underrepresented STEM graduate students’ faculty aspirations” to the Studies in Graduate and Postdoctoral Education. I have now received reviewers’ comments, and having given them full consideration I regret to inform you that we cannot accept your manuscript for publication. The reviewers’ comments are presented below, and I hope they are helpful to you as you continue this important work.

Thank you for considering the Studies in Graduate and Postdoctoral Education for the publication of your research. I hope this disappointing outcome will not discourage you from submitting to the journal in the future.

Yours sincerely,
Dr. Karri Holley
Editor, Studies in Graduate and Postdoctoral Education
kaholley@ua.edu

Comments to Author:

Reviewer: 1

Recommendation: Reject

Comments:
This study investigates the factors that may enable doctoral students from underrepresented minorities towards an academic career in the STEM subjects which is an important contribution to theory and supervision practice. However, in the current version of this paper it is difficult to ascertain whether the results are valid and reliable and how they align with the theoretical underpinnings of social cognitive career theory. The paper would need a better conceptualisation of the literature, and alignment of theory with the methodological approach. Please state what kind of study this is, exploratory, phenomonlogical? There is no background information about the student cohorts, characterisistc about the institutions, nor a break-down of responses. It is unclear why a survey leaves out results pertaining to US White Americans, and also why International students are excluded (just on not being able to fulfill an academic career in the US? How about their intention?). Overall, it would interesting to present all of your results for comparison between the different students groups and see whether some of the factors your have found may also apply to non-URM doctoral candidates or whether there is a difference.

Additional Questions:

  1. Originality: Does the paper contain new and significant information adequate to justify publication?: This study investigates the factors that enable doctoral students from underrepresented minorities towards an academic career in the STEM subjects which is an important contribution to theory and supervision practice. However, in the current version of this paper it is difficult to ascertain whether the results are valid and reliable and how they align with the theoretical underpinnings of social cognitive career theory. The paper is therefore not suited for publication in its present form. The main points that need attention are outlined under the subheadings below.

  2. Relationship to Literature: Does the paper demonstrate an adequate understanding of the relevant literature in the field and cite an appropriate range of literature sources? Is any significant work ignored?: The literature review does not provide much depth about the already known factors and ambiguities of e.g. race, gender, minority group etc that may impact success and career choices of students after the completion of their doctorate. In fact, I found that the various concepts that may play a role are not well teased out and appear biased towards ‘racial tension’, ‘microaggression’, and ‘unfavourable institutional climate’, which I believe are all valid but not well conceptualised against the myriad of other factors that might play role and have been reported in the literature. A quick search on Google Scholar brought up the following examples (below) of what had already been tried and I would at least expect some debate around what is already known and where the gaps are in this field. Often, the complexities that may lead to certain career decisions are oversimplified by the authors and not evaluated critically against what other researchers have found. For example, page 2 line 38 “ being underrepresented can be more extreme in STEM and students report isolation, loneliness……” How does this compare to other disciplines, is this a gendered phenomenon, a minority student group phenomenon, applicable to international students versus domestic students living away from home for example? I am unclear which students were characterised as URM; it appears that international students were also part of this study but are not considered here as visa requirements prevent those students from embarking on an academic career in the US (although it might be interesting to know their challenges and hurdles compared to URM and US White American students who are also not part of the results and will be considered in a separate paper as stated by the authors on page 10).
    Examples of previous studies using SCCT and STEM and doctoral as the key words:
    Mendez, S. L., Martin Conley, V., Keith, R. S., Haynes, C., & Gerhardt, R. (2017). Mentorship in the engineering professoriate: exploring the role of social cognitive career theory. International Journal of Mentoring and Coaching in Education, 6(4), 302-316.
    Ali, S. R., Rodriguez, S. L., Kim, J. C., Xiao, M., & ISU, M. E. (2017). IINSPIRE LSAMP Career Development Study Design & Piloting: How Using Social Cognitive Career Theory Can Improve STEM Experiences for URMs.
    Curtin, N., Malley, J., & Stewart, A. J. (2016). Mentoring the next generation of faculty: Supporting academic career aspirations among doctoral students. Research in Higher Education, 57(6), 714-738.

  3. Methodology: Is the paper’s argument built on an appropriate base of theory, concepts, or other ideas? Has the research or equivalent intellectual work on which the paper is based been well designed? Are the methods employed appropriate?: The authors are proposing to investigate their phenomenon of interest through the lens of SCCT, which is based on earlier work by Bandura (1986) and further developed by Lent which needs to be made more explicit to be able to better understand the origins of this theory and why it is well suited for this study. There is no justification given at all why SCCT, nor are the different measures aligned with a conceptual model or framework to clearly see upfront which aspects form this line of inquiry. The authors use previously tested scales but do not provide information about their reliability (Cronbach’s alpha) and how the adapted scale might have changed in their hands. There is no mention that adapted survey items were piloted at all. Therefore, the trustworthiness of results is questionable. It is not necessary to include all of the different questions in the methods section, that can be done by providing the new survey in an appendix. What is important though, is to clearly describe the differences between an existing and new scale and an overview of the operationalised constructs by item number. Moreover, there is no breakdown of where the responses came come. A table relevant to demographic groups or detailed description about the cohort needs to be included (why those universities, how do they compare/contrast).
    Non-parametric testing seems appropriate for this type of data. The output for a factor analysis does not include enough detail to decide whether the four factors found are robust (see stats guides for reporting a factor analysis; include rotation, number of items, matrix etc). In table 2, a comparison has been made using the averages, but the authors state that the data is not normally distributed, therefore, medians are more appropriate. The statistical test used is not mentioned at all.
    Why were only 19 out of 24 questions included in the regression analysis? To just say the model would become “unwieldy” including all questions is not an objective justification nor based on the output of the test (p. 12 line 35 onwards).

  4. Results: Are results presented clearly and analysed appropriately? Do the conclusions adequately tie together the other elements of the paper? If the authors have applied a categorisation or classification system to their analyses, are the categories employed mutually exclusive and do they represent a common hierarchical level? (i.e. could the categories employed be conflated? Is one or more of them subsumed within another?): The outcomes do not well align with the proposed theory; this part needs to be better conceptualised. It is unclear why the teaching values construct is considered most important (p. 14 line 15 onwards) when only 2 items fall under this construct named ‘values’, one related to salary and the other to teaching. Furthermore, in table 3, values teaching and research appear to have the same loading and significance; not sure why one can be more important as the data do not suggest this (equally important).

  5. Implications for research, practice and/or society: Does the paper identify clearly any implications for research, practice and/or society? Does the paper bridge the gap between theory and practice? How can the research be used in practice (economic and commercial impact), in teaching, to influence public policy, in research (contributing to the body of knowledge)? What is the impact upon society (influencing public attitudes, affecting quality of life)? Are these implications consistent with the findings and conclusions of the paper?: The contribution to SCCT is not explained at all, neither is the significance of this study for supervision practice. For example, how would a higher doctoral completion rate among URM student groups contribute to society at large, the economy, but also to students’ communities? This is only touched on by citing one paper and not further elaborated (Jaeger et al., 2017). The implications do not point to already existing good models elsewhere or guidelines based on previous research; i.e, this section has no cross references to the literature. In other words, there is a lack of evidence and it is therefore hard to gauge which possible solutions might be better than others considering the context.

  6. Quality of Communication: Does the paper clearly express its case, measured against the technical language of the field and the expected knowledge of the journal’s readership? Has attention been paid to the clarity of expression and readability, such as sentence structure, correct application of the rules of grammar, jargon use, acronyms, etc.: At times there are incomplete sentences, especially in the abstract which needs re-wording (sentence structure and content). STEM is not defined when it first appears and URM is wrongly defined in this context. URM refers to ‘underrepresented minority…’not ‘underrepresented graduate students’. I had trouble locating the field until a bit further into the introduction.
    Consider re-phrasing PhD to doctoral candidate/student or PhD without stops. Add doctoral to key word list, and correct URM in key word list to acceptable abbreviation.

EAB Editorial comments:

This is an important topic and one I commend the authors for undertaking. I think more work in this vein is necessary to fully understand the complexities of how individuals from diverse backgrounds pursue faculty positions. That being said, I have concerns about the current state of this manuscript and I’m not confident they can be fully addressed in a revision. 1) The theoretical framework doesn’t seem to match well with the survey design, analysis, and presentation of results, which leaves me curious as to how the framework helped shape the study. 2) The manuscript isn’t especially well-suited for the international audience of the journal. The journal publishes US centric work but more care needs to be taken as to what this study means for an international audience. 3) I never got the sense of what “URM” meant in the study and how that definition was consistent through all parts of the manuscript. I was especially interested in how other aspects of an individual’s identity might intersect to shape trajectories and outcomes.

Here are some of my initial thoughts:

  • The fact that there is only 1 reviewer is interesting; that’s the first time that I’ve encountered that happening. At least it was a swift rejection though so it can be re-worked.

  • I absolutely agree that SCCT as a model has to be fully integrated. Because SCCT is so complicated and used so often, there are lots of models for how to structure blocks of variables to fit under SCCT models…we will want to be clear on what version of the SCCT model that we’re working with and how all of our variables fit. Since we’re sort of retro-fitting, this is going to be a bit more challenging.

  • Like you, I bristle and the mention of comparison to white students. I think that with increased robustness and following (and citing) models of other successful papers that don’t use the white comparison may be helpful. Of course, we will want to have robust enough data to have the ability to breakdown URM a bit if we’re going to not do a white comparison group.

  • I agree with more description of the sample; that would even help those of us looking at the paper to understand more about it. Agree with the greater call for validity, too.

  • I appreciate this reviewer’s promotion of mine and my colleagues’ SCCT work. I will say that because it’s such a complicated model, we ultimately had trouble running models due to not robust enough data.

  • I appreciate the desire of the reviewer to want clear descriptions - I think this is helpful both to this paper and to us as a NIC to talk about this. And, the question of international students is always there and should be clearly addressed in our description.

  • In terms of possible next venues, I would want to see the most current version of the paper (I have the NIC one, but not this one). I’m thinking:

  • Journal of Diversity in Higher Education

  • Journal of Women and Minorities in Science and Engineering

These are some open source ones that I’ve had great experiences with, too:

  • International Journal of Education in Mathematics, Science and Technology (Mack Shelley at Iowa State is the editor)

  • Journal Committed to Social Change on Race and Ethnicity (JCSCORE) - the NCORE journal