Get Rejected on a Regular Basis

As an interdisciplinary researcher and family demographer, I have been frustrated by the lack of data to answer some of what I believe are the more important research questions. As a tenure-track faculty member at a research intensive university, my (federal) grantsmanship expectations are high if I want to get promoted to full, or to have the money to get resources for my grad students and myself. Thus, I thought I would marry the two problems – write grants to collect data to answer research questions I am currently unable to answer with available population data. However, it is very competitive to get federal grants, and I wanted to write a post for everyone who is constantly trying to get grants about my experience, and with the encouragement to keep trying!

National Couples’ Health and Time Study (NCHAT)

In 2015, I went to a conference at the fabulous Center for Family and Demographic Research at Bowling Green State University titled: Same-Sex Couples: Frontiers in Measurement and Analysis. As I heard about the lack of population data on same-sex couples, I also started to think about how it had been since 1988 that population researchers had a study focused on family functioning in the US, the National Survey of Families and Households. I decided I could write a grant to for a new study focused on family functioning in the US that would include both same and different-gender couples. I wrote the grant, and submitted it to NIH in February of 2015 as an R21. It was scored, which meant that it was in the top 50% of grants and was actually discussed by the review committee, so I revised and resubmitted in November. It was scored again, with a better score, so I submitted it again in 2016. Because NIH only allows one revision, this third submission was technically a “new” submission. Although it was scored again this third time, the score was worse, and it was apparent that I was not going to be funded by the study section I was sending the grant to. My program officer also pointed out that because I was being reviewed with R01s, but I had half the space with my R21 to explain and justify my scientific premise and decisions. I rewrote it as an R01, and tailored it for a new study section. I submitted it in 2017. It scored well at the new study section, so I revised and resubmitted it. It scored even better, and was finally in the fundable range for a new investigator, which at NIH is someone who has never held an R01. After about three and half years, and 5 submissions, I finally got funded! We are getting ready to go in the field, and the study is called the National Couples’ Health and Time Study (NCHAT).

History of Submissions and Award

Submission Date

Mech-anism Revision Grant Title Study Section Score Percentile

Summary Statement Quote

2/19/2015 R21 Pilot Study for the National Study of Family Dynamics SSPB 46 38 The reviewers agreed that this is a highly significant area of research but that the weaknesses in the approach reduce the overall impact of the project to a moderate level.
11/16/2015 R21 Yes Pilot Study for the National Study of Family Dynamics SSPB 33 27 Overall, the majority of the panel agreed the significance of improving research on same-sex families outweighs any minor weaknesses, and the project will have a high impact on fields of sociology and research methods.
6/9/2016 R21 Pilot Study for the National Study of Family Dynamics SSPB 50 50 Overall, in balancing the strengths of the significance with weaknesses in the approach, the panel agreed the project will have a moderate impact on family demography.
2/6/2017 R01 Mechanisms Underlying Sexual Minority Health Disparities in the United States HDEP 36 24 Overall, the majority of reviewers concurred that the significant application has potential for moderate scientific impact on the field.
11/6/2017 R01 Yes Mechanisms Underlying Sexual Minority Health Disparities in the United States HDEP 24 10 Overall, the reviewers agreed that the findings of this application are likely to have a high impact on the understanding of dyadic minority stress processes to address health disparities among sexual and gender minority populations. Awarded: September 14, 2018

Work and Family Life 2020 Study (WAFLS)

While I was submitting and working on the NCHAT R01, I had a second project I was also trying to get funded. In 1980, a group of researchers at the University of Nebraska (Alan Booth, Lynn White, David Johnson, John Edwards) collected data on about 2000 married individuals in the US. In 2000, Alan and David, who were now at Penn State, and their colleagues Paul Amato (who chaired my dissertation) and Stacey Rogers, collected data on a new cohort of about 2000 married individuals. Given all the speculation by academics and the media about how marriage has changed over time, I had the idea to collect data on a new cohort in 2020, and this time, because same-gender marriage is now legal, to collect data on same-gender marriage as well. Without going into too many details, I first submitted to the National Science Foundation’s Sociology Directorate in 2016. I was told the cost was too high for NSF. So, I switched to NIH. Even though I put it as my third choice, I was assigned to the same study section that had not funded NCHAT. I wish I had reached out to my program officer at the time about switching study sections, but I did not. The grant was not discussed which means that it did not score in the top 50% of applications. I did a bit of searching, and found a study section that I thought might fit well, re-framed the grant for this study section, and resubmitted. It was not discussed again. I was demoralized, but my other R01 had just been funded, so I decided to take a chance and re-submit it. This was the grant’s last chance because it would not be awarded until 2019, and data collection needed to start in 2020. The resubmission got a score of 20 and a 4%, my best scoring and percentile ever! Thus, three years after it’s first submission, this grant was just awarded as well.

History of Submissions and Award

National Science Foundation

Submission Date Division Grant Title Program Reviewer Ratings Program Officer Notes
8/15/2016 Social and Economic Sciences Work & Family Life Study 2020 Cohort: Examining Change in Marital Functioning among Different-Sex Spouses over 40 Years, and Benchmarking Marital Functioning among Same-Sex Spouses Sociology Good, Very Good, Good Despite the value of such a survey, particularly as it relates to family life and same-sex marriage, the reviewers agreed that the cost of the project was out of reach for NSF. At the end of its deliberations, the Sociology Advisory Panel recommends that this proposal be declined.

National Institutes of Health – National Institute on Aging

Submission Date Mech-anism Revision Grant Title Study Section Score Percentile Summary Statement Quote
6/5/2017 R01 A Life Course Examination of Marital Functioning and Health among Individuals in Same and Different-Sex Marriages SSPB Not discussed
2/1/2018 R01 The All-or-Nothing Marriage? Marital Functioning and Health Among Individuals in Same and Different-Gender Marriages SPIP Not discussed
11/5/2018 R01  Yes The All-or-Nothing Marriage? Marital Functioning and Health Among Individuals in Same and Different-Gender Marriages SPIP 20 4 Following the discussion, the panel agreed that the application’s high significance and numerous strengths resulted in a study with a high potential impact on the field of close relationships and health.
Awarded: August 15, 2019

“If you aren’t getting rejected on a regular basis, you aren’t taking enough risks”
– Audra Teel and Erica Robeen (my sisters)

My sister Erica Robeen told me this quote, which she attributed to my sister Audra Teel. These rejections were not easy. Sometimes when I am feeling really demoralized I lay on the floor in my office in my shame spiral for about 5 minutes then get up and keep going, and I definitely did that after receiving some of these scores/non-scores/reviews. But I decided awhile ago to not let imposter syndrome win, and to not give up because the worst thing that could happen was rejection, and I had been there, done that. I already posted recently about how white cis-men are primarily in charge of our nation’s biomedical and social science health research agenda, and I think it is because they are more likely to attribute bad reviews to external factors, and just keep submitting, whereas women, persons who are not cis-men, and persons of color are more likely to attribute it to some kind of internal factor. I hope my experience can give everyone who has doubts about their abilities some hope. I will also mention that throughout this process, I went to lots of grant writing trainings, followed grant writing advice, had my grants reviewed by professional grant writers (David Morrison), colleagues who were external to Ohio State (Laura Argys and Robert Pollak), colleagues who were at Ohio State (Dean Lillard, Natasha Slesnick, and Ann O’Connell), my co-investigators and consultants on the various grants (Wendy Manning, Gary Gates, Hui Zheng, JaNelle Ricks, Corinne Reczek, Miles Taylor, Ben Kail, Tonda Hughes), and grad students and colleagues in writing groups or college-sponsored grant writing groups. I also had help from the Institute for Population Research’s grantswoman extraordinaire Jill Morris, who handled all the uploading and budgets, and read over everything for me for all of these submissions, plus more not listed here. I kept her busy for a lot of years, but she never complained once! Some of these activities costed money and some individuals were paid for their services, and these monies came from either myself or my college. I also want to say that both grants improved immensely throughout the revision process. What was funded was much better than what I first submitted, the comments from the study section reviewers made both studies much much better and I am grateful for their time.

I am now going up for full professor, and in preparing my dossier, I realized that I submitted 12 proposals between 2014 and now that were not funded, and two that were. Getting rejected on a regular basis paid off for me. Maybe it will for you too?

A Publishing Primer

One irritating thing about starting anything new, whether it is grad school or a job, is all of the jargon that no one seems to like to explain. So, in this post I  explain what publications are, and the publication process.

What Academics Mean When They Talk About Publications

A publication generally refers to a piece of writing that is published in a journal or in a book. Popular press (i.e. magazine, newspaper) and blog pieces generally do not count as “publications.”

It would be cool if we could get credit for publishing in these journals. photo credit: Moleskines mostly. via photopin (license)

It would be cool if we could get credit for publishing in these journals.
photo credit: Moleskines mostly. via photopin (license)

All Publications Are Not Created Equally

There are several different types of publications, and they are not all equally respected. And, further, the respect that each gets varies across disciplines. If you are in psychology, journal articles are generally most highly respected. If you are in some subdisciplines of sociology, books are most highly respected.

Journal Articles

Journal articles in peer-reviewed journals are held in highest esteem, and the more the journal the article is published in gets cited, the higher the esteem of the journal, in general.

Peer review means that the paper was reviewed by other scholars in the field, most likely professors or advanced graduate students, and the author had to respond to the reviews to get published.

Editor reviewed means that the paper was reviewed by the editor only, and the author only had to respond to editorial comments to get published.

There are also journals where authors can pay to have their papers published, and this are usually regarded as the lowest quality.

Continue reading

Do I have an effect? Definitions, Examples, and an Infographic on Causal Language

Causal language is a term that gets thrown around often in peer reviews because causal language irritates many seasoned reviewers. I knew when I started these posts on causal analysis, that I wanted to write something on causal language. I googled causal language and looked at what came up – and I could not find any lists of words that were causal, or recommendations for what to use instead. So, dear reader, this post is intended to help you 1) when you are crafting your papers, and 2) when a reviewer says to you “this paper is trying to make too many causal claims”.

First, I wanted to give you an example of a causal claim, and I thought I would find one in my own work. In looking at a few papers, I found this statement in Kamp Dush & Amato (2005): “the influence relationship happiness on subjective well-being”. Why was my use of “influence” in appropriate? Because subjective well-being could have predicted relationship happiness of course! Any time your DV can reasonably predict your IV, stay clear of causal language.

Continue reading

How Structural Equation Modeling is Ruining Family Research

photo credit: BrittneyBush via photopin cc

How I feel when I see a structural equation model of cross-sectional data with 15 latent variables predicting 3 different outcomes. photo credit: BrittneyBush via photopin cc

Look out readers! This is my first of a series of posts I am working on related to causal analysis.

About two weeks ago, I attended the National Council on Family Relations (NCFR) annual meeting. It is a long meeting – I usually get there on Tuesday, and don’t leave until Saturday. And, while at the conference, I go from 7 am until 9 pm. In fact, I didn’t even leave the hotel two days! This might explain why I went off the rails during the Q&A at the last session I went to, but what set me off was a theme I saw throughout the conference.

Here is the crux of the problem: early career scholars are so focused on fancy statistics (i.e. structural equation modeling, latent class analysis) that they 1) forget about theory and the justification of their research question, and 2) present papers so complicated that no average person can understand, and even the non-average person who has a PhD and tenure cannot understand it.  But, I do not want to lay all the blame on early career scholars – we senior scholars are the ones creating these monsters!!

In most graduate programs, students are required to take several methods courses. I was required to take at least 6 when I was in grad school, and our students are required to take at least 6 as well. Unfortunately though what is happening is that we are overemphasizing the importance of cutting-edge methods and statistics, and underemphasizing the importance of constructing coherent research questions that have strong theoretical justifications.

Why are these complicated statistics ruining family science? They are ruining family science because they are making conferences boring and incoherent, and leading to the rejection of papers from these family science scholars, and a lack of publications can make it hard for these students to get a job. I talked to several colleagues and students about this issue, and each could give me examples of presentations they went to where they could not even figure out what the research question was, or why they should care.

Continue reading

Where should I submit my paper?

I wish we could get credit for publishing in these kind of journals. photo credit: yelahneb via photopin cc

I wish we could get credit for publishing in these kind of journals. photo credit: yelahneb via photopin cc

When you are in an interdisciplinary department, deciding where to submit your paper is fun, and confusing. I already told you about the time my student and I rewrote a paper we had rejected from Demography for the Journal of Family Psychology (it was accepted). But, how do you decide which journal to submit to? What factors do you consider?  My student Sara Sandberg-Thoma have been working on a paper and discussing where to submit it, which is where the idea for this blog came from. Sara and I came up with the following tips on how to decide where to submit your paper

  1. Where are the papers you are citing published? The paper Sara and I are working on could be submitted to several journals – it crosses a few disciplines. As we were reading the paper out loud, I noticed Sara was citing a paper in a journal we hadn’t discussed that I really like – Social Science & Medicine. I suggested we submit the paper there.
  2. What is the turn-around that you want? Sara is going on the job market soon, so we wanted to submit the paper to a journal with a pretty quick turn-around time. I also considered turn-around time when I was on the tenure track – I didn’t want my paper languishing for months. I would rather get a quick reject and move on to the next journal. Social Science & Medicine, the journal we decided on, has a pretty quick turn-around time, so we thought the paper could potentially be in press before she went on the market.
  3. Where else have you published? Other journals in our field have pretty quick turn-around times – Journal of Marriage and Family (JMF), Family Relations (FR), Journal of Family Psychology (JFP) – but Sara already has two first-authored papers in JMF, so we decided to go for Social Science & Medicine because she hadn’t published there before.  I don’t think it is a good idea to have most of your papers in the same journal, though JMF is awesome, so she has a good problem.
  4. How good is the paper? I think Sara’s paper is really good, so I think she should be able to get it in a high-impact factor journal. You will sometimes hear academics talk about top-tier, second-tier, and third-tier journals. So, before Sara submitted to FR, a journal I love but that is probably second tier, I thought we should try a higher-tier journal first. Again, since she had two papers in JMF, I thought we should go for a different journal with a high impact factor – again a deciding factor for Social Science & Medicine.
  5. Who do you want to read your paper? Another factor to consider is who you want to read your paper. Would you like psychologists to read it? Maybe you should go for JFP or a psych journal. Do you want sociologists to read it? Maybe you should go for Social Forces or Journal of Health and Social Behavior. This is also important in considering who will review your paper. In my post about the demography paper that ended up in JFP, I discussed how to write your paper for different audiences. Consider your audience before you submit. Skim a few other papers in the journal. This will give you a sense of the flavor of the journal, and you can adjust your paper accordingly.
  6. What do other people think? Sarah Schoppe-Sullivan and I have a writing group with our grad students, and we always get feedback from this group. Do you think this paper is good enough for JMF? What kind of reception do you think it will get at JFP? Advice from others can really help, and can also help you see flaws in the paper that you can fix before you submit. Just don’t wait around too long for the advice! I have a colleague who is constantly seeking advice from several people, and his/her papers never get submitted, thus his/her CV is lacking – not a good situation to be in when on the tenure track. So, get some advice, then submit it!
  7. When should I shoot high? If you already have some really great publications, like Sara does, she can afford to get rejected from a high-impact journal first, then resubmit to a lower impact journal. The extra time it will take to be rejected from the higher-impact journal shouldn’t hurt her if she resubmits to a quicker turn-around lower-impact journal. However, if she had fewer publications and she was going on the job market soon, I might suggest she try for a lower impact journal that would be unlikely to reject her paper, especially if it had a quick turn-around. But, she has some wiggle room give her current publication record, so I think she can afford the risk of rejection and shoot for a higher-tier journal. So, if you have already been productive for the point at which you are in your career, then shoot high. Or, if you have tenure, why not try to submit that paper to Child Development or American Sociological Review. You can take the risk.

Continue reading

Hair Flipping and Hiring

For many years now, basically since I was in charge of my own hair, I have had long hair. Before that (circa 1984) my mom always told her friend Sandy who cut my hair to leave it no longer than shoulder length; I have three sisters and my mom did not want to mess with that much hair. I longed for long flowing locks, preferably straight and blond. Thus, ever since I have been in charge of my own hair, I have had long, wavy/straight hair, though I have not gotten around to the blond. I never thought much about my long hair, at least in terms of my career, until I went on the job market.

photo credit: rachel a. k. via photopin cc

photo credit: rachel a. k. via photopin cc

In 2004-2005, I went on the market for the first time. I was also pregnant that year, and I am not one of those women who can hide a pregnancy. I had my son in May, so at the time of my interviews in January and February, I was pretty pregnant. I ended up going on four on-campus interviews, but did not receive any of the tenure-track offers. What happened I wondered? One university in particular stood out to me. The department chair had told me several times – “you are clearly our first choice”. The chair could not have been more complimentary to me, as were others in the department. But when the chair called to tell me I didn’t get the job, the chair said the faculty vote split between me and another candidate, so they hired no one.

The next year at my annual conference, I saw the chair, whom I had really liked. I asked “what happened?” The chair proceeded to tell me about the faculty meeting where the decision was made. The chair said, in all seriousness, that one of the faculty had said “she flipped her hair too much during her job talk”, and basically implied that I was “flighty”. I do not remember if the chair used the exact word “flighty”, but basically the chair implied that several faculty members thought that I was ditzy. Reeling, I expressed shock, and the chair followed-up with “Well, I just said ‘she’s pregnant’, you need to cut her a break.”

Continue reading

How to publish your paper rejected by Demography in the Journal of Family Psychology

One of the best things about being in an interdisciplinary department is that you can publish where you want – there is not a group of “top journals” that you have to push your papers in.  My colleagues in Sociology are trying to get papers in the American Sociological Review, American Journal of Sociology, and Social Forces. My colleagues in developmental psychology are going for Developmental Psychology and Child Development. In my interdisciplinary department, the only guideline we have used is that the journal have an impact factor over 1 if possible (though this is not a hard and fast rule).

Therefore, I can send my papers where I think they best fit and get a readership. Family and intimate relationship research is particularly interdisciplinary, and as such, I have a lot of options. I have submitted papers to Demography, American Sociological Review, Journal of Family Psychology, Developmental Psychology, Journal of Marriage and Family, Journal of Social and Personal Relationships, etcetera. Of course, I do not have papers published in all of these journals, but I have tried them all at one point or another!  What is different for interdisciplinary researchers is that after a paper is rejected at one journal, such as Demography, you might want to resubmit it to another journal in a different field, such as the Journal of Family Psychology. That is the story of this paper.

Continue reading