Understanding the Dissemination of Influential Parenting Research: A Rapid Review with Article-Level Metrics Free

The systematic review has served as one of the most thorough and reliable methods of scientific literature synthesis because of its rigor and transparency (Higgins & Green, 2011). These reviews are an objective method for determining consistency of findings and generalizability within fields of research. However, the average systematic review takes over a year to complete and may require a large team of researchers, which may make them less efficient for researchers operating with limited time and staffing (Borah et al., 2017). Further, this lengthy time frame makes it difficult for systematic reviews to stay current with new research and, as a result, they are often outdated by the time of publication (Bastian et al., 2010; Roberts et al., 2015). This presents an additional challenge to those who wish to rapidly translate and disseminate the latest research findings to the public. If the results of systematic reviews are biased toward older, outdated findings, policy makers and public health professionals may not have accurate information to pass on to consumers.

One solution to overcome these limitations is the use of a rapid review. Rapid reviews omit certain steps of the systematic review process and require less time to complete, enabling faster turnaround times while retaining the distinguishing features of systematic reviews—objectivity and transparency (Haby et al., 2016; Tricco et al., 2015). While rapid reviews offer a faster method of research synthesis, they lack a consistent, refined methodology and may be prone to scientific bias. For example, rapid reviews may overemphasize poorly designed studies in the interest of producing a timely synthesis of the findings (Grant & Booth, 2009). One way to protect against this is to rely on standard metrics of high quality and influential research, including traditional metrics like number of citations and modern metrics like Altmetric scores, respectively.

Traditionally, the number of citations a study receives is a good indication of its quality and credibility in the scientific literature (Saha et al., 2003), although this is not a foolproof metric, as noted by more recent scholarship (Aksnes et al., 2019). Accordingly, traditional methods of establishing an article’s impact and quality may need to adapt, especially as more scientific literature is translated and disseminated online. Indeed, a number of such methods have been proposed in recent years. It is increasingly common for a scientific study to be described in a news article or posted and shared on social media, though not always in relation to the study’s perceived quality within the scientific community (Costas et al., 2015). This allows for more rapid measurement of the general public’s exposure and interpretation of a research article. Altmetric scores are alternative metrics of measuring a study’s influence that consider platforms like blog posts, social media mentions, and mainstream media attention when assigning a score to a given scientific work (Priem et al., 2010). The score assigned by Altmetric conveys information about the amount of attention surrounding specific studies, tracking the studies that are highly influential to the public. Further, many scientists and scholars have joined social media and their most common activity on social media sites is the sharing of their own content or linking to other scientific content (Van Noorden, 2014). Thus, the scientific community likely plays a role in the Altmetric scores of many pieces of literature through what is shared on Twitter, science blogs, and news outlets.

Examining influence in both the scientific literature and the popular press, including social media, is vital within dynamic fields such as adolescent health and character education research where there can be a disconnect between the findings from the scientific community and the broader discussion in the media. This can make it difficult for the general public, and particularly parents and educators, to determine if advice is evidence-informed or anecdotal. For example, books and news articles sometimes portray adolescence as a time of “storm and stress” that parents must survive (Kolbert, 2015), even though scientific research offers a more nuanced perspective that demonstrates how parents and teens can thrive through adolescence. Educator and character development researcher Hersh contends conversations about moral and character education arise on social media and are increasingly common, particularly on college campuses (Hersh, 2015). A rapid review which examines this intersection between science, media dissemination, and interpretation by the general public may provide character education and child-development researchers with the ability to correct misinterpretations and improve public understanding. By including the Altmetric score, researchers can evaluate the influence an article has in the mainstream media and on social media. The inclusion of both citations in the scholarly literature and Altmetrics could influence dissemination strategies.

This rapid review protocol was developed out of a need to quickly translate and disseminate foundational and emerging research findings for the Center for Parent and Teen Communication (CPTC). CPTC was established to help build youth with the character strengths that will position them to be their best selves, contribute to their communities and be prepared to lead fulfilling and meaningful lives. To accomplish this mission, CPTC disseminates evidence-informed content (e.g., articles, videos, and podcasts) posted to a website (www.parentandteen.com) and social media (@parentandteen) to support parents and caregivers to raise healthy adolescents with desirable character strengths who are poised to become successful adults. For a given parenting or adolescent development topic, content producers at CPTC develop a list of proposed article titles which CPTC’s literature review team uses as guide to rapidly search the scientific literature for studies worthy of translation and dissemination in these articles.

This article describes a novel methodology for the rapid review of influential literature using parental discipline—a critical component of character education —as a case study. We summarize the results of that case study and present an analysis of the prevailing themes about discipline in the academic literature and the popular press by examining article-level metrics, comparing and contrasting interpretation of findings by the scientific community and the general public.

Reviews were conducted by a team of three researchers at the CPTC. This team reviewed the proposed article titles for the section of the website designed to teach parents about discipline and outlined the research that would be needed to produce evidence-informed, translational articles. Search terms were generated via team discussion. Once appropriate search terms were developed, databases were searched for influential scientific literature. Table 1 displays the proposed website content about discipline and the search strategy for Scopus and APA PsycNet. The search strategy aimed to capture articles focused on parental discipline and punishment of adolescents that would form the evidence base for the proposed written content. Searches were limited to peer-reviewed scientific articles focused on adolescents. The search was also limited to articles that were written in English if this search filter was available. We defined literature as “influential” if they met at least one of two inclusion criteria during the screening process.

First, articles that met certain citation thresholds were considered highly influential within the scientific literature. Table 2 lists the thresholds for highly cited papers according to Thomson Reuters. Highly cited was defined as, “the minimum number of citations received by the top 1% of papers in the research field published in the specified year” (Thomson Reuters, 2017). Given the interdisciplinary nature of this work, citation thresholds from four research fields were used: clinical medicine, neuroscience and behavior, psychiatry and psychology and general social sciences. The average citation threshold of these four fields was used to determine if a paper should be included in the review. The 2006 citation threshold was used for any papers published before 2006.

Second, Altmetric scores were used to identify articles which have been influential outside of the scientific literature. This measure is generally only applicable to works published after 2011, as social media mentions were far less common before then (Costas et al., 2015). According to guidelines produced by the creators of Altmetrics, a score of ≥20 is considered highly influential among the lay public (“Putting the Altmetric,” 2019). APA PsycNet and Scopus were searched because these databases allow sorting articles by number of times cited and the Altmetric bookmarklet functions well within these sites. The bookmarklet is a free downloadable toolbar for web browsers that quickly calculates the Altmetric score based on the article Digital Object Identifier (DOI) listed on the webpage (https://www.alt-metric.com/products/free-tools/bookmarklet/).

Once sorted, articles were scanned for the above-mentioned search criteria. Specifically, if the article met the highly cited threshold for the appropriate year and/or had an Altmetric score ≥20 (if published during or since 2011) it was extracted. A list of extracted articles was developed and downloaded so that article titles and abstracts could be further scanned for inclusion. Traditional systematic review methods were used to determine if a study should be included or excluded from the final synthesis. Specifically, two bachelor-level independent reviewers read the extracted titles and abstracts and assessed whether or not each article should be included in the final synthesis. If reviewers differed on whether an article should be included, this was reconciled during a consensus meeting overseen by the first author. During this meeting, the relevance of each article to the proposed written content was discussed before a group decision was made about inclusion or exclusion of the article.

Two syntheses were produced to understand the findings documented in the studies included in this review. The first synthesis involved summarizing the results from highly cited studies to understand the scientific perspective on parental discipline. The second synthesis involved summarizing the results from studies with high Altmetric scores and reviewing the lay literature (i.e., newspaper articles, blogs, and social media posts) that referenced a given study. This process allowed the authors to holistically compare and contrast findings that were influential in both the scientific literature and the lay literature.

Over 2,000 articles were identified using this search strategy. Fifty-two eligible articles were included in the review after screening for article-level metrics and exclusion based on reviewer consensus (Figure 1). Article citations and article-level metrics are displayed in the Appendix. Nineteen articles had an Altmetric score greater than 20, which indicates high influence in the media (range: 0–485). Thirty-one articles were cited more than 220 times, the 2006 citation threshold for influence, in the scientific literature (range 0–904). Two articles met the citation number threshold for their respective publication year and achieved an Altmetric score greater than 20.

The highly cited studies captured in this review suggest that effective parental discipline involves setting and enforcing appropriate rules for children based on their development and maturity, implemented via reasoning, explanation, and talking with the child. For parents of adolescents, this means embracing their emerging independence while keeping them safe from danger and harmful behaviors. Parents who practice consistent discipline have children who can understand and feel a sense of fairness when facing consequences (Ge et al., 1996; Lempers et al., 1989). Conversely, research suggests that inconsistent discipline—such as, preferential treatment of different children, mood-based decisions—is associated with increased delinquent behaviors and substance use (Jensen & Whiteman, 2014; Lempers et al., 1989).

Poor parental discipline can take two forms: too permissive or too strict. Parents who are too permissive or strict in their disciplinary practices risk putting their children at an increased likelihood of delinquent behaviors, substance use/abuse, poor achievement, and depressive symptoms (Jensen & Whiteman, 2014). Children who are supervised and monitored by their parents are less likely to participate in risk behaviors like smoking and drinking (Griffin et al., 2000; Hawkins et al., 1999). Both overly harsh/punitive and permissive disciplinary practices negatively impact academic performance (Eckenrode et al., 1993; Tang & Davis-Kean, 2015).

The rate of corporal punishment declines as children grow older, according to the findings of this review (Straus & Stewart, 1999). Recent research suggests that parents understand that corporal punishment is not an effective disciplinary practice (Chen et al., 2016). Research also demonstrates that children who are physically disciplined are more likely to exhibit violent behaviors (Espelage et al., 2000) and may be at an increased risk of both being a bully and being victimized by bullies (Ehrensaft et al., 2003).

Adolescents with special needs, such as attention-deficit hyperactivity disorder (ADHD) or conduct disorders, present different disciplinary needs to parents. Mothers of children with conduct disorders were significantly more likely to be poor at supervising their children’s behavior and consistently disciplining their children than mothers of children without conduct disorders (Frick et al., 1992; Lytton, 1990). Yet, research shows that they can benefit from the same effective disciplinary practices as adolescents without special needs, especially when paired with appropriate levels of parental warmth and support (Zehe & Colder, 2014). Children with ADHD are more likely to see their ADHD persist throughout adolescence when their parents have high stress levels (Miranda et al., 2015).

Through educational interventions, parents can be taught how to discipline their adolescents more effectively. This can improve relations between parents and adolescents and promote the child’s positive character development. Risk behaviors and problems in children and adolescents can be best managed by informing parents about the effectiveness of various practices and the benefits of certain kinds of discipline and monitoring practices (Beardslee et al., 2003; Zehe & Colder, 2014).

In high-risk neighborhoods where there are high rates of community violence, parents may need to utilize different discipline practices to keep their children safe. Exposure to community violence can lead to an increase in an adolescent’s risk of aggressive behavior, including bullying (Espelage et al., 2000; Gorman-Smith & Tolan, 1998). However, greater parental monitoring and firm discipline, accompanied by reasoning and explanation, can counteract this risk, as studies show that these parenting practices are associated with less engagement in risk behaviors (Griffin et al., 2000).

Three studies with high Altmetric scores focused on disciplinary practices in families with special needs, including parents with depression and children/early adolescents with ADHD. Of these three studies, Musser et al. (2016) attained the highest Altmetric score because it was described in 34 news outlets, typically with the headline “Study Ties Parents’ Criticism to Persistent ADHD in Kids.” News stories varied in the level of detail they provided about the study and most added a qualifying statement about the study’s observational design, noting that it could not prove causation. Alternatively, the study by Miranda et al. (2015) was tweeted about 31 times reaching at most over 57,000 users. Most of these were retweets of a link to the study from users in Japan and did not offer any additional commentary on the study.

Six studies focusing on adolescent risk behaviors (e.g., fighting, sexual behaviors, and substance use) received high Altmetric scores. Chen et al. (2016) received high Altmetric scores as it was cited in 19 news outlets, most often with the titles suggesting parents play a “big” or “vital” role in “preventing teen fighting. The articles focused on the research participants’ acknowledgment that fighting prevention starts at home with education about the consequences of fighting and how to utilize nonviolent strategies to resolve problems. Calvete et al. (2015) was cited primarily on Twitter where users embedded the link and added lines such as “exposure to violence is what causes violent behavior” and “lack of parental warmth” is linked to “narcissism (& aggression toward parents).” The Altmetric score for the study by Okigbo et al. (2015) showed 11 tweets that provided links to the article with a short line highlighting the importance of parent-teen communication around sex education. The four other studies focused on adolescent substance use behaviors with Jensen and Whiteman (2014) attaining the highest Altmet- ric score with citations in 10 news outlets. Titles in the news outlets stressed that having a “favorite” child is linked to substance use in families and the articles referenced the research findings that children who felt “less favored” in “disengaged” families were four times more likely to abuse substances. Zehe and Colder (2014) was cited by two news outlets and six Twitter users. The news articles highlighted the research conclusions that when parents’ attitudes about adolescent drinking behavior relaxes over time, adolescents’ drinking behavior may increase. Finally, Ryan et al. (2010) was cited by three news outlets and three Twitter users who added links to the article in their Tweets but did not offer other information about the research.

Five studies with high Altmetric scores explored the impact harsh or punitive parenting strategies have on children and adolescents. Wang and Kenny (2014) received the highest Altmetric score of 532 with mentions from 44 news outlets and 136 Twitter users, with an estimated maximum reach of over 600,000 people. The news headlines included titles such as “Science Says Parents of Unsuccessful Kids Could Have These 6 Things in Common” and “Yelling at Teens May Be More Harmful Than You Think.” The article reached Twitter users internationally, with the majority being from Japan. Tweets included the link to the article and phrases such as “Link between parents’ harsh verbal discipline and adolescents’ conduct problems.” Kim et al. (2013) had the next highest Altmetric score and was cited in 35 Tweets and 20 news outlets, typically with the headline, “The last of the Tiger parents” alluding to the research findings showing that adolescents were more successful when they experienced supportive parenting styles. Research by Tang and Davis-Kean (2015) regarding punitive parenting techniques and student achievement was cited most notably on Twitter when a leading parenting expert in Australia embedded the article in his tweet reading: “Study of the Day: Kids not doing well at school? Punishing them for underperformance only makes things worse.” Research by Mackenzie et al. (2015) was mentioned in two news outlets that focused on the “vicious cycle” spanking sets in motion and both included direct quotes from the study author. Finally, research by Lansford et al. (2011) was linked in tweets predominantly by Japanese Twitter users.

Five studies with high Altmetric scores reviewed various topics including parental support, child temperament, racial differences in school punishment, sexual identity, and language. Lippold et al. (2017) had an Altmetric score of 370 and was mentioned in 43 news outlets who were primarily reposting an original National Public Radio article: “Moms Need Social Support, and Not Just in the Baby Years.” This article stressed the importance of social support for parents of adolescents. Weinstein et al. (2012) explored parental support and sexual identities. This research was cited most often in news outlets and tweets that reached international audiences in Chile, Brazil, Mexico, Saudi Arabia, Japan, India, the United Kingdom, and the United States. Schofield et al. (2017) was cited in nine news outlets, generally with the title, “Bad Behavior May Not be a Result of Bad Parenting but a Lack of Common Language.” The articles noted how the study showed that language barriers between parents and adolescents lead to difficult communication dynamics. Slagt et al. (2016) received the most attention on Twitter with users retweeting original tweets from two prominent Twitter users (each with over 30,000 followers) who both used their accounts to highlight parenting information. Hannon et al. (2013) was cited mostly on Twitter with the article link embedded in tweets explaining that girls with darker skin are three times more likely to get suspended than those with lighter skin.

The purpose of this article was to use parental discipline as an illustrative example to describe the methodology of a novel rapid review of influential literature. This review method could help character education and child-development researchers identify disconnects between study findings and how the media summarizes those findings for the lay public. Further, it could improve translation and dissemination of research to parents, educators, and other caregivers of adolescents. In general, the results from our review suggest that news sites and blogs occasionally report study findings accurately, but sometimes misinterpret and exaggerate these findings. These results are discussed below, in addition to the strengths and limitations of this review methodology.

The highly cited studies captured in this review indicate that effective parental discipline for adolescents involves consistent practices that are designed to allow them to push boundaries while ensuring their safety and well-being (Ge et al., 1996; Lempers et al., 1989). These older, highly cited studies have relatively low Altmetric scores, suggesting little penetration outside of the scientific literature. Alternatively, a study by Jensen and Whiteman (2014) which examined inconsistent discipline in the form of preferential treatment of different children received an Altmetric score of 105, yet has only been cited eight times, suggesting greater exposure to the lay public via blogs and news sites. This cross-sectional study showed that preferential treatment is associated with increased delinquent behaviors and substance use, but only in certain circumstances. For example, in families that scored low on measures of conflict and intimacy, adolescents who received less paternal intimacy were more likely to report delinquency, and those who perceived less favorable maternal treatment were more likely to report substance use. Notably, in “markedly warm” (i.e., low conflict/high intimacy) families, there were no significant associations between parents’ differential treatment and adolescents’ risk behaviors.

These nuances were sometimes, but not always, reported by news outlets. A 2017 Wall Street Journal article accurately summarized these details by stating,

Conversely, a 2017 Lifehacker article entitled, “Admit to Yourself That You Probably Have a Favourite Child” failed to report these details and simply states,

The absence of these important details may be caused by word limits placed on the author of the Lifehacker article. Nonetheless, by using the word “consistently” this article mischaracterizes the cross-sectional findings of the Jensen and Whitehead (2014) study, implying that the findings are longitudinal. The article title and the absence of the null finding about markedly warm families also may incorrectly lead parents to believe that preferential treatment of children is a feature of all families that they are powerless to change.

Harsh parental discipline is another topic that has consistent findings in the scientific literature and strong interest among the lay public, according to the studies captured in this review. It is well-established in the literature that harsh disciplinary practices are associated with numerous poor outcomes for adolescents, such as decreased academic achievement and behavioral problems (Eckenrode et al., 1993; Tang & Davis-Kean, 2015). Provocative findings such as these are typically reported by the news media but are prone to exaggeration and misinterpretation. For example, a study by Wang and Kenny (2014), which reported on the negative longitudinal effects of harsh verbal discipline on adolescent’s behavior and mental health, achieved the highest Altmetric score in our review. This study was reported in the Wall Street Journal with an article title reading, “Study Says Yelling is as Hurtful as Hitting” (Petersen, 2013). However, the study does not report that there is an equal effect between yelling at children and hitting them. In fact, physical punishment was controlled for when the study authors examined the effect of harsh verbal discipline.

This discrepancy was repeated on social media as well, with several Twitter users repeating this false equivalency and providing a hyperlink to the study. It is possible that these Twitter users never read the article, as research demonstrates that 59% of users share hyperlinks without clicking on and reading the content they are sharing (Gabielkov et al., 2016). This speaks to how inaccuracies about scientific findings can quickly spread on social media and to the importance of understanding popular narratives about research studies.

Other news outlets reported the results of this study more accurately by avoiding hyperbole. For example, the New York Daily News published an article with the headline, “Yelling at Your Teen Can Backfire, Impact Mental Health: Study” (2013). The study’s methods and findings are reported more accurately and the study author is paraphrased offering some evidence-informed parenting advice, “Take a breather if you feel a surge of anger or worry, and Wang suggests talking to your teens about the consequences of misbehaving, rather than yelling.” It also offers some important context from previous research—90% of parents report yelling at their children (Straus & Field, 2003). To equate physical and verbal discipline, as the Wall Street Journal did, is to suggest that these parents are putting their children at risk for the same outcomes as parents who physically discipline their children. This could be discouraging to this large percentage of parents, especially because research captured in this review also suggests that parents understand that corporal punishment is not an effective disciplinary practice (Chen et al., 2016). The outcomes associated with both of these harsh parenting practices are worthy of accurate reporting in the media in a way that helps parents feel empowered to effectively discipline their adolescents. The review methodology proposed here allows for comparisons of research and popular narratives in a given area.

In sum, the studies in this review suggest that fair discipline, enforced consistently, is associated with the best outcomes among adolescents. These findings compare favorably with traditional systematic reviews. Both our rapid review and other systematic reviews came to similar conclusions about the effects of parental discipline. For example, a systematic review conducted by Ryan et al. (2010) found that parental discipline was associated with reduced alcohol use, a finding that is supported by our rapid review. Moreover, this rapid review supports findings from systematic reviews demonstrating that harsh or physical parental discipline is associated with callous/ unemotional traits or violent behaviors among teens (Hawes et al., 2014; Waller et al., 2013). This suggests that a rapid review of influential literature, like the one described above, is unlikely to come to contradictory conclusions simply because the review excludes studies that do not meet predetermined thresholds of influence.

Our rapid review methodology has several strengths. Our review requires less time and staff to complete than the average systematic review (Borah et al., 2017), enabling faster turnaround time and inclusion of more up-to-date findings for translation and dissemination. Moreover, our process is iterative, meaning that our reviews can be revised quickly as studies become more influential. This overcomes an important limitation of systematic reviews, which can be outdated by the time they are published (Bastian et al., 2010; Roberts et al., 2015). This rapid review methodology also helps to understand the alignment between popular and academic narratives. It allows us to determine if scientific studies are described in the media in ways that maintain the integrity and accuracy of the research.

This methodology also has some weaknesses. Articles that receive a large number of citations are not always high quality. In fact, there are numerous examples of questionably designed or retracted studies that are highly cited, such as the Stanford prison experiment (Le Texier, 2019; Zimbardo et al., 1971) and studies linking vaccines to cases of autism (Flaherty, 2011; Wakefield et al., 1998). It is important to rely on more than the number of citations when deciding which studies should be included in the review. This is why elements of traditional systematic reviews, such as dual independent review and group discussion and consensus, were integrated into this methodology. This ensures studies are in fact relevant to the review and high quality. A subsequent check on the studies included in this review determined that none were retracted since this review was conducted.

Altmetrics also have some limitations. They are a relatively new tool for measuring a study’s impact and engagement among the general public. As such, Altmetrics creators can only offer a suggested score of 20 for an influential study, pending further research into measures of online impact and engagement. (Liu & Adie, 2013). Furthermore, high scores can be inflated by wide dissemination of an article across affiliated news sites (e.g., National Public Radio sites) or via wire services. Researchers can also influence the Altmetric score by repeatedly promoting their work on social media. It is because of these limitations that we complemented the use of Altmetrics with other metrics, such as number of citations, and accepted systematic review techniques. In addition, it is difficult to know how the lay literature impacts parents’ attitudes, beliefs, and intentions about discipline. Many parents use social media to find information about parenting (Duggan et al., 2015). Similarly, discussions about character education increasingly occur on social media (Hersh, 2015). Future research should examine how translations of scientific findings pertaining to both parenting and character education disseminated via social media alter parents’ and educators’ beliefs and behaviors in raising and teaching children.

Traditional methods of reviewing scientific literature must adapt to the ever-changing online landscape including social media sharing and consumer demand for timely news. The novel approach to rapid literature review described in this article was designed to address these realities and identify disconnects between scientific findings and the popular press. By uniting traditional citation metrics with the use of Altmetrics that measure online and social media attention, this rapid review methodology was able to effectively examine influential literature from both the scientific community and the lay literature. In practice, this could allow researchers to address gaps or misunderstandings in the public’s knowledge about parenting. For example, researchers could use this methodology to quickly assess what the scientific literature demonstrates and how the public is discussing study findings on social media platforms, such as Twitter. From their own social media accounts, researchers could then accurately translate and disseminate findings or intervene to correct inaccuracies or misinterpretations in widely shared news stores. This level of public engagement by researchers could have profound implications for educators, policy makers, and parents and improve their understanding of scientific studies.

Andrew C. Pool, Danielle Doolittle, Nora Laberee, Shannon Traurig, and Kenneth R. Ginsburg, Center for Parent and Teen Communication, Craig-Dalsimer Division of Adolescent Medicine, Children’s Hospital of Philadelphia; Melinda C. Bier and Christopher D. Funk, Center for Character and Citizenship, College of Education, University of Missouri-St. Louis. Kenneth R. Ginsburg is also affiliated with the Department of Pediatrics, Perelman School of Medicine, University of Pennsylvania. We would like to acknowledge Elyse Salek, Eden Pontz, and Jacques Louis for providing insightful commentary on the initial drafts of the manuscript. Funding was provided by grant #60721 from the John Templeton Foundation.