The Parenting to Reduce Adolescent Depression and Anxiety Scale: Assessing parental concordance with parenting guidelines for the prevention of adolescent depression and anxiety disorders

Aims and hypotheses

In this paper, we used a sample of Australian parents of adolescents aged 12–15 to: (1) validate the newly developed PRADAS as a criterion-referenced measure of parental concordance with the Guidelines; (2) examine levels of parental concordance with the Guidelines; and (3) examine correlates of parental concordance with the Guidelines. We examined the reliability and validity of the scale to fulfil the first aim. Reliability was assessed via an estimate of the agreement coefficient, as well as 1-month test-retest reliability. Validity of the scale was determined based on its face validity and convergent validity indicators. Convergent validity was assessed via correlations between the scale and two previously validated measures of parenting: the Inventory of Parent and Peer Attachment (IPPA) and the Emotions as a Child scale (EAC). As these measures each have some overlapping content with our scale, we predicted at least moderate correlations. We predicted positive correlations with the subscales assessing positive parenting practices, and negative correlations with the subscales assessing undesirable parenting practices. We also expected small to moderate correlations between the nine PRADAS subscales and moderate to high correlations between the subscale scores and the total score. Finally, we explored the correlates of parental Guidelines concordance with participant characteristics (e.g., demographics, mental health history, adolescent depression and anxiety symptoms). We predicted small correlations between parental Guidelines concordance and symptoms of depression and anxiety in adolescent participants. Examination of other correlates were exploratory in nature, hence no hypotheses were specified.

Participants and procedures

Participants were parent-adolescent dyads participating in one of two randomised controlled trials (RCTs) of online parenting interventions aimed at preventing adolescent depression and anxiety disorders. Eligible participants were parents or primary caregivers of an adolescent aged 12 to 15 years, who resided in Australia and had regular internet access. No exclusion criteria were specified. Parent participation was not dependent on adolescent participation; as such, parents could participate without their child. Only one parent-adolescent dyad per family was eligible to participate. Recruitment was via online parenting networks, social media, university networks, mental health organisations, and schools across Australia. Ethics approval was obtained from the Monash University Human Research Ethics Committee (approval numbers CF14/3886–2014002023 and CF14/3887–2014002024).

Parent participants self-selected by responding to advertisements and registering and providing consent for themselves and their adolescent via the trial websites. Parents and adolescents were provided with participant explanatory statements, detailing participation requirements, privacy, and standard limits to confidentiality (e.g., disclosure of risk of harm to self or others, child abuse). A member of the research team called adolescent participants to explain the study and obtain their verbal assent. Adolescents who agreed to participate were guided through their online baseline assessment over the phone, which included the child-report version of the Spence Children’s Anxiety Scale (SCAS) and Short Mood and Feelings Questionnaire (SMFQ). Assistance to complete the measures was provided by the researcher where necessary (e.g., in the case of learning disabilities). Adolescents were reimbursed for their participation with an e-voucher. Following submission of the adolescent baseline assessment, an automated email was sent to the parent participant containing the link to their baseline assessment. Parent participants completed the PRADAS, and the parent-report versions of the IPPA, EAC, SCAS, and SMFQ.

The two RCTs from which participants were drawn assessed similar interventions (i.e., a brief and intensive version of the same online parenting program) and targeted similar populations. Further, data used in the current paper was collected prior to any intervention being received. It was therefore deemed acceptable to combine baseline data for the purposes of this paper. Demographic characteristics of participants from the two samples were compared to assess for significant differences on key variables. There were no significant differences between samples for the following variables: parent or child age; parent or child gender; parent employment status; parent marital status; parent–child relationship (i.e., mother/father/other); parent- or child- report scores on the SCAS or SMFQ; or total score on the PRADAS (all ps > .05). Significant differences were found for three demographic characteristics: (1) percentage of parents speaking a language other than English (8.6% in sample 1 vs. 15.8% in sample 2, χ²[1, N = 711] = 8.62, p = .003); (2) highest parent qualification, with a greater number of participants in sample 1 having post-graduate qualifications (32.9% vs. 24.9%) and more parents in sample 2 having secondary school (year 7 to 12) qualifications (13.9% vs. 7.7%), χ²[5, N = 711] = 15.75, p = .008; and (3) state of residence, with more participants in sample 2 residing in Victoria (86.1% vs. 21.4%, χ²[7, N = 711] = 316.63, p < .001). As these variables were not outcomes, nor expected to significantly affect results, we opted to combine the samples in order to attain a larger and more diverse sample of parents in Australia.

A total of 350 and 361 parents registered and completed baseline assessments for the two trials respectively. The final sample therefore comprised 711 parents. Most parents were female (89.5%), with a mean age of 45.15 years (SD = 5.66). Six hundred sixty adolescents also consented to participate and completed baseline assessments. Adolescents had a mean age of 13.66 years (SD = 1.05) and 47.3% were female. Table 1 presents additional participant demographic characteristics.

Measures

Parent and child mental health characteristics

Table 3 presents frequencies for parent and child history of mental health problems and parental concern about their adolescent’s risk of developing depression or anxiety. Most parents expressed at least “a little” concern about their adolescent’s risk of developing depression (79.9%) or anxiety (79.0%). Over half (58.5%) of parent participants reported experiencing either past or current mental health problems themselves. Regarding adolescent mental health history, parents reported that 15.1% of adolescents had been diagnosed with a mental health problem in the past, and 22.1% believed that their child had experienced a mental health or behavioural problem that had not been formally diagnosed. Similarly, 17.2% of adolescents were reported to be experiencing current mental health issues, with 24.1% of parents concerned that their child may be currently experiencing an undiagnosed mental health or behavioural problem.

Concordance with the Guidelines

Table 4 presents descriptive concordance statistics for the nine subscales and the total score. Concordance rates for most subscales were low, with less than 20% of parents scoring within the concordant range for seven of the nine subscales. This may in part be due to the fact that cut-off scores for concordance were intentionally set to be high. Two subscales were notable exceptions to this pattern: (1) Relationships with others had a much higher concordance rate of 77.5%. The mean score of this subscale was close to the highest possible score (5.11 out of 6.00) and there was less variance in responses, representing a ceiling effect; (2) Professional help-seeking, which also had a higher concordance rate than other subscales, at 49.37%.

Further exploration of items within the relationships with others subscale revealed ceiling effects for four of the six items (94% to 99% of participants concordant), explaining the overall ceiling effect. These items assessed parental encouragement of socialisation, being kind to others, and talking to the adolescent about social problems (see File S1 for scale items). The remaining two items in this subscale, which were both negatively worded, had concordance rates of 70.2% and 53.2%.

Examination of the professional help-seeking subscale revealed ceiling effects for three of the nine items. Two of these items assessed parental response to the question “If you noticed a persistent change in your teenager’s mood or behaviour, how likely would you be to: (insert question)”. Ninety-nine percent of parents responded that they would encourage their teenager to speak to them about what was going on, and 98.7% endorsed that they would try to determine whether the change was caused by a temporary situation or more ongoing problem. Further, 93% of parents stated that they would know where to seek appropriate professional help for their teenager if needed. Other items within this subscale had concordance rates between 45.9% and 83.1%.

Reliability of the scale

Agreement coefficients for the subscales and the total score are presented in Table 4. Agreement coefficients were above the recommended .75 level for seven subscales, and was very high (.97) for the total score. Two subscales had lower reliability: relationships with others (.68); and professional help-seeking (.61). Reliability was therefore deemed to be high for the total score, acceptable to high for seven of the subscales, and questionable for two subscales.

Test-retest reliability was calculated for a smaller subsample of 175 participants in the waitlist control group of one of the RCTs, based on data from baseline to 1-month follow-up assessments. Test-retest reliability was deemed to be acceptable for the total score, r = .76, p < .001. Subscale test-retest reliability ranged from .29 to .72 (p < .001), with five of the nine subscales having test-retest reliability between .59 and .72 (see Table S1 for individual subscale test-retest reliability).

Correlations between subscale scores and total score

We examined the correlation matrix of subscale scores and the total score (see Table 5). With the exception of the relationships with others subscale, all subscales had significant moderate to high correlations with the total score, as predicted. Also as hypothesised, most subscales had small to moderate correlations with the other subscales, reflecting the unique skills assessed within each domain. The relationships with others subscale had the lowest correlations with other subscales.

Refinement of the scale based on analyses

Based on the above analyses, we chose to remove the relationships with others subscale from the final version of the scale for the following reasons: (1) there was a ceiling effect for the subscale score, attributable to ceiling effects for four of the six items; (2) questionable reliability (agreement coefficient = .68); and (3) this section had a low correlation with the total score (r = .23; in contrast to all other subscales which had correlations ≥.51) as well as low correlations with the other subscales (rs .00 to .21). After removing the 6-item relationships with others subscale, the agreement coefficient for the total score was unchanged (.97) and the test-retest reliability of the total score increased slightly (from .76 to .78).

Although the professional help-seeking subscale also had ceiling effects for three out of nine items, this may be at least partially attributable to the demographic characteristics of our sample, rather than the items. Most parents were highly educated, and had already taken a step towards seeking professional help by enrolling in an RCT aimed at reducing risk of depression and anxiety in their adolescent. Moreover, given that many parents and adolescents in our sample had a history of mental health issues, it is likely that these parents had greater knowledge of mental health support services than parents who have never had reason to access these. Further, the subscale total score did not have a ceiling effect, and the subscale had a moderate correlation with the total score. We therefore chose to retain this subscale, despite its lower reliability.

All subsequent analyses are based on the revised total score calculated without the items comprising the relationships with others subscale.

Convergent validity

To examine convergent validity, correlations between the total and subscale scores of the PRADAS and the IPPA and EAC were examined (see Table 6). We predicted positive correlations with all IPPA scores, as the items from the alienation subscale were already reverse scored. Four of the five EAC subscales (punish, neglect, override, magnify) assess undesirable parenting practices and were not reverse scored, hence we predicted our measure to correlate negatively with these subscales and positively with the reward subscale. In support of the convergent validity of the PRADAS, there was a moderate to strong positive correlation between the PRADAS total score and the IPPA total score (r = .61, p < .001). The total score also had moderate to strong correlations in the predicted directions with the trust and communication subscales of the IPPA (rs = .56 and .61, ps < .001), as well as three of the five EAC global subscales (reward, r = .59; punish, r =  − .49; neglect r =  − .56; all ps < .001). These subscales each have some overlapping content with our scale, hence the strength of the correlations are appropriate to the degree of overlap of constructs assessed. There were also small to moderate correlations in the predicted direction with the IPPA alienation subscale (r = .43, p < .001), as well as the EAC override and magnify subscales (rs =  − .32 and −.36, ps < .001).

Examination of the correlations between the PRADAS subscale scores and subscales of the IPPA and EAC also revealed small to moderate correlations in the predicted directions. For example, the parent–child relationship subscale had a correlation of .51 (p < .001) with the IPPA total score and trust subscale, and lower correlations with the communication and alienation subscales (rs = .45 and .35, ps < .001), as well as the EAC reward, punish, and neglect subscales (rs =  − .42 to .46, all ps < .001). Likewise, the involvement, dealing with problems, and coping with anxiety subscales all had small to moderate correlations in the predicted directions with various subscales of the IPPA and EAC (see Table S2 for subscale correlations). Two subscales of our measure assessed content not measured by the IPPA or the EAC: health habits and professional help-seeking. As expected, these subscales had the lowest correlations with all subscales of the IPPA and EAC (all —r— < .32; see Table S2). Overall, we interpreted these results as support for the convergent validity of the PRADAS.

Correlates of guidelines concordance

Parental concordance with the Guidelines

We examined levels of parental concordance with established Guidelines in an Australian community sample of parents of adolescents aged 12 to 15 years. Overall, parental concordance with the Guidelines was relatively low, with less than 20% of parents deemed concordant in six of the eight domains. This is unsurprising given that we intentionally set high cut-off scores for concordance. In contrast, the professional help-seeking domain had the highest concordance rate, at 49%. As discussed, this may be a result of our sample characteristics. Alternatively, parents may have scored highly in this domain due to the hypothetical nature of the questions. The items assess how likely a parent would be to respond in certain ways if they noticed a persistent change in their adolescent’s mood or behaviour, as well as how likely they would be to seek professional help if needed. It is possible that parents may overestimate their likelihood to take particular actions when posed in hypothetical scenarios. As posited by theories such as the Theory of Planned Behaviour (Ajzen, 1991), and confirmed in numerous empirical studies (e.g., Armitage & Conner, 2001; McEachan et al., 2011; Sheeran, 2002), intentions do not always translate into action.

Concordance was lowest (4% of parents) for the family rules subscale, which assesses the establishment and consistency of rules and consequences. Closer examination of item responses revealed that less than half (47%) of parents endorsed having set specific, defined rules for their teenager’s behaviour in most areas. Similarly, only 27% of parents answered that they had set specific, defined consequences for breaking the rules, and 16% said that their teenager had been involved in developing most of the rules. Promisingly, most parents (86%) stated that they adapt the family rules to their teenager’s maturity and responsibility, and 80% said that they speak with their teenager about why their behaviour is not acceptable when enforcing a consequence. These findings are consistent with findings from a similar measure designed to assess parental concordance with guidelines for the prevention of adolescent alcohol misuse (Yap, Jorm & Lubman, 2015). This measure also assessed consistency of family rules, and found the family rules subscale to have the lowest concordance rate (7.2%). Together, these findings highlight an important target for future preventive programs, particularly given that parental disciplinary practices are associated with risk for both internalising and externalising disorders (Berg-Nielsen, Vikan & Dahl, 2002; Gryczkowski, Jordan & Mercer, 2010; Yap et al., 2014a).

The remaining six domains had concordance rates between approximately 10% and 20% of parents. Although this small percentage of parents scored above the cut-off score for concordance with the entire domain, it is promising to note that mean subscale scores were close to or above the mid-range of possible scores. In other words, most parents answered within the concordant range for at least half of the individual items comprising each subscale, although they did not reach the threshold set for mastery of the skill area. This was also the case for the total score, which had a mean of 47 out of a possible 73. This means that while very few parents were considered concordant according to the cut-off scores set for “mastery” of each domain, on average parents were adhering to over half of the recommendations in the Guidelines. It is therefore plausible that many parents, particularly those who scored close to the cut-off, may move into the concordant range with intervention, where parents are supported to make changes in these parenting skill areas.

It could be argued that the designation of cut-off scores for concordance was somewhat arbitrary, and that the cut-off scores could be set lower given the low concordance rates obtained in our sample. As discussed, we chose high cut-off scores that indicate close-to-absolute-concordance with the Guidelines’ recommendations. This was based on the expertise of the developing authors, the strong evidence base for the parenting factors assessed, and the purpose for which the scale was initially developed. The optimal cut-off scores for a criterion-referenced test such as this will depend on individual applications. For example, high cut-off scores may be appropriate when the goal is to evaluate intervention effects or to identify areas for improvement, whereas lower cut-off scores may be appropriate if attempting to determine a “bare minimum” level of competency. In the case of the PRADAS, we aimed to develop a measure that could be used to identify domains of parenting that could be improved by parents so they become more concordant with the Guidelines. From a public health perspective, prevention approaches that attempt to shift all individuals along the continuum of risk are likely to have greater effects at a population level than approaches only targeting those at highest risk (in this case, parents with the lowest PRADAS scores; e.g., see seminal work by Rose, 1993). To explore this, we examined scatterplots of PRADAS scores and scores on the symptoms measures with locally weighted smoothing (LOESS) lines fitted. These curves indicated that, even at the higher end of PRADAS scores, adolescent symptoms were lower, supporting the value of encouraging all parents (even those with moderate and high scores) to increase their concordance with the Guidelines. Finally, we conducted additional exploratory analyses with lower cut-off scores (one score less than the original cut-offs for all subscales). With these lower cut-off scores, reliability of the eight subscales and the total score reduced. Together, these findings support the use of high cut-off scores for the current purpose of the PRADAS. It should also be noted that the cut-off scores for concordance are one of two methods of interpreting PRADAS scores. The continuous scores are also calculated (i.e., total score out of 73, or subscale scores), and these can be interpreted alone or with reference to the subscale cut-off score. In many settings, the continuous scores are likely to be of most interest, as they allow interpretation of an individual’s score in comparison to the range of possible scores. This may be of particular value in clinical contexts, as well as to track change over time.

Correlates of parental concordance with the guidelines

We examined correlations between parental concordance with the Guidelines and parent- and child-report measures of adolescent depression and anxiety symptoms, as well as participant characteristics. We found small but statistically significant correlations between parental Guidelines concordance and adolescent symptoms. The correlations were slightly larger for parent-reported symptoms compared to child-reported symptoms, suggesting that parents who rate themselves as more concordant with the Guidelines also report their adolescents to have fewer symptoms of depression and anxiety. Additional exploratory logistic regression analyses revealed a small effect of the PRADAS in predicting current symptom elevation status (i.e., above or below clinical cut-off) for three of the four symptom measures (SCAS-P, SMFQ-P, SMFQ-C). To follow up on these findings, we also compared mean PRADAS scores for participants above and below clinical cut-offs. In line with the logistic regression results, PRADAS total scores were significantly lower for participants who scored above the clinical cut-off on the SCAS-P, SMFQ-P and SMFQ-C (see Table S4). Together, these findings suggest that there is a relationship between PRADAS scores and current clinically elevated symptom status, albeit a modest one. Similarly, there was a small, significant correlation between Guidelines concordance and parental concern about their child’s risk of developing both depression and anxiety. Parents with higher concordance were less concerned about their child’s mental health. The findings regarding adolescent symptoms should be interpreted with caution, due to the small effects obtained and design of the study. The cross-sectional analyses limits our ability to assess the theorised longer-term risk factors that the PRADAS is designed to assess. That is, the factors assessed by the scale are expected to increase or decrease risk of depression or anxiety problems developing throughout adolescence. More rigorous assessment of the relationship between PRADAS scores and adolescent symptoms requires prospectively collected data, which would more accurately assess the predictive validity of the scale. Additionally, the symptom measures could be considered “state” measures (i.e., assessing current/recent symptoms), whereas the PRADAS may be more likely to capture “trait” level parenting practices. A cross-sectional design may not adequately capture the relationship between these variables. Also, we intentionally recruited a community sample, yielding mean scores on the symptom measures that were below clinical cut-offs. It would be of interest to examine the associations between PRADAS scores and symptom levels in a clinical population. Finally, meta-analyses of the associations between parenting and child/adolescent symptoms have also reported small effect sizes (e.g., McLeod, Weisz & Wood, 2007; McLeod, Wood & Weisz, 2007; Yap et al., 2014a; Yap et al., 2014b). Given these points, it is unsurprising that the associations between PRADAS scores and symptom measures were small.

Regarding demographic characteristics, there were small significant correlations of parent female gender and higher levels of education with higher Guidelines concordance. This should be interpreted in light of the over-representation of female, highly-educated parents in our sample. This is in line with previous findings of higher rates of mental disorders among children of parents with lower education (e.g., Lawrence et al., 2015; Meltzer et al., 2003; Merikangas et al., 2010), as well as an association between parental education level and treatment outcomes in parenting intervention studies (e.g., Reyno & McGrath, 2006). Additionally, in a survey of Australian parents’ beliefs about the role of parenting in the prevention of depression in young people, parents with tertiary-level education were more likely to hold beliefs consistent with the evidence regarding parental over-control as a risk factor for depression (Yap & Jorm, 2012). The current findings contribute further evidence highlighting the need to consider socio-demographic factors, such as parental education level, when considering strategies to target in preventive parenting programs.

Limitations and future directions

The current findings should be interpreted in the context of study limitations. Our sample had an overrepresentation of highly-educated mothers from intact families, limiting the generalisability of findings to fathers, those with lower levels of education, and different family situations. Nonetheless, while our sample may not be representative of the general population, it is likely representative of parents who self-select to participate in online parenting programs. Our sample characteristics are similar to those often reported in online parenting intervention studies (e.g., Dittman et al., 2014; Enebrink et al., 2012; March, Spence & Donovan, 2009; Sanders, Baker & Turner, 2012). Importantly, given the correlation found between education and parental Guidelines concordance, concordance rates in our sample may be higher than would be found in less educated populations. It is an important, yet challenging, direction for future research to recruit more diverse samples of parents into research of this nature.

The current study is the first psychometric evaluation of the newly developed PRADAS. Use of the PRADAS in future research will allow for further refinement of the measure. This would be particularly valuable to improve the utility of the subscale scores, so as to provide eight short scales that can be used with confidence to assess the various domains of parenting covered in the Guidelines. The current form of the scale is quite long, with 73 items, and mean completion time just under 18 min (M = 17.76, SD = 12.93). It would therefore be valuable to develop a short-form of the scale, which could have greater utility in clinical settings. Additionally, development of an adolescent-report version of the PRADAS would allow cross-validation between parent- and adolescent-report, which is not captured in the current version of the scale.

The associations between PRADAS scores and adolescent symptoms were small. The cross-sectional nature of the study limits our ability to assess the prospective predictive validity of the scale. We are also unable to ascertain concurrent validity based on diagnostic status, which cannot be determined from symptom measures such as those used in the used in the current study. However, it should be noted that the PRADAS is designed to assess parenting factors which evidence and expert consensus suggest would influence risk of adolescent depression and anxiety; the measure in itself does not purport to predict future risk. Results of the PRADAS should therefore not be interpreted as indicative of risk for the adolescent.

To our knowledge, the PRADAS is the first assessment measure to comprehensively assess the range of parenting risk and protective factors shown to influence the development of depression and anxiety disorders in adolescents. This is a valuable addition to the growing body of research investigating prevention of depression and anxiety in young people. It has potential for use in both research and clinical settings. The scale may serve as an assessment measure to tailor and evaluate preventive interventions targeting parenting factors, such as in the RCTs mentioned above. For investigators interested in specific domains of parenting, the option of eight subscales assessing different risk and protective factors may be of particular interest. Clinically, the PRADAS could be used to identify individual parents’ strengths and areas for improvement, which may be valuable to clinicians working with parents. Future studies could examine parental Guidelines concordance in clinical versus community samples, including parents of adolescents receiving treatment for depression or anxiety. It would be of interest to ascertain whether the same parenting factors are relevant to treatment or relapse prevention outcomes in a clinical population. If the PRADAS was found to contribute valuable information in this population, it could be used to tailor interventions for parents of adolescents experiencing depression or anxiety disorders.