Development of clinical prediction rule for diagnosis of autistic spectrum disorder in children Open Access

Autistic spectrum disorder (ASD) is a neurodevelopmental disorder with a prevalence of 1:68 in children (Christensen, 2016; American Psychiatric Association, 2013). Early diagnosis with early intervention yield ameliorated long-term outcomes (Filipek et al., 2000; Granpeesheh et al., 2009; Landa, 2008). Because ASD is a disorder with a multitude of signs and symptoms, the diagnosis process requires massive history taking from the caregiver together with time-consuming clinical observation by experienced clinicians (Wing, 1988; Falkmer et al., 2013). Doctors working in countries with inadequate specialists and resources are challenged by the ASD diagnosis. These general practitioners, limited by time and experience, may underdiagnose ASD, resulting in delayed treatment. Having clear and concise predictors to facilitate the initial diagnosis of ASD in busy clinical practice would benefit both doctors and patients (Zwaigenbaum et al., 2015). This study aimed to develop a prediction rule for the diagnosis of ASD in children from baseline characteristic profiles, risk factors, history and clinical observation.

We conducted a population-based study from January to December 2018 in consecutive children aged 2-5 years suspected of ASD who visited Thammasat University Hospital. Eligibility was based upon the patients having any one of the following chief complaints: delayed speech (no discrete words by 18 months or no phrases by 24 months or no complete sentences speech by 36 months); social or play problems, e.g. preferred to be left alone; repetitive behaviors or restricted interests; behavioral or emotional regulation problems; or doctors/parents concerned that the child may have had ASD. Patients were excluded if they had any of the followings: severe chronic medical illness or physical disability, congenital anomalies/syndromes or hearing problems, had already been diagnosed with ASD, the main caregiver did not attend with the child and the caregiver was not able to communicate in Thai.

The potential predictor variables included demographic data and risk factors, i.e. gender, age, chief complaint, level of communication, birthweight, maternal and paternal age, family history of autism or developmental delay, caregiver level of education, history of child’s ASD symptoms and symptoms from clinical observation ( Appendix). All variables were selected based upon a review of the existing literature (Devlin and Scherer, 2012; Ozonoff et al., 2011; Gardener et al., 2009; Hultman et al., 2011; Ozonoff et al., 2009; McCoy et al., 2009; Clifford et al., 2013; Allison et al., 2012; Srisinghasongkram et al., 2016; Pornnoppadol et al., 2002; Panyayong, 2011; Krivichian, 2014; Maenner et al., 2013; Tsheringla et al., 2014; Dow et al., 2017; Ozonoff et al., 2008; Watt et al., 2008). While the caregiver filled out form on the demographic specifics, risk factors and history of child’s ASD symptoms, a general practitioner observed patients’ symptoms according to a prepared checklist. Both steps took less than 20 minutes per patient. All patients, then, were independently assessed by trained research assistants using ASD standardized diagnostic tools (Huerta and Lord, 2012). The Developmental, Dimensional and Diagnostic Interview short form and Autism Diagnostic Observation Schedule (Santosh et al., 2009; Chuthapisith et al., 2012; Lord et al., 2000). ASD diagnosis was made, in accordance with The Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5), by a child psychiatrist or developmental and behavioral pediatrician using clinical assessment and information from both tools. Other diagnosis and comorbidities were given following the DSM-5 criteria.

ASD and non-ASD groups were compared for evidence of differences (p-value) in clinical characteristics with t-test or exact probability test as appropriate. Prediction by each characteristic was calculated using univariable logistic regression and presented as an area under the receiver operating characteristic (AuROC) curve and its 95 per cent confidence interval (95% CI). Clinical predictors with a high AuROC curve and p value <0.01 were selected and processed with multivariable logistic regression with backward stepwise selection (p < 0.1) to aid the selection of the best variables. The discriminative performance of the model was calculated by an AuROC curve. The regression coefficient of each clinical predictor was divided by the smallest coefficient of the model and transform into an item risk score. Scores for each clinical predictor were added up to obtain a total risk score. Score prediction of ASD diagnosis was done by using a total score as the only summary predictor in the logistic model. Discrimination of the score was presented with an AuROC curve. Calibration of the prediction was analyzed with Hosmer–Lemeshow statistics. Scores predicting risk and observed risk were compared and presented in a graph. Internal validation of the score was done by logistic regression with the bootstrap method. Risk scores were categorized into risk levels. The predictive ability of each risk score level was calculated and presented as a likelihood ratio of positive, 95% CI and its significance level. This research was approved by the research ethics committees of the Faculty of Medicine, Thammasat University.

One hundred and thirty-nine patients were enrolled (Table I). All patients had a complete assessment of ASD, and 104 (74.8 per cent) were diagnosed with ASD. In non-ASD group, diagnoses were language disorder (7.9 per cent), attention-deficit hyperactivity disorder (7.9 per cent), typical development (5.0 per cent), global developmental delay (2.9 per cent) and childhood-onset fluency disorder (stuttering) (1.5 per cent).

Eighty-five predictors from the patient profile, history taking and clinical observation were assessed. The association between all predictor variables and diagnosis of ASD determined using univariate analyses and the prediction ability measured by using AuROC were shown in the  Appendix. Predictors that had p <* 0.01 from univariate analyses were the level of communication, 11 symptoms from history taking and 16 symptoms observed by the clinician (Table II). These 28 variables were processed with multivariable logistic regression with backward stepwise selection (p < 0.1).

The best multivariable clinical predictors for the diagnosis of ASD from the multiple logistic regression were level of communication, history of rarely making eye contact or looking at faces, history of not showing off toys or favorite things, did not follow the clinician’s eye direction when called and signaled with eyes to look at things far away and had low frequency of social interaction with the clinician or the caregiver in the room. These five clinical predictors were each categorized into two levels. An item score of 1 was assigned to each predictor (Table III).

A summary risk score was obtained by adding up the item scores. The discriminative ability of the derived risk score, which ranged from 0 to 5, could directly be observed by the different percentage distribution between ASD and non-ASD groups (Figure 1).

The risk score predicted a diagnosis of ASD with an AuROC curve of 91.0 per cent (95% CI, 85.8-96.1) (Figure 2) and with the p-value for the Hosmer–Lemeshow goodness-of-fit test of 0.67. Internal validation by the bootstrapping method (1,000 replications) reduced the AuROC curve to 83.26 per cent (95% CI, 76.0-90.5).

When translating into absolute risks, the score predicted the risk of diagnosis of ASD increased when the risk score moved upward, with close calibration to the actual or observed risks (Figure 3).

The risk scores were categorized into three risk groups, low (0) when the slope of the risk curve was lowest, moderate (1-3), and high (4-5) to facilitate clinical interpretation. The positive likelihood ratio for the diagnosis of ASD was 0.04 in the low risk, 0.45 (95% CI, 0.34-0.59) in the moderate and 16.62 (95% CI, 2.38, 116.05) in the high categories (Table IV).

This clinical decision rule has been developed to help general practitioners for predicting the diagnosis of ASD in children aged 2-5 years old.

Research in the past from the UK found that parents of children with ASD brought them to hospital from age 2 ± 1.92 years, but the average age of diagnosis was 5.7 years. In the first visit, usually with a general practitioner, less than 10 per cent of patients received diagnosis, and 26-30 per cent were told “no problem/no worry”. The other 50 per cent were referred to specialists (Howlin and Asgharian, 1999). This older study may convey the situation in Thailand and other developing countries today. Furthermore, in these countries where specialists are less than adequate, the referring process may take years. Caregivers who are not confident in the diagnosis may be lost to follow-ups, and the early intervention will be delayed. This clinical decision rule would allow the general practitioners to make the initial diagnosis of ASD based upon the clear and evidenced rule. Having more confidence regarding the initial diagnosis, they are able to provide disease-specific initial recommendations and management for caregivers and families.

Recently, several screening questionnaires for autism have been developed (Allison et al., 2012; Srisinghasongkram et al., 2016; Pornnoppadol et al., 2002; Panyayong, 2011; Krivichian, 2014). This decision rule would facilitate the initial diagnosis in patients with positive result from the screening process. This risk score is highly accurate in the predicted diagnosis of ASD (the AuROC being 91.0 per cent). We chose the cutoff score of 4 to classify patients into a very high-risk group. We chose a high cutoff score because we want this decision rule to be highly specific so the doctors would be confident in the initial diagnosis.

To apply this rule in practice, patients with four or more of these predictors, namely:

• delayed speech for their age;

• history of avoiding eye contact/meeting others’ gaze;

• history of a pattern of not showing objects to others;

• poor response when the clinician attempts to draw attention to something in a distance; and

• low frequency in reciprocal social interaction with the clinician or the caregiver in the room are at substantial risk of having ASD (positive likelihood ratio = 16.62).

A doctor can discuss the ASD diagnosis and give psychoeducation to the family. Also, initial management can be done promptly, i.e. referral to a speech therapist, occupational therapist or developmental stimulation program. Patients with one to three predictors may or may not have ASD and should be referred to specialists. Patients with no predictors are at low risk of having ASD. They can be managed as per other diagnoses or observed.

The strength of this study is that it was a population-based study conducted in routine clinical practice with limited observation time. Patients and doctors would represent target groups that results were intended to be used. The results also showed which ASD symptoms can be observed in the time-limited outpatient situation. The diagnosis process was based on the reference standard for the ASD diagnosis. As all variables were collected before the specialist assessed the patients, the bias of information would be reduced. Furthermore, as the format of the rule includes a simple list of history taking and clinical observations, it would make this rule clinically sensible for the busy general practitioner to apply it in routine practice.

However, the number of patients in this study was small, and the derived score is likely to be space domain specific. Also, as all data were collected in Thai, cultural and language effects should be considered. Clinical predictors in our setting may not be directly applicable to other settings. Model adjustment, either selection of different clinical predictors and/or different scoring weights, should always be considered for application to a new setting. Also, it is necessary for the model to have an external validation to provide sufficient evidence about its performance.

This simple and practical clinical decision rule may help non-specialists to make the initial diagnosis of ASD in children. Caregivers of the very high-risk patients may be informed about the disease and its caring process that will improve the quality of care.

Table AI 

Table AII 

Table AIII