Surrogate model-assisted machine learning optimization of aerospace coating deposition processes for enhanced performance and uniformity Open Access

The aerospace industry demands exceptional coating quality with strong requirements for thickness uniformity, microstructural consistency and adhesion strength to ensure component reliability under extreme operating conditions and the chemical vapor deposition (CVD) process offers a viable route to achieve these stringent performance criteria. CVD is a pivotal technique for the fabrication of high-quality thin films and coatings, plays a crucial role in various industries, including semiconductors, optoelectronics, photovoltaics and protective surfaces (Boscher et al., 2021; Du et al., 2022; Katsui and Goto, 2020). Its ability to precisely control material composition, crystalline structure and layer thickness has made it indispensable in both industrial and research settings (Bhatia et al., 2015; Takahashi et al., 2009). However, achieving optimal film characteristics, particularly the deposition rate and uniformity, necessitates stringent control over complex, interdependent process parameters such as temperature, pressure, gas flow velocity and precursor composition (Arias Louie J et al., 1997; Hu et al., 2015; Kim and Lee, 2018). Traditionally, understanding and optimizing CVD processes has relied heavily on experimental trials and computational fluid dynamics (CFD) simulations. Although CFD offers valuable insights into the transport phenomena and reaction kinetics within a reactor, it is often computationally intensive and time-consuming, especially when exploring large design spaces with multiple parameters (Kleijn, 2002; Kota and Langrish, 2007; Zhou et al., 2023). This computational burden limits its effectiveness in real-time control and rapid process optimization scenarios. Consequently, there has been a growing interest in integrating machine learning (ML) approaches to build surrogate models that approximate CFD predictions at a fraction of the computational cost (Bounds et al., 2024; Páez, 2024; Roznowicz et al., 2024; Schneider et al., 2022).

ML, particularly supervised regression algorithms, offers a data-driven pathway for capturing the nonlinear relationships between input parameters and performance metrics in CVD systems (Botchkarev, 2019; Plevris et al., 2022). Previous research has demonstrated the efficacy of ML models such as support vector regression, random forest and artificial neural networks (ANNs) in predicting deposition characteristics from a given set of input parameters (Artrith and Urban, 2016; Bilgiç and Gök, 2024; Hia et al., 2023). However, these models often exhibit limitations in scalability, interpretability or generalization depending on the complexity of the process and the data set used. In our previous work, a comparative analysis of various ML models, including ANN and Xtreme Gradient Boosting (XGBoost) was conducted to model the CVD process using the susceptor temperature and gas inlet velocity as input features. The target outputs were the deposition rate and film uniformity, both of which are critical indicators of the product quality in CVD applications. The study concluded that XGBoost consistently outperformed other models in terms of prediction accuracy, exhibiting the lowest mean square error and highest R² score when compared with the ground-truth data obtained from CFD simulations.

Building on these promising results, the present work proposes an advanced machine learning framework that leverages the predictive capabilities of the XGBoost model to optimize the CVD process. Rather than merely predicting outcomes based on predefined inputs, this study aims to invert the problem by identifying the optimal combination of susceptor temperature and gas inlet velocity that would yield the maximum deposition rate while ensuring uniformity within the desired thresholds.

To achieve this, the proposed ML framework incorporates not only a pretrained XGBoost surrogate model but also an optimization engine, such as Bayesian Optimization or Genetic Algorithms, to iteratively explore the input space. This approach combines the speed of surrogate modeling with the strategic exploration capabilities of optimization algorithms, creating a closed-loop system for design and process refinement (Bilgiç and Gök, 2024; Mohan and Samad, 2014; Queipo et al., 2004; Xu et al., 2013). In contrast to conventional Design of Experiments or full-factorial studies, which can be prohibitively expensive in terms of time and resources, the proposed framework requires fewer simulations and yields superior insights. By combining the predictive accuracy of XGBoost with the strategic depth of optimization algorithms, this study provides a comprehensive and efficient approach for advancing CVD process development. The insights gained from this research not only confirm the usefulness of ML in complex physical processes, but also encourage broader adoption in smart manufacturing and material discovery.

In the present study, a machine learning-based optimization framework was developed to enhance the CVD process, with a focus on maximizing the deposition rate and minimizing the uniformity parameter. While prior work has established the predictive accuracy of the XGBoost model in replicating CFD simulation results, this study concentrates on the optimization of these predictions using a surrogate modeling approach. The overall methodology involves the following steps: using XGBoost Predictions as Surrogate Data. Fitting polynomial regression models: To enable fast and interpretable optimization, surrogate models were constructed by fitting second-order polynomial regression equations to the log-transformed XGBoost predictions for D and Up, which were then optimized using the sequential least squares programming (SLSQP) algorithm. The results obtained were then compared with a previously implemented optimization technique to quantify the accuracy and effectiveness of the ML-driven optimization approach.

In the previous phase of the research, a comprehensive data set was generated via CFD simulations of the CVD process (George, 2006). The simulations modeled the deposition of thin films using varying combinations of susceptor temperatures and inlet gas velocities. Two key output variables, namely, the deposition rate and uniformity parameter, were extracted from the simulations. The underlying CFD model has been previously validated, lending credibility to the results.

Two ML models (ANN and XGBoost) were trained to predict D and Up based on these inputs and their architectures are presented in Figure 1.

The comparison between actual (i.e. CFD values) and predicted values is crucial for assessing the accuracy of a ML model. Ideally, the points should align closely along the curve fitting line, indicating perfect prediction accuracy. From the graph, it is evident that the XGBoost model produces predictions that are more tightly clustered around the actual values, demonstrating superior accuracy and minimal deviation. In contrast, the ANN model shows greater scattering, indicating a higher prediction error. Even with hyperparameter tuning, the ANN model still exhibits noticeable deviations, although it performs better than the base ANN model. Table S1 summarizes the performance of the different ML models on the training, validation and test data sets. The error calculated from the different ML models were compared and the results are summarized in Table S2. The improved performance and lower percentage error of the XGBoost model can be attributed to its gradient boosting mechanism, which effectively minimizes errors and enhances predictive accuracy. The XGBoost model outperformed the ANN model in terms of predictive accuracy and robustness, and was selected for further use (Figure S1). The present study leveraged the predictions from this model to guide the optimization of the process parameters.

To facilitate efficient mathematical optimization, the predictions obtained from the XGBoost model over the test data set were used to construct a surrogate model in the form of a polynomial regression equation. The surrogate modeling approach serves as a simplified yet representative approximation of the original ML model, designed specifically to reduce computational complexity during the optimization process (Khachai, 2011; Tenreiro Machado and Lopes, 2017). Given that the XGBoost model is inherently nonlinear and consists of an ensemble of decision trees, it is computationally intensive and not well-suited for direct use within iterative optimization algorithms (Dolotov and Zolotykh, 2020; Firoozabadi and Babaeizadeh, 2019). Therefore, by building a polynomial regression-based surrogate model, the complexity of the optimization task is significantly reduced while maintaining acceptable accuracy in capturing the relationships between the input and output variables (Forrester, 2008; Giunta et al., 2003). In this study, separate polynomial equations were generated for the two primary output variables: the deposition rate and uniformity parameter. These equations express D and Up as functions of the input parameters such as susceptor temperature (⁠$Tsus$⁠) and inlet gas velocity (⁠$Vin$⁠). The test data set, which contains the XGBoost predicted values of D and Up over a well-sampled design space of the input parameters, serves as the basis for fitting these polynomials. The regression model includes linear, quadratic and interaction terms to capture the nonlinear dependencies and cross-effects between the temperature and velocity. Second-order polynomial regression models take the form of equations (1) and (2):

A critical design choice in this surrogate modeling process is the application of a logarithmic transformation to the dependent variables. This transformation has several purposes. First, it helps to linearize the exponential-like behavior that is often observed in physical processes, such as CVD, where deposition rates and uniformity can change drastically with temperature and flow velocity (Fang and Feng, 2023). Second, the logarithmic scale reduces heteroscedasticity, i.e. the nonconstant variance in the prediction residuals, thereby improving the stability and robustness of the regression model (Khaled et al., 2019). Moreover, polynomial regression models are compatible with a wide range of gradient-based optimization algorithms, including Sequential Least Squares Programming (SLSQP), which requires continuous and differentiable functions (Joshy and Hwang, 2024; Ma et al., 2024). The proposed RSM informed ML framework with data-driven model optimization is presented in Figure 2.

The core objective is to assess the effectiveness of the proposed ML-based optimization framework by comparing its outcomes with the results from the original high-fidelity CFD simulations and from prior optimization methods based on traditional surrogate modeling techniques (George, 2006). This comparative evaluation was carried out across three distinct optimization cases, as detailed below, with the aim of quantifying improvements in prediction accuracy, computational efficiency and solution quality. These objectives are framed as constrained optimization problems using surrogate models derived from XGBoost predictions. In all cases, the decision variables are susceptor temperature and inlet gas velocity, whereas the objectives and constraints involve the deposition rate and uniformity parameter.

In this scenario, the optimization problem is formulated to maximize the logarithmic deposition rate log (D), while ensuring that the logarithmic uniformity parameter $log(Up)$ remains below a prespecified threshold $log(Up¯)$⁠. This setup reflects a common industrial objective in which a high deposition throughput is prioritized without compromising film uniformity beyond acceptable limits. The optimization problem is formulated as shown in equation (3):

Maximize: log (D)

subject to:

Here, $Up¯$ denotes the maximum allowable value of the uniformity parameter, and $Tmin$⁠, $Vmin$⁠, $Tmax$⁠, $Vmax$ define the permissible operating ranges.

Here, the optimization seeks to minimize $log(Up)$ while maintaining the deposition rate above a desired threshold $log (D¯)$⁠. This inverse trade-off is useful in scenarios where surface uniformity is critical, such as in optical coatings or electronic devices that require highly homogeneous film deposition. The problem is posed, as shown in equation (4):

Minimize: log (⁠$Up$⁠)

subject to:

where $D¯$ is the minimum required deposition rate based on process feasibility or quality requirements.

The third scenario considered a composite metric deposition-to-uniformity ratio to balance the trade-off between a high deposition rate and good uniformity. This is expressed by equation (5):

Maximize: log $(DUp)$

subject to:

This approach favors solutions that simultaneously improve both the performance metrics in a balanced manner. These optimization tasks were efficiently solved using constrained nonlinear optimization algorithms, such as Sequential Least Squares Programming, implemented in SciPy’s optimize.minimize function. The solver was initialized at multiple feasible points to ensure global feasibility and to avoid local minima.

To evaluate the performance of the proposed ML-based optimization framework, the results obtained from the SLSQP were compared with the baseline results from a previously developed optimization technique. This reference method, possibly involving metaheuristics, provides known optimal values for $D$ and $Up$ under similar constraints. A comparison was made across all three cases listed in Section 2.2.2. The percentage error was computed using equation (6):

This allows a quantitative assessment of the closeness of the ML optimization results to the benchmark. The evaluation not only verifies the validity of the ML optimization framework but also highlights its computational efficiency and generalization potential.

The polynomial regression–based surrogate models accurately reproduced the XGBoost predictions for both deposition rate and uniformity. The high coefficients of determination (R² = 0.998 for deposition rate; R² = 0.989 for uniformity) and low RMSE values (0.007 and 0.004, respectively) demonstrate strong agreement. These metrics align closely with similar surrogate modeling approaches in complex thermal–fluid systems (Forrester and Keane, 2009; Koziel and Pietrenko-Dabrowska, 2020), confirming that a second-order polynomial representation is sufficient to approximate the nonlinear responses observed in CVD processes.

To validate the fidelity of the surrogate models used in the optimization framework, residual plots were generated for both the deposition rate [log (D)] and the uniformity parameter [log (Up)]. These plots display the residuals, defined as the difference between the predicted values from the polynomial regression model and the original XGBoost-based predictions, as a function of the input variables (susceptor temperature and inlet velocity). A well-fitted model should exhibit residuals that are randomly distributed around zero, with no discernible structure (Cook and Tsai, 1985).

As shown in Figure 3, the residuals for both surrogate models were tightly clustered around the zero line, indicating a lack of systematic errors or heteroscedasticity. This suggests that the polynomial models did not suffer from significant underfitting or overfitting. These results provide strong justification for the use of surrogate models in constrained optimization routines. Their accuracy ensures that the optimization outcomes are representative of the underlying physical trends captured by the original ML model without the computational complexity of reevaluating the full XGBoost model.

To better understand the behavior of the surrogate models, both three-dimensional (3D) response surface plots and two-dimensional contour plots were generated. These plots illustrate the dependency of the deposition rate and uniformity parameters on the input parameters, such as susceptor temperature and inlet velocity. Figure 4 shows the 3D surface plot and corresponding contour plot of the predicted deposition rate. The surfaces were smooth and continuous, confirming the adequacy of polynomial regression as a surrogate.

In this case, the optimization objective is to maximize the logarithmic deposition rate while maintaining the uniformity parameter below a specified threshold. The optimization problem was solved multiple times for different specified values of $Up¯$ and the corresponding optimal values of susceptor temperature and inlet velocity were recorded. For each optimization run, the ML-optimized results for D and Up were compared against the values obtained via CFD simulations under the same $Tsus$ and $Vin$ settings, as well as against previous results from Response Surface Methodology based optimization. The optimized deposition rates showed deviations below 3% compared with CFD simulations (Table 1). The XGBoost-based surrogate model slightly underpredicted deposition rates, a conservative tendency consistent with findings by Nguyen et al. (2011), where ML-based surrogates tended to avoid overfitting in high-gradient regions. In contrast, traditional RSM optimization overpredicted rates by 1%–4%, which could lead to unrealistic operational targets.

The ML framework thus provided more physically consistent results, confirming the suitability of XGBoost surrogates for constrained optimization in multiparameter CVD systems. Furthermore, the ML-based framework successfully satisfied the uniformity constraints across all optimization runs, as verified by comparing the predicted and actual up values from CFD. This reinforces the generalizability and robustness of the surrogate model and optimization pipeline.

In Case 2, the optimization task involved minimizing the logarithmic uniformity parameter while ensuring that the deposition rate remained above a prespecified threshold log $D¯$⁠. This formulation represents a practical scenario in CVD processes, where the priority is to achieve better film uniformity without compromising the minimum acceptable deposition rate required for functional performance. The variation of the predicted uniformity across the design space is illustrated in Figure 5, which presents the 3D response surface and corresponding contour distribution generated from the XGBoost model. The ML-based optimization achieved uniformity parameter predictions within ±5% of CFD results, closely matching the performance of traditional surrogate models but with reduced computation time. The model captured the nonlinear trade-off between film uniformity and deposition rate – an effect similarly observed by Kim and Lee (2018) in CVD of Si and by Niu et al. (2023) in GaN film growth.

Some test cases showed overprediction (positive error), where the model estimated a better uniformity than CFD, whereas others exhibited underprediction (negative error), indicating conservative estimates. This trend was consistent across various threshold levels of $D¯$⁠, suggesting that both surrogate models effectively captured the overall response behavior but with comparable limitations. Although the ML model did not consistently outperform the RSM in every case, it demonstrated equivalent accuracy while offering advantages such as faster optimization time and greater adaptability owing to its data-driven nature. The similarity in error margins validates the credibility of the ML framework, particularly considering that it was built from high-fidelity XGBoost predictions, which better generalizes nonlinear dependencies. It was also observed that the surrogate models occasionally struggled near the constraint boundaries, likely owing to the reduced data density in those regions. Despite this, the ML approach-maintained stability and convergence across all optimization runs. A quantitative comparison of optimal conditions and prediction errors for multiple specified values of log D is provided in Table 2, further confirming the consistency and robustness of the proposed optimization approach.

The third optimization formulation focuses on maximizing the ratio log(D/Up), which represents a comprehensive performance metric that integrates both the deposition rate (D) and film uniformity (Up). This ratio inherently balances the trade-off between throughput and quality, making it an ideal target for optimizing CVD processes, where both aspects are critical for industrial viability. Unlike the previous cases, which involved constraint-driven multi-objective formulations, Case 3 was framed as a single-objective unconstrained optimization (subject only to physical bounds on the susceptor temperature and inlet velocity). This simplified structure not only facilitates efficient convergence but also aligns with real-world scenarios in which a single efficiency metric can streamline process tuning.

The ML-optimized solution achieved an average 3.2% higher deposition-to-uniformity ratio than RSM optimization and exhibited a mean deviation of less than 5% from CFD evaluations (Table 3). The surrogate model effectively balanced deposition kinetics and morphological uniformity, reaffirming the framework’s robustness. Such combined metrics have been advocated in advanced coating optimization studies – Long et al. (2013) used similar performance ratios for SiC coatings to evaluate reactor-scale uniformity versus deposition efficiency. The alignment of the present findings with these prior trends underscores the physical credibility of the ML-driven optimization outcomes.

The 3D surface plot and contour plot depicting the behavior of the surrogate model for log(D/Up), along with the identified optimal point, are presented in Figure 6, which visually illustrates the optimization landscape and model response. A quantitative summary of the ML, RSM and CFD results is presented in Table 3. This table highlights the predicted values of the deposition rate, uniformity parameter and D/Up ratio for all three approaches.

Importantly, the average deviation between the ML-optimized predictions and CFD-evaluated values was less than 5%, confirming the reliability of the surrogate model even in a complex, nonlinear objective space. This consistency demonstrates the robustness of the ML framework and its ability to generalize across various optimization scenarios. Therefore, the optimization of the D/Up ratio via the proposed ML pipeline not only enhances the process performance, but also provides a scalable approach for future adaptive CVD process control.

This study presented a surrogate model–assisted ML optimization framework for the CVD of aerospace coatings. By integrating XGBoost predictions, polynomial surrogate modeling and SLSQP-based constrained optimization, the framework enabled comprehensive exploration of the process parameter space with minimal computational cost. The optimized model achieved deposition rate prediction errors below 3% and uniformity prediction errors within 5% under all constraint conditions, with a 3%–5% improvement in the deposition-to-uniformity ratio compared to traditional RSM methods. In addition, the computational cost was reduced by approximately 60% relative to direct CFD-based optimization. The identified optimal process window, with susceptor temperatures ranging from 880 to 1220 K and inlet velocities between 0.4 and 1.2 m/s, achieved both high deposition efficiency and superior film uniformity. Compared with earlier surrogate and experimental CVD optimization studies, the present framework demonstrated comparable accuracy with significantly lower computational demand. Overall, the developed ML framework provides a scalable, physics-informed and computationally efficient approach for accelerating coating process design and its modular architecture makes it adaptable to various aerospace coating applications, including thermal barrier, oxide film and environmental barrier coatings. Future work will focus on extending this framework toward multi-objective reinforcement learning and real-time feedback control for next-generation intelligent manufacturing systems.

The supplementary material for this article can be found online.