Pattern recognition with limited data: an AI model inspired by BCR-Net and SwitchNet Open Access

Despite advancements in deep learning (DL) for large-scale data analysis, many AI applications lack access to extensive datasets [1]. This challenge is evident in personalized systems, where DNNs rely on small datasets from search histories, inventory reports, and biometric measurements. Since deep learning focuses on mastering knowledge, personalization demands precise modeling while avoiding overfitting when working with small datasets. For example, in a recent MAPI survey, 58% of research participants reported that the most significant barrier to deploying AI solutions pertained to a paucity of data resources [2]. Therefore, making deep learning of AI work with small data has become a significant predicament. Limited research has been done in this area, even though there is a great need to design intelligent pattern recognition solutions in many application domains [3, 4]. Consequently, in this paper, we design a Pattern Recognition System (PRS) that recognizes patterns with small data using Deep Neural Network (DNN) and IP. Some inverse problems can be viewed as pattern recognition problems [5], thus making them amenable to deep neural networks. Deep learning (Deep Neural Networks, DNNs) has seen a remarkable surge in capabilities, emerging as highly effective instruments for addressing pattern recognition challenges, particularly in domains such as medical image segmentation and classification [6, 7]. Based on deep neural networks, deep learning methods have demonstrated superior performance and increased reliability compared to traditional approaches in a wide range of applications. Nonetheless, achieving these advanced results comes at the cost of requiring more hardware resources [8]. However, the multi-layer structure of DNNs can be adapted to learn effectively even with limited data. To allow IP-enabled PRS to make compelling predictions, we investigate how inverse problems connect to deep learning with small data.

We designed a model inspired by the BCR-Net and SwitchNet to successfully discover the relationship between deep learning and IP. We leverage and adapt a variant of BCR-Net and SwitchNet [9–11]. The adopted version is naturally sparse and computationally efficient. We also design a model that captures the underlying patterns in the region of interest (ROI). As a sparse model, it selectively captures the essential relationships between input and output variables and among the output variables themselves. Constructing a sparse PRS model focuses on learning the crucial dependencies that aid in predicting patterns, disregarding unnecessary ones.

To evaluate the performance of PRS, we applied it to the pattern recognition-based reconstruction of an inhomogeneous object. The BCR and IP-based deep learning model of PRS only requires a small amount of data to recognize patterns, making it an easily deployable model in practice. Furthermore, our experiments demonstrate that the PRS model performs well in effectively uncovering data dependencies, making it a desirable and practically feasible prediction tool due to its low computational requirements. Also, IP-based PRS, due to its sparse nature and ability to capture only the necessary dependencies between the input and output variables, is an interpretable model desirable for many AI models. Combining these two tools in one pattern recognition descriptor shows that IP-based augmentation of our model can improve the average pattern recognition performance.

In order to address the adverse effects of limited data availability and its associated costs, numerous researchers have recently directed their efforts toward developing systems capable of functioning effectively with smaller datasets. Numerous studies have explored the challenges of learning from limited data, and our approach builds on four key areas that have emerged in this domain: transfer learning and autoencoder-based initialization, data augmentation and compact architecture, physics-informed modeling, and meta-learning strategies for few-shot learning.

Transfer learning remains one of the most effective approaches for low-resource scenarios. In this setting, models trained on large-scale datasets are fine-tuned on a smaller target domain, leveraging the reusability of early-layer features across tasks [12, 13]. Closely related, stacked autoencoders have been used to pretrain deep neural networks in small-data regimes, enabling better weight initialization and convergence [14].

Data augmentation and model regularization techniques have also been extensively explored to improve generalization when labeled samples are scarce. Data-augmentation-assisted deep learning has proven effective in classifying and monitoring contexts [15]. At the same time, elastic-net and sparsity-promoting structures such as partially connected DNNs offer competitive results in synthetic and practical tasks [16]. FuCiTNet, a GAN-inspired model, fuses class-inherent transformations to enhance generalization from few examples [17]. Further advancements include the development of feature generalization layers (FGLs) [18], multi-branch and parallel convolutional networks [19–21], and domain-specific augmentation strategies tailored to healthcare and bioinformatics applications [22–25].

Recent efforts have also focused on embedding domain knowledge or structural priors directly into network architectures. Our work is influenced by BCR-Net and SwitchNet, which model integral operators and form the foundation of our architecture. Wavelet-based decomposition has long been used to compress operators and represent functions efficiently in hierarchical bases, a principle that underpins the BCR-Net architecture. More recent work has explored incorporating wavelet structures directly into convolutional neural networks to enhance feature extraction and classification performance [26]. In the broader class of physics-informed neural networks (PINNs), deep architecture has been applied to forward and inverse problems with embedded physical constraints [27]. Hybrid approaches combining residual learning with structured priors have also gained traction [28]. Additional related works include YOLO-based segmentation models [29], transformer-inspired SegFormer architectures [30], and hybrid methods such as PCA-GOA for masked face recognition tasks [31, 32]. Unlike those models, our inverse formulation embeds mathematical operator structure directly into the architecture.

Complementing these efforts are meta-learning and few-shot learning frameworks, which seek to generalize across tasks using minimal data per class. Prototypical Networks operate by embedding inputs into a metric space and classifying based on proximity to class prototypes [33]. Matching Networks extend this paradigm by incorporating attention-based weighting of support-query relationships [34]. MAML (Model-Agnostic Meta-Learning) focuses on learning initialization parameters that enable fast adaptation to new tasks through just a few gradient updates [35]. While powerful, these meta-learning techniques typically require access to large task distributions and extensive episodic training. In contrast, our approach leverages structured inverse operators to support learning under data scarcity without relying on external tasks or data sources.

In summary, prior research demonstrates a range of practical and theoretical strategies for addressing pattern recognition under small-data constraints. Within this context, our approach aims to contribute a unified, inverse-operator-guided deep learning framework that supports generalization in limited-data settings.

In this section, we explore the development of a model capable of performing pattern prediction tasks with limited data. Specifically, we aim to reconstruct an unknown, complex function in high-dimensional space by effectively matching available data. Additionally, we provide a comprehensive overview of the data generated, which serves as a means to validate the performance of our model. The PRS model is evaluated on both synthetic datasets and the BBBC real-world dataset to test its real-world applicability.

In this section, we aim to outline the model to predict patterns based on synthetic data generated from regions of interest (ROI). Pattern recognition involves reconstructing an unknown high-dimensional nonlinear function that matches data, which can be formulated as a classic inverse problem. Inverse problems are generally modeled using physics-based principles, which can aid in reconstructing high-dimensional nonlinear functions with available data. However, inverse problems typically do not require significant data compared to DNN. Inverse map framework for DNN can solve pattern recognition problems with small data.

We frame the pattern recognition task as an inverse problem by leveraging BCR-Net and SwitchNet. This study introduces an AI-powered model integrating inverse problem (IP) techniques with neural network architectures for enhanced small-data learning. This novel approach to the pattern recognition problem demonstrates empirically that it performs better than the SVM model, which is the most widely used pattern recognition technique. In the following section, we elaborate on the rationale behind selecting the aforementioned model and highlight the distinctive features of our system.

In this paper, we use two groups of synthetic data sets for experimental evaluation of the proposed method. The first of these data sets is based on the inclusion function in which we have two small objects to recognize in the domain; for the experimental evaluation of our proposed model, we employ two sets of synthetic data. The first set revolves around the inclusion function, wherein we have two small objects within the domain that need to be recognized in the domain [−2,2], shown in Figure 1.

The input values for our synthetic data sets are uniformly generated within the range [−2,2]. The corresponding target values are derived using the characteristic functions of the two objects involved in the inclusion function,

In the second data set, a classification problem having two manifolds is shown in Figure 2. The goal is to detect the location of the blue and purple classes. Specifically, we look for the center of these two objects. Table 1 dataset Summary summarizes the characteristics of both datasets used. The inclusion dataset is fully synthetic, while the manifold set comes from the BBBC022 real-world dataset. Each set was divided into 80% training and 20% testing samples.

The dataset comes from the Broad Bioimage Benchmark Collection (BBBC022) and comprises fluorescence microscopy images of human U2OS cells, capturing the effects of small-molecule treatments on cell morphology. The dataset structure and annotation details are available in the Supplementary Material. It includes high-quality images and manually labeled ground-truth annotations, serving as a gold standard for evaluating machine learning models in image processing and pattern recognition. These expert-verified annotations enable precise assessment of our model’s performance in image-based profiling tasks. In this case, the class priors and class-conditional densities are known, allowing for the straightforward determination of corresponding values for input data [27, 28].

A core challenge for inverse problems and artificial intelligence (AI) is reconstructing an unknown high-dimensional nonlinear function that matches data. Inverse problems generally have physics-inspired models to represent them, which may help reconstruct high-dimensional non-linear functions with the help of data [29, 30]. However, inverse problems do not require much data compared to DNN; if we use an inverse map framework for DNN, we may be able to solve them with small data. In other words, inverse frameworks inspired by physics help us solve DNN problems with small data. The inverse map architecture in this context serves as a natural data augmentation method.

We first introduce deep learning through the lens of an inverse map architecture, which offers a potential solution to the challenge of requiring large datasets. Unlike many applications in deep learning, such as image processing and pattern recognition, which require a massive amount of data, the required amount for inverse problems may be minimal. We plan to leverage an inverse map to design an NN module based on an integral operator, assemble a DNN from this module, and then use data to train the weights of the inverse map. The IP framework has two major strengths: it works well with limited data, and it is naturally suited for regression problems where the goal is to recover continuous values or parameters from indirect measurements. As a result, pattern recognition can be framed as a supervised regression problem, enabling precise reconstruction even in data-scarce conditions. In this approach, data and parameters work together, enhancing the overall predictive performance. This better architecture, motivated by physics and mathematics, can even work with small data.

To describe a physical situation in mathematical terms, one has to make a certain idealization so that the problem may be amenable to mathematical treatment. Without oversimplification, the model’s success is governed by how close to reality the idealized model is. Various mathematical models have been used in pattern recognition to explain the dynamic behavior of a system. Many involve the formulation and solution of inverse problems, which are generally modeled by considering the model’s components, design variables, and system parameters as deterministic values. In the inverse map framework used here, the dominant factor influencing the accuracy of pattern identification is attributed to the contrast estimator. Here, we briefly describe how the pattern recognition problem of NN can be converted into a suitable inverse problem [5]. In the framework of NN, we have the following integral equation of the first kind

where

Moreover, $A$ is an operator from $L∞(Ω)$ into $L∞(Ωk)$⁠. The integral equation mentioned above belongs to the first kind of Fredholm integral equations, featuring a kernel of Riesz type. Significantly, the operator $A$ in (2.1) can be classified as a convolution integral operator employing the Riesz-type kernel. Initially, it is crucial to note that the equation represents a highly ill-posed problem, yet a unique solution f to equation (2.1) exists and is unique. Furthermore, a logarithmic stability estimate supports the solution’s stability [5]. In order to address the issue of ill-posedness, instead of Equation (2.1), we solve the regularized equation:

where $α$ is a regularization parameter and $I$ is the identity matrix.

In (2.2), we may consider $f(x)≈(A*A+αI)−1A*g(x)$⁠. As described in [6, 7], first use the SwitchNet module to represent $A$ and its adjoint $A*$⁠, and then second, represent $(A*A+αI)−1$ integral operator by BCRNet to define the neural network architecture as

A simplified overview of the proposed Pattern Recognition System (PRS) model is provided in Figure 3. Input data $g(x)$ is first processed by SwitchNet, which approximates the operators $A$ and $A*$⁠. The output is then passed to BCR-Net, which approximates the inverse operator $(A*A+αI)−1$⁠. This structured design utilizes low-rank approximations and wavelet-based decompositions to produce the predicted output, $f(x)$⁠.

We suggest a combined neural network architecture, incorporating BCRNet, a newly developed neural network architecture inspired by the non-standard form proposed, and SwitchNet, a neural network model equipped with an Inverse Map. Although it is feasible to address this problem using a fully connected network, the parameter counts increase exponentially with the input and output data dimensions. By exploiting the inherent low-rank structure of the inverse conductivity problem defined in (2.2), the introduction of BCRNet and SwitchNet architectures significantly reduces the number of parameters involved, thereby streamlining the training process.

In inverse map $g(x)$ → $(x)$ , $f(x)≈(A*A+αI)−1A*g(x)$ where $A*$ and $(A*A+αI)−1$ can be represented with a 1D CNN and BCR-Net, respectively.

We now discuss the motivation modules SwitchNet and BCRNet and show how to use the new modules to correctly solve these newest problem applications. All successful neural network (NN) architectures, such as fully connected, convolutional, recurrent, and residual networks, are fundamentally grounded in underlying principles of mathematics and physics. Convolutional neural networks are commonly used for image processing tasks, recurrent neural networks for speech and sequence data, and residual neural networks represent one of the most advanced architectures in deep learning. Fully connected NN has a dense operator behind it, whereas the convolution NN has a translation-invariant operator. All these neural networks have mathematics behind them.

The natural question is, given the integral operator in the inverse problem that was stated in (2.2), what would the neural network architecture be associated with the integral operator? We have the right building blocks if we can build the neural network architecture for this integral operator. When represented as a matrix, the integral operator $A$ exhibits a key property [Figure 4]: Its off-diagonal blocks are numerically low-rank, allowing for a sparse representation. This enables these blocks to be replaced with efficient numerical approximations, significantly enhancing computational performance.

Therefore, the plan is to find a convenient sparse-compact representation of the integral operator. Here, we use the BCR-net method inspired by the non-standard wavelet form proposed by Beylkin, Coffman, and Rokhlin [8] for an approximation of a non-linear integral operator given as an $nxn$ matrix $A$ utilizing merely tens of thousands of parameters in the following way:

For a given integral operator, the square matrix can be written as a product of three matrices because of the low rank of the off-diagonal blocks. In Figure 5, the first and last matrices represent the redundant wavelet transformation. This one-to-one orthogonal mapping preserves the degrees of freedom while retaining wavelet and scaling coefficients within the transformation. This transformation increases the size by a factor of two. At the same time, the middle matrix, known as the nonstandard form, consists of three diagonally dominant matrices that provide an efficient approximation of the original operator, forming the core idea of BCR-Net.

A natural question arises: why is this linked to neural networks? As shown in Figure 6(a–c), the computation at each scale can be effectively encoded within a neural network architecture.

When the input vector is applied, it corresponds to a three-layer network. The first step involves multiplying two diagonal matrices, and assuming these operators are translation-invariant, this operation transforms into a convolution, mapping one channel to two channels. The middle operator acts as a transformation from two channels to two, followed by a final step that reduces the two channels back to one. This structure defines the neural network architecture for a single layer. The complete neural network is constructed by stacking these layers sequentially. The top layer represents the finest.

scale, where wavelet coefficients are computed. These coefficients are combined with scaling coefficients to propagate information to the next layer. The three-layer structure is illustrated in Figure 6(b).

However, for significant problems, one may have many layers. The idea of transforming this into a neural network is to insert a few more internal layers and introduce a non-linearity by inserting ReLUs, so this will give us a nonlinear generalized version of BCR-net shown in 6(c).

Given an $nxn$ integral operator $A$⁠, we may approximate nonlinear operator $A$ by partitioning it into $n1/2x$ n^1/2, applying low-rank approximation to each block, and generalizing this via inserting ReLUs [Figure 7].

In this section, we investigate the efficacy of the PRS learning algorithm introduced in the previous sections. One way to objectively test such a model is to apply it to computational pattern recognition network models for which the mechanisms are entirely known, as described in Section 3.2.

In this study, we denote ‘real object’ by $fr$ and ‘computed object’ by $fc$⁠. Generating $g(x)$ in the integral equation (2.1) is essential to evaluate the method. Initially, we define $f(x)$ and subsequently compute $g(x)$ numerically. Afterward, we utilize the numerical values of $g(x)$ as our dataset and extract the underlying pattern within the region of interest.

To ensure transparency and facilitate reproducibility, we present a high-level pseudocode and a formal algorithmic description of the proposed Pattern Reconstruction System (PRS) model. The pseudocode captures the essential conceptual steps of the methodology, abstracted for clarity. At the same time, the accompanying algorithm provides a more detailed procedural breakdown aligned with the mathematical framework introduced in Section 2.

The pseudocode outlines the construction of training pairs using the inverse operator $A$⁠, the generation of forward data, model training, and performance evaluation. In contrast, the formal algorithm incorporates dataset discretization and numerical computation of the integral operator as defined in Equation (2.1), offering a comprehensive view of the PRS training and prediction pipeline.

Both formats are presented side by side in Table 2 to aid in comparative interpretation.

Table 3 presents our experiments’ detailed model configurations and hyperparameter settings. These include the choice of optimizer, learning rate, batch size, regularization coefficient, network architecture, and activation functions. The configurations were selected based on standard practices and empirical tuning to ensure fairness and optimal performance across all models.

The first group of experiments was carried out for synthetic data sets on recognizing two small objects in the domain $[−2,2]x[−2,2]$⁠. The reconstruction closely approximates α the inclusion domain, providing a highly accurate representation [Figure 8]. Based on the presented reconstructions, the numerical experiments mentioned earlier have demonstrated an intense match between the predicted and actual inclusion domains. The model generated each inclusion domain with an accuracy of 92.67%.

In the second set of experiments, we aimed to detect the centers of the blue and purple classes in the classification problem illustrated in Figure 2. In Figure 8, the center of the location of the objects was well produced, and the proposed PRS model could distinguish between the two objects. To optimize efficiency, the key factors to adjust in Eq. (2.1) are the regularization parameter and the number of grid points. These parameters play a crucial role in the overall performance of the process.

To validate the reconstruction ability of the proposed model, we used the above-described synthetically generated data whose distributions are known (Figure 9(a) and (c)). The results of this experiment are presented in Figure 9(b) and (d). The PRS model demonstrated strong and consistent performance across all three datasets SOD, SMD, and MMD, as defined in Table 4, spanning tasks from small-object classification to complex multi-manifold recognition. To comprehensively evaluate PRS, we compared it with two established baseline models, SVM and LSTM, using seven metrics: Average Squared Error (ASE), Misclassification Rate (MR), Kolmogorov–Smirnov statistic (KS), Precision, Recall, F1-score, and ROC-AUC.

As shown in Table 4, PRS outperforms both SVM and LSTM in most metrics across all datasets. For instance, in the MMD dataset, which presents the highest complexity with 10 classes, PRS achieves a lower ASE (0.0519), lower MR (0.0680), and higher KS (0.6600), indicating better alignment between predicted and actual distributions. Furthermore, PRS leads in classification metrics with a Precision of 0.90, a Recall of 0.91, an F1-score of 0.905, and an ROC-AUC of 0.92, surpassing both SVM and LSTM.

These trends hold across the SOD and SMD datasets as well, where PRS consistently yields better or comparable ASE and MR, and shows powerful results in Recall and ROC-AUC, which are crucial for high-sensitivity applications. KS statistics, a measure of distributional similarity, also confirms PRS’s advantage in maintaining output stability and accuracy across classes.

The ROC analysis presented in Figure 10 highlights the superior discriminative performance of the PRS model in the MMD object recognition task. Compared to SVM and LSTM, PRS consistently achieves higher sensitivity across a wide range of specificity thresholds. Notably, aligning the PRS validation curve (dashed line) with its training and test curves indicates minimal overfitting and robust generalization to unseen data. These results underscore PRS’s effectiveness in handling complex, multi-manifold classification challenges and its advantage in data-scarce environments.

To get a well-rounded comparison, we tested the PRS model against traditional approaches LSTM and SVM, as well as more advanced techniques such as Prototypical Networks and Transfer Learning, which are known to perform well with limited data. These comparisons helped us better understand where PRS regarding accuracy and efficiency. While Table 5 shows that PRS takes longer to train than the other four models, this drawback is offset by not requiring repeated cross-validation to tune complexity parameters. Additionally, as it generates models with fewer variables, the computational time for evaluating test points, often the primary concern in practical scenarios, is significantly reduced. Based on the average training times for the BBBC022 dataset, the PRS model demonstrates a competitive balance between performance and computational efficiency. While its training times, 62.3 seconds for Dataset 1 (Inclusion) and 75.1 seconds for Dataset 2 (Manifold), are higher than those of simpler models like SVM (15.8s/20.3s) and LSTM (38.2s/46.7s), PRS outperforms them in terms of accuracy, making the extra training cost justifiable. Compared to Prototypical Networks (50.0s/58.0s) and Transfer Learning (70.0s/85.0s), PRS sits comfortably in the middle, offering accuracy nearly on par with Transfer Learning but at a slightly lower training cost. This makes PRS a reasonable alternative to established models, particularly in scenarios where high pattern recognition accuracy is essential. However, complete reliance on heavy pretraining (as in Transfer Learning) is less desirable.

The PRS model demonstrates strong performance on controlled datasets, yet its generalization to noisy or highly imbalanced real-world data has not been extensively tested. Training time is higher than conventional models, though offset by lower testing cost. Future work will explore model robustness to noise and domain transfer.

Table 6 compares the performance of five different models, LSTM, Prototypical Networks, Transfer Learning, SVM, and PRS on the BBBC022 dataset (MMD), a benchmark for pattern recognition in bioimage analysis. Transfer Learning led the pack with a top accuracy of 92%, but PRS followed closely at 91%, showing it can match the best while offering unique strengths in adaptability and computational efficiency. Prototypical Networks also delivered solid results at 88%. LSTM and SVM lagged with 82% and 80%, respectively, highlighting their limitations in this setting.

Overall, PRS proves to be a strong and reliable contender for pattern recognition tasks in biomedical imaging.

The experiment showcases our framework’s successful implementation and effectiveness while demonstrating the practicality of IP-based PRS. The comparative findings reveal that it achieves 90% accuracy across multiple applications. Furthermore, we have compared our IP-based PRS model to baseline models, LSTM and SVM, and advanced models for small data scenarios such as Prototypical Networks and Transfer learning. Our results surpassed those of SVM, LSTM, and Prototypical Networks by 11%, 9%, and 3%, respectively. On the other hand, Transfer Learning did better than the PRS by 3%, but the PRS model offers a favorable trade-off between accuracy and training efficiency. The proposed inverse map framework for DNN makes it possible for the model to achieve strong generalization from limited data. That makes it well-suited for small data prediction problems where both predictive accuracy and computational efficiency are critical.

Nevertheless, the model has limitations, highlighting the need for further examination.

Future work will explore domain generalization, adaptability to noisy data, better numerics, and extending PRS to more complex and heterogeneous datasets. Integrating physics-based operators into deep learning offers a promising path for improving performance in data-scarce applications.

Additionally, the PRS model was developed using moderate computing power (a mid-range GPU setup), and its sparse architecture allows it to maintain practical training times without requiring specialized hardware. This makes PRS a cost-effective and accessible solution for small-data problems in academic or clinical settings.

Overall, we aim for this approach to serve as both a challenge and a benchmark. By leveraging the close relationship between inverse problems and pattern recognition, we can create an innovative learning algorithm suitable for handling small data sets.

The supplementary material for this article can be found online.