Design and evaluation of software reference architecture: a systematic mapping study

Software Reference Architectures (SRA's) are reusable architectural frameworks that guide the development of software systems within an application domain [1–4]. Researchers argue that SRA's serve as architectural blueprints from which concrete system architectures can be instantiated, enabling organizations to ensure consistency and quality even when development is distributed across teams, locations, or organizational boundaries. Their use has expanded across sectors such as healthcare, automotive, finance, and critical systems. The well-established examples, such as CORBA in distributed computing, AUTOSAR in automotive software, and FHIR in healthcare information systems, demonstrate how SRAs can reduce design effort and streamline system integration in these domains [5, 6]. The benefits commonly attributed to SRAs include reduced development cost, increased architectural consistency, improved interoperability, and enhanced maintainability [7].

Despite their relevance, research on how SRAs are designed and evaluated remains underexplored. Much of the existing literature focuses on emphasizing and classification. Thus, there is a need to consolidate current knowledge on the SRA design and evaluation process. To address the research gap, this study employs a systematic mapping study, a method that provides an overview of a research field by classifying and structuring existing literature. A systematic mapping approach is particularly appropriate in the context of SRAs, where research is conceptually diverse, spans multiple domains, and exhibits variation in design and evaluation practices. The guidelines proposed by Ref. [8] were used, as they are popular for conducting systematic mapping studies in the software engineering domain. A mapping study will help clarify the current state of the field and identify areas where empirical knowledge remains limited [9]. The objectives of the study are to characterize the evolution of SRA research over time, identify the knowledge sources that inform SRA design, examine the methods, frameworks, tools, and architectural artifacts used to design SRAs, and analyze the evaluation techniques, quality attributes, and applied metrics used to assess their effectiveness.

The key contributions of the systematic mapping study include: 1) providing an evidence-based synthesis of how SRAs are designed and evaluated, 2) offering a comprehensive classification of the design and evaluation approaches, and 3) setting a research agenda that aims at strengthening the SRA design and evaluation process. The remainder of this paper is structured as follows. The background section reviews the theoretical background and related research on SRA's. The methodology section outlines the design and execution of the mapping study. The reporting section presents the results of the classification and analysis. The discussion section presents the study's key findings and implications. Finally, the paper concludes with a summary of key insights and recommendations for future work.

Software architecture defines the high-level structure, components, and interactions of software systems, outlining the design principles that guide their development [10, 11]. Over time, the field has diversified into several subdisciplines, including enterprise architecture, service-oriented architecture, and software reference architecture (SRA) [12, 13]. Among these, SRAs play a distinctive role by combining domain-specific knowledge with architectural best practices to guide the development of software systems within a particular domain [14]. SRA encapsulates domain knowledge and reusable design patterns in architectural layers, components, interface definitions, and communication protocols [15]. It provides a structured framework for building domain-specific software applications while ensuring architectural consistency and interoperability across multiple systems. For example, CORBA, AUTOSAR, SAP, and IBM's Cloud Reference Architecture serve as blueprints for deriving concrete architecture tailored to specific contexts. SRAs are designed to be instantiated and adapted to meet the specific requirements of individual projects, enabling developers to build systems that align with a shared architectural vision [16, 17].

SRA's offer a range of benefits that make them valuable assets in software engineering practice. According to Ref. [18], one of the primary advantages is reusability, as SRAs enable the reuse of common functionalities, architectural components, and design patterns across multiple software products within a domain. This reuse promotes consistency, reduces redundancy, and streamlines the development process. In addition, SRAs contribute to risk reduction, since they are typically derived from thorough analyses of domain requirements and proven design solutions, lowering the likelihood of system failure during implementation [19]. Another significant advantage is cost and time efficiency, as SRAs facilitate component reuse and standardization, thereby accelerating development cycles, minimizing rework, and shortening the time-to-market for new applications [20]. These benefits enhance software quality, maintainability, and interoperability across diverse systems. However, adopting SRAs is not without challenges. One major limitation concerns the potential reduction of innovation, as software reference architectures often prescribe specific component choices and structural constraints that may restrict creative or unconventional design decisions [20], leading to architectural uniformity that hinders experimentation and domain-driven customization. Moreover, design flaws or suboptimal decisions made during the creation of an SRA can propagate to all derived systems, amplifying architectural weaknesses and leading to systemic inefficiencies [7, 21]. These challenges highlight the importance of rigorous design, continuous evaluation, and regular revision of SRAs to ensure they remain adaptable, relevant, and capable of supporting innovation while maintaining architectural coherence.

Secondary studies have been conducted in recent years to synthesize knowledge on SRAs across different domains. Most of these studies focus on specific application areas rather than providing a broad analysis of SRAs. For instance Ref. [22], surveyed architectural frameworks in edge computing, providing insights into industrial adoption trends. Similarly [23], Internet of Things (IoT) architectural styles, identifying recurring architectural solutions and design constraints. In another domain-specific effort [24], surveyed cyber reference architectures, focusing on their structural patterns and security implications. Beyond domain-focused reviews, a few studies have examined SRAs more generally. For example [25], context, goals, perspectives, and application domains. Table 1 summarizes the focus and contributions of existing secondary studies on reference architecture across different domains.

These studies collectively contribute valuable insights into how SRA's are defined. However, they tend to emphasize categorization and classification rather than exploring the processes underlying the design and evaluation of SRAs. The present study extends this line of research by adopting a more process-oriented perspective. Rather than focusing solely on categorization or domain-based characteristics, this study examines the end-to-end lifecycle of SRAs, including their design and evaluation. By integrating both design and evaluation dimensions, this systematic mapping provides a more comprehensive synthesis of SRA research, revealing methodological patterns, evidence gaps, and opportunities for future work.

This study followed a systematic mapping study, which provides a structured and repeatable process for identifying, classifying, and visualizing the state of research within a software engineering domain. Systematic mapping focuses on how research is distributed across publication outlets, topics, domains, methods, and evaluation strategies. The mapping process consisted of the following stages: (1) defining research questions, (2) searching for relevant studies, (3) applying inclusion and exclusion criteria, and (4) data extraction and synthesis.

To guide the systematic mapping and ensure structured data extraction and analysis, three research questions were formulated. These questions reflect both the current state of SRA research and the methodological practices associated with SRA design and evaluation. Table 2 lists the research questions and their goals.

The authors selected the Scopus database for its comprehensive indexing of peer-reviewed literature in software engineering. To complement database retrieval, backward and forward snowballing were applied to ensure the inclusion of studies referenced within the selected papers. Search was limited to English-language, full-text publications after 2010. The search string was iteratively refined through pilot tests: (“software reference architecture” OR “reference architecture” OR “reference model”).

The following Inclusion/Exclusion criteria were developed to filter out irrelevant papers. After applying these criteria, n = 75 primary studies were selected for detailed analysis.

1. IC1: The study provides empirical, conceptual, or methodological insights into SRAs.

2. IC2: The study focuses on the design and evaluation of a Software Reference Architecture.

3. IC3: The paper contains sufficient information to address the defined research questions

1. EC1: Not a primary or empirical study (e.g. review, position paper).

2. EC2: Not written in English.

3. EC3: Full text unavailable.

Data extraction was performed based on the following categories:

1. Bibliographic Information: Author(s), publication year, venue type (journal/conference), country/region, collaboration type (academic, industrial, or mixed), and Application domain.

2. SRA Design Characteristics: Knowledge sources (e.g. standards, domain analysis, stakeholder input), design approaches (model-driven, pattern-based, variability-oriented), modeling notations, and design tools.

3. SRA Evaluation Characteristics: Evaluation methods (e.g. case study, simulation, expert review), metrics and quality attributes (e.g. modifiability, performance, reusability), and measurement methods.

The data extraction process was collaborative to avoid bias. A pilot test was also conducted to ensure the data's relevance. Mixed-method data collection was conducted to provide both qualitative and quantitative feedback. Collected data were stored in a spreadsheet and further analyzed to provide visual presentations.

The findings are structured according to the three research questions. By synthesizing quantitative distributions and qualitative evidence, the results highlight the depth of methodological practices.

SRA research has shown steady and accelerating growth. Early contributions from 2010 to 2014 (12%, n = 9) were largely conceptual, introducing foundational frameworks for SRA modeling. However, the period from 2018 to 2025 (33.3%, n = 25) witnessed a substantial rise in publications, coinciding with the broader digital transformation in Industry 4.0, cyber-physical systems, and AI-driven infrastructure. The peak output occurred in 2024–2025, reflecting strong interest in SRA across industrial, cloud, and intelligent system domains. This trend signifies a methodological shift from exploratory theoretical studies to empirically validated, domain-oriented SRA's. The analysis reveals nearly one-third of all publications are concentrated in the final two years of the analysis window (2024–2025), strongly suggesting that SRA research is not merely growing but entering an exponential phase of expansion. This trend is likely to continue and even accelerate, driven by several key technological frontiers such as IoT, AI, and Large Language Models (LLM). Figure 1 illustrates the publication trend after 2010.

The analysis of publication venues indicates the field's academic maturation and credibility. A strong majority of the studies (58.7%, n = 44) were published in peer-reviewed journals, indicating a focus on disseminating mature, validated, and significant research contributions. Conference papers (41.3%, n = 31) serve as a vital platform for presenting novel ideas and early-stage research as well as fostering discussion. Research on the design and evaluation of SRA is disseminated through a diverse set of high-impact, reputable venues. Leading publishers include Elsevier (n = 20), Springer (n = 13), IEEE (n = 12), and MDPI (n = 11 papers). Prominent journals include Journal of Systems and Software, Software and Systems Modeling, IEEE Access, MDPI Sustainability, Automation in Construction, and Expert Systems. Major conference outlets include ACM, IEEE, and Springer. The publication pattern indicates that the field has reached a significant level of maturity, with research sufficiently well validated for dissemination in peer-reviewed journals.

SRA research has expanded dramatically to address architectural challenges across a wide spectrum of industries. The most prominent category, Cloud, IoT and Cyber-Physical Systems (24%, n = 18), reflects contemporary technological trends and includes SRAs for DevOps in multi-cloud IoT applications, IoT-enabled smart buildings, smart grids, and Industrial IoT. Followed by Data and Artificial Intelligence (16%, n = 12), encompassing big data systems, cloud-agnostic data analytics, distributed AI services, and LLM-integrated systems. Thirdly, Healthcare and Well-being (12%, n = 9) includes internet-delivered psychological treatment, virtual reality exposure therapy, health information systems, and healthcare privacy. Moreover, Manufacturing and Industry 4.0 (14.7%, n = 11) includes SRAs for digital manufacturing, digital twin-based predictive maintenance, and logistics. Following this, Security and Privacy (14.7%, n = 11) focuses on security of cloud systems, IoT privacy, and cybersecurity applications. Furthermore, the Mobility and Robotics category (8%, n = 6) includes SRAs for autonomous and unmanned vehicles and for social head-gaze generation in social robotics. Finally, research in Emerging and Niche Domains (14.7%, n = 11) addresses architectures for virtual human integration in the metaverse, blockchain, and culture-aware mobile systems. This diverse domain coverage demonstrates how SRAs have become critical tools for structuring complex systems across virtually all sectors of the modern digital economy. Figure 2 shows the distribution of SRA studies by application domain.

A significant proportion of the studies (49.3%, n = 37) were conducted through multi-institutional collaborations, of which international collaborations accounted for (n = 20). Furthermore, (n = 13) of the studies featured industry-academia collaboration, partnering academic research with industrial practice to ensure relevance and applicability of architecture. Geographically, European institutions lead the research, contributing to (n = 37) of the studies, with strong representation from Germany, the Netherlands, and Spain. Seconded by the Asia-Pacific region (22.7%, n = 17), led by Australia, followed by America and Brazil (18.7%, n = 14). The data suggests that academics largely drive SRA design compared to industries.

The design of SRA is informed by four primary knowledge sources, each contributing distinct perspectives and employing specialized methods for knowledge acquisition. The results suggest a combination of sources that have been used. Research shows that studies have utilized multiple sources for knowledge acquisition.

Stakeholder Input (88%, n = 66) is the most prevalent method for requirements gathering using semi-structured interviews and structured workshops with architects, developers, and domain experts [27], often supplemented by surveys to validate findings from larger groups [28]. Domain Analysis (65.3%, n = 49) provides the methodological backbone for systematic design, particularly in product line engineering contexts, by creating structured frameworks and ontology development to establish a precise domain vocabulary [29]. Formal Standards (46.7%, n = 35) serve as blueprints that assist in designing SRA's through a set of guidelines often tested and verified, such as the ISO/IEC/IEEE 42010 standard for architecture description, which structures documentation and conforms to domain-specific standards [30]. Existing Architectures (82.7%, n = 62) provide evidence of working models from prior implementations, primarily through systematic studies to learn current designs or patterns, case study analysis, reverse-engineering [31], and understanding evolutionary trends [32]. Table 3 describes four common sources of knowledge identified in this research, together with the methods employed and examples.

SRA's are designed using a range of well-established architectural approaches, each contributing unique strengths to the resulting blueprint. Studies have used multiple design approaches to achieve a holistic product. Pattern-Based Design (77.3%, n = 58) applies recurrent solutions to recurring architectural challenges, drawing on both high-level and fine-grained design patterns. This includes Layered architecture [33, 34], Microservices [30, 35], pipes-and-filters [36], and MAPE-K. Complementing this, Model-Driven Engineering (MDE) (69.3%, n = 52) relies on abstraction through formal or semi-formal models to manage architectural complexity. Modeling languages such as UML [37, 38], SysML [32], and emerging ArchiMate [39–41] capture structural and behavioral details. Component-Based Design (66.7%, n = 50) is another approach, which decomposes systems into cohesive, loosely coupled components with well-defined interfaces. This technique enhances critical quality attributes such as reusability, maintainability, and replaceability. It is applied to multi-agent microgrid management [42], decentralized warehouse control [43], and self-adaptive middleware for wireless sensor networks [44].

Managing architectural complexity is further supported by the Viewpoint-Based Approach (65.3%, n = 49), which structures architectural descriptions into coherent views aligned with stakeholders. Guided by ISO/IEC/IEEE 42010 frameworks such as the 4 + 1 View Model, Views and Beyond, and the RM-ODP reference model [36, 45–49]. Methodology-Driven Design (54.7%, n = 41), in which Design Science Research (DSR) methodology is prevalent, develops artifacts through iterative cycles of problem identification, construction, demonstration, and evaluation [27, 50, 51]. Dedicated SRA methodologies, such as ProSA-RA [52, 53], and empirically grounded multi-phase methods reinforce rigor, traceability, and repeatability [54–56].

Domain-Driven Design (DDD) (36%, n = 27) strengthens SRA development by embedding domain semantics, as supported by Feature-Oriented Domain Analysis (FODA) to model commonalities and variabilities in domains such as smart farming [47] and digital-twin predictive maintenance [28]. Finally, Standards-Based Design (34.7%, n = 26) is critical in regulated environments, incorporating international standards and technical specifications, including IEC 62264, the Industrial Internet Reference Architecture (IIRA), and NIST cybersecurity guidelines to ensure compliance with industry's best practices [57–59]. Figure 3 shows the different approaches used for designing SRAs.

SRA design is substantiated by three main categories of supporting frameworks and methods that provide structured guidance for implementation. They are Architectural frameworks and standards (46.7%, n = 35) [36, 39, 58–62], Development methodologies (54.7%, n = 41) [27, 50–52, 54–57, 63], and Domain-specific standards and reference models (34.7%, n = 26) [42, 57, 59, 64]. Moreover, several modeling notations were employed to articulate architectural structures, behaviors, and domain concepts. Table 4 summarizes the various modeling notations used in the design of SRA, as well as the specific categories and their frequencies within each category.

Multiple evaluation methods were identified for assessing SRA's, with many studies employing a multi-method approach to strengthen validation and demonstrate comprehensive evaluation. Case Study evaluation was the most prevalent approach (70.7%, n = 53), typically involving instantiating the SRA into a prototype or full system within a specific domain context to demonstrate feasibility and utility [38, 42]. Expert review (30.7%, n = 23) was conducted through structured workshops, interviews, or surveys to identify design flaws and risks, such as using the Architecture Tradeoff Analysis Method (ATAM) workshops and the Framework for Evaluation of Reference Architectures (FERA) [52, 65]. Prototype implementation (41.3%, n = 31) served as concrete technical validation. Comparative analysis (16%, n = 12) involved systematic comparison against existing alternatives [66], while simulation (5.3%, n = 4) was reserved for domains where real deployment was impractical, such as in unmanned forest management systems [67].

The rigorous evaluation of an SRA necessitates structured methodologies to assess its quality, utility, and robustness. Beyond selecting evaluation types such as case studies or expert reviews, researchers frequently employ established architectural frameworks to guide comprehensive analysis. Namely,

Architecture Tradeoff Analysis Method (ATAM) (10.7%, n = 8) was utilized to identify risks and tradeoffs among quality attributes like modifiability and security [38, 65, 68, 69], Software Architecture Analysis Method (SAAM) (9.3%, n = 7) was applied to assess modifiability and extensibility through change impact analysis [30, 70], Framework for Evaluation of Reference Architectures (FERA) (6.7%, n = 5) provided a dedicated checklist-based approach for evaluating documentation completeness and clarity [44, 52, 57].

Beyond these specialized frameworks, broader evaluation paradigms were also employed, including Design Science Research (DSR) principles that assessed feasibility and usefulness through case studies and surveys, and Standardized Questionnaires such as System Usability Scale (SUS), Usability Evaluation Questionnaire (UEQ), and Technology Adoption Model (TAM) that provided quantitative measures of usability and acceptance [27, 29, 41, 51]. Figure 4 illustrates the frequency of different evaluation methods.

Development and Maintenance Qualities emerge as the most extensively evaluated category, with reusability being the most frequently assessed attribute (74.7%, n = 56), typically measured through code reuse ratios and development effort metrics [45, 71]. Extensibility is another prominent concern (60%, n = 45), evaluated through component integration ease [38], while maintainability (53.3%, n = 40) and modifiability (52%, n = 39) are analyzed via code metrics and scenario-based methods [30]. Operational Qualities constitute the second major category, with performance evaluation appearing in (n = 54) studies through throughput and latency metrics, scalability (n = 48) via load testing, and resource efficiency in (n = 35), focusing on energy and computational consumption.

Functional and Business Qualities demonstrate substantial presence, with usability evaluation (49.3%, n = 37) using standardized scores, functional correctness (44%, n = 33) through requirement coverage, and completeness in (36%, n = 27) against domain needs [27, 39, 57]. Cross-Cutting Concerns show significant variation, with security appearing most frequently (56%, n = 42) through threat coverage analysis, interoperability (45.3%, n = 34), and adaptability (41.3%, n = 31) through change response evaluation [36, 62, 72]. Table 5 of shows the detailed categories, quality, and attributes.

SRA research has expanded rapidly over the past decade, shifting from predominantly conceptual proposals to empirically grounded, domain-oriented studies, with a marked surge in 2018–2025 and peak activity in 2024–2025. Publication venues indicate the field's maturation; most contributions appear in peer-reviewed journals, while conferences remain important for early dissemination. Domain distribution mirrors global digitalization trends with Cloud/IoT/CPS, Data and AI, and Industry 4.0 accounting for the largest share, followed by healthcare and well-being, security and privacy, mobility and robotics, and several other niche domains. Collaboration patterns show a highly networked community; nearly half of all papers are multi-institutional, and a substantial proportion are international. Industry-academia collaboration is comparatively limited, offering an opportunity to narrow the research-practice gap.

SRA design is consistently multi-source. Across studies, four main sources of architectural knowledge are identified: stakeholder input, followed by analysis of existing architectures, domain analysis, and formal standards, and most designs combine more than one. The selection method is strongly domain-dependent. This underlines that knowledge sourcing reflects regulatory intensity, system criticality, and technological maturity rather than a single best practice. SRA design relies on a combination of complementary approaches rather than a single method. Pattern-based, model-driven, component-based, and viewpoint-based techniques form the core design foundation, supported by formal methodologies such as design science research and domain-specific or standard-driven practices. Designers increasingly use multi-notation modeling that combines UML, SysML, ArchiMate, feature models, and informal diagrams to capture different architectural perspectives. A wide range of artifacts, from reference models and meta-models to component libraries, workflow diagrams, and prototypes, demonstrates that SRA design is both systematic and practice-oriented, integrating conceptual rigor with real-world applicability.

Evaluation practice is characterized by triangulation. Mostly, studies combined case studies with prototype implementation, expert review, comparative analyses, and controlled experiments, while simulations are the least common but have gained demand in recent years. Established frameworks (ATAM, SAAM, FERA) provide structure, while controlled user/field experiments remain comparatively rare. Measured qualities include development and operation, reusability, maintainability, modifiability, and performance and scalability occur most frequently, but there is no universal metric set. Measurement choices are consistently tailored to domain objectives and constraints.

The implications outlined below are intended to guide researchers working to advance the theoretical and methodological foundations of SRA design and evaluation.

1. Increased collaboration between industry and academia will enable SRAs to address real operational constraints and deployment environments. Such partnerships will also enhance external validity and facilitate practical adoption.

2. The widespread use of specialized frameworks such as ProSA-RA, FERA, etc., suggests that SRA development benefits from formalized methodologies. Researchers and practitioners should adopt these structured processes to enhance the reproducibility and quality of the resulting architecture.

3. Most studies use multiple modeling notations, including UML, SysML, ArchiMate, and domain-specific diagrams, suggesting designers embrace a multi-notation toolkit to capture richer architectural communication.

4. Develop standardized evaluation protocols and metric baselines. Current evaluations are diverse and hard to compare. Establishing shared metric sets, benchmark tasks, and reusable evaluation frameworks would improve consistency and reproducibility across studies.

5. Tool support is critical for operational adoption. Automated model checking, configuration assistance, and code generation pipelines can make SRAs more maintainable and easier to instantiate consistently.

6. Build shared repositories of architectural models and reusable assets. Open access component libraries, model templates, and instantiation examples would accelerate reuse and help establish baselines for comparison and extension.

7. Investigate SRA challenges in fast-evolving digital ecosystems. Domains involving AI-driven autonomy, distributed edge computing, and digital twins introduce dynamic architectural demands. Research is needed to understand how SRAs can evolve and adapt in these environments.

This study may be subject to a few threats to validity. This study primarily relied on the Scopus database for its comprehensive coverage of peer-reviewed software engineering research and its robust indexing of major publishers. However, studies indexed exclusively in other digital libraries (e.g., IEEE Xplore, ACM Digital Library, Web of Science) may not have been captured. The process of determining study eligibility required interpretation of titles, abstracts, and, in some cases, full texts. Judgments were particularly necessary when studies discussed architectural frameworks without explicitly labeling them as SRAs, and extracting design and evaluation characteristics required interpreting narrative descriptions and architectural documentation. Variation in terminology, differing levels of reporting detail, and heterogeneous documentation formats posed challenges for consistent extraction.

This systematic review synthesized findings from n = 75 primary studies published between 2010 and 2025, offering a comprehensive overview of how SRA is designed and evaluated. The results show that SRA design depends on synthesizing multiple sources of knowledge, including stakeholder needs, existing systems, domain analysis, and formal standards applied in proportions that reflect each domain's requirements. A core set of design approaches emerged as consistently dominant, such as pattern-based, model-driven engineering, component and modular design, and viewpoint-based modeling. The review also indicates that evaluation practices are becoming increasingly structured and evidence-based. While case studies and prototype implementations remain foundational, expert review and selective quantitative assessment are increasingly incorporated to strengthen evaluation. As software ecosystems continue to expand and integrate emerging technologies such as AI, digital twins, and distributed autonomy, the role of SRAs as reusable architectural blueprints will become even more critical. The findings of this systematic review highlight several promising directions for advancing research and practice in SRAs. Advancing methodological rigor, practical relevance, and tool maturity will therefore be essential to enhancing the long-term effectiveness and adoption of SRAs.

All authors’ contributed equally to this article.