Expanding the pioneer shipping route for vaccine distribution: a reinforcement learning-based approach Open Access

Vaccines, as cold-chain products, are among the most successful and cost-effective health interventions, particularly in low- and middle-income countries (LMICs) (Anupindi et al., 2022). Their characteristics—such as packaging size, storage temperature, and shelf life—affect logistical handling throughout the supply chain, including storage, transportation, distribution, and administration. For instance, the Pfizer mRNA vaccine requires ultra-cold storage (around −70°C). It has a nonstandard dose size of 0.3 mL, along with a strict 21-day dosing interval, which necessitates a highly coordinated logistics system. In many LMICs—where infrastructure may be limited, transportation networks are fragmented, and healthcare access varies widely—vaccine design must consider the realities of long-distance distribution, unreliable power supplies, and delivery to remote areas to ensure fair and adequate coverage. Additional logistical challenges stem from geographic fragmentation and underdeveloped infrastructure (Berkley et al., 2013). Ensuring vaccine efficacy requires a cold-chain system that maintains a temperature range of 2–8°C (GAVI, 2022). However, distributing vaccines across remote, multi-island regions faces challenges such as unreliable electricity, long transit durations, and limited cold storage facilities, all of which threaten temperature control (Jedermann et al., 2017).

In archipelagic countries, sea transport is critical for ensuring connectivity and offers greater efficiency than air transport, particularly for reaching dispersed islands (Hernández Luis, 2002). Remote islands in LMIC often face significant transport limitations: they lack fixed transportation services, are not commercially viable, have low population densities and incomes, and are frequently inaccessible by air owing to the absence of airport infrastructure. This highlights the crucial role of sea transport, particularly pioneer shipping, in bridging the connectivity gap. Pioneer shipping is a government-subsidised maritime service to connect remote areas that are not served by commercial operators (Riadi and Kurniawan, 2024). The national shipping company is often mandated to operate these routes with financial support from the government. The primary objective is to facilitate the movement of goods and promote economic and social development in remote regions (Humang et al., 2019). Given these characteristics, pioneer shipping has the potential to support vaccine distribution to remote island areas that are accessible only by sea transport. However, this potential has not been explored in existing studies, although it enables the delivery of small-volume shipments—typical of vaccine demands in regions with low population densities—more reliably and cost effectively.

Despite its notable advantages, vaccine distribution using the pioneer shipping route is challenged by the risks associated with sea transport, such as delays, weather, sea conditions, and handling errors (Martínez de Osés and Castells, 2008). This study investigated the possibility of utilising pioneer shipping for delivering vaccines (and other cold-chain-based products), which has remained unexplored in prior research. The study was conducted within the framework of the routing optimisation problem, specifically the vessel routing problem, which primarily focuses on determining cost-effective routes and schedules for vessels delivering goods to multiple destinations (Brouer et al., 2014; Christiansen et al., 2013). The vessel routing problem is more complex than land-based problems, as it requires the consideration of sea conditions, port accessibility, fuel constraints, and time windows in addition to meeting the requirements of cost efficiency and service reliability (Wang and Meng, 2012). To avoid confusion with the acronym VRP, which is the commonly used abbreviation for vehicle routing problem, we refer to our problem as the pioneer vessel routing problem (PVRP). Despite advances in the research on vessel routing problems, existing models rarely incorporate cold-chain-specific constraints, such as perishability and urgent delivery, which are crucial for vaccine distribution. These have been considered in this study.

Building on the above background, this study contributes to the literature by addressing the underexplored challenge of vaccine distribution in archipelagic and remote regions. This study explicitly optimises the utilisation of pioneering shipping routes by proposing a novel reinforcement learning (RL)–based vessel routing framework that optimises delivery by considering perishability of goods and wave-induced variations. The modelling framework leverages clustering and attention mechanisms to prioritise the delivery of vaccines across multiple shipping routes, enhancing both efficiency and vaccine preservation. The model was tested on actual pioneer shipping routes in East Nusa Tenggara to assess its applicability. The results of the numerical experiments offer valuable insights for policy formulation and managerial decision-making in the context of pioneer shipping operations. The remainder of this paper is structured as follows: Section 2 reviews relevant literature in vaccine logistics and maritime routing. Section 3 presents the modelling framework and formulation and explains the RL solution approach. Section 4 discusses the results of the numerical experiments. Section 5 shares policy and managerial insights, and Section 6 concludes with implications and directions for future research.

This section highlights past vaccine distribution studies and underscores the need to integrate the vessel routing problem and its solution algorithms to ensure efficient vaccine delivery to remote islands. Over the past decades, extensive research has been conducted to explore the challenges associated with vaccine distribution and logistics (Cheyne, 1989; Xu et al., 2021a). The risks, uncertainties, and potential disruptions associated with the vaccine supply chain contribute to its fragility and complexity (Duijzer et al., 2018; Lin et al., 2020). Recent studies on vaccine distribution have highlighted issues such as cold-chain maintenance (Saif and Elhedhli, 2016), demand uncertainty (Shittu et al., 2016), and infrastructural limitations (Al Theeb et al., 2024) as challenges facing vaccine distribution. Notably, vaccine supply chains differ from traditional ones because of stringent temperature requirements and the need for urgent delivery (Al Theeb et al., 2024; Xiao et al., 2025). Effective distribution during the final stage of the vaccine supply chain is crucial for achieving large-scale immunisation coverage. In this regard, implementation studies, such as that by Pavoncello et al. (2025), emphasise the need for vaccine implementation frameworks to adapt to local contexts, particularly in low-resource and remote settings.

The complexity of deploying vaccines to vast and varied geographies has led researchers to explore a wide array of logistics and optimisation models (Kibanga et al., 2025; Xiao et al., 2025). However, most models have assumed a robust infrastructure, overlooking the heterogeneity of accessibility across regions and often neglecting the necessity of vaccine equity across these regions. Xu et al. (2021b) emphasised the importance of conducting a comprehensive analysis of optimal vaccine allocation strategies by considering various equity and efficiency objectives. Despite the progress, distributing vaccines to remote island regions remains a significant hurdle owing to the inherent logistical complexities of maritime transportation (Cano-Marin et al., 2023) and cold-chain distribution (Xiao et al., 2025). Past studies have often neglected the specific needs of geographically isolated areas, particularly remote islands in archipelagic settings. Furthermore, the maritime logistics for vaccine distribution has often been addressed as an afterthought rather than an integral part of the planning process.

VRP has been extensively studied in the context of logistics. Many variants of the classical VRP, including time windows, capacity constraints, and multi-echelon structures, have been applied to cold-chain supply chain and vaccine delivery problems. However, most of these cold-chain VRP problems involve land transportation and its associated uncertainties, such as traffic conditions, particularly in urban areas. For example, Al Theeb et al. (2024) presented a two-echelon VRP for developing countries, considering two-echelon problems involving mobile satellites serviced by large vehicles and small vehicles that pick up and deliver the vaccine to the final customers. The environmental aspect of routing problems has also gained traction. Guo et al. (2018) developed and solved VRP problems for cold-chain distribution by considering multiple factors such as traffic status, total delivery cost, product condition, and carbon emissions. Song et al. (2020) modelled a standard VRP that incorporated different time windows, vehicle types, and varying levels of energy consumption as the key parameters. More recently, the incorporation of drones and hybrid delivery (utilising vehicles and drones) has been considered (Gunaratne et al., 2022). Although previous research has yielded positive results and has contributed to improving vaccine delivery to remote areas, vaccine delivery applications have been generally limited to land-based or short-range logistics (Petroianu et al., 2021). Prior research has seldom addressed the challenges associated with inter-island transportation, particularly those related to small-volume shipments predominantly managed by pioneering shipping networks.

Despite advances in routing research, the specific problem of vessel routing for vaccine delivery remains underexplored. Maritime routing enhances the routing problem complexity because of sea conditions (Timas and Mohammadi, 2025) and irregular ship schedules (Seimetz Chagas et al., 2023). In a recent study (Sharif et al., 2024), the state of the art in maritime transport optimisation was reviewed, emphasising the importance of coordinated scheduling and routing of vessels to improve safety. Some researchers have addressed the complexities of ship-based delivery in archipelagic nations (Accorsi et al., 2021)and recommended customised routing algorithms with strict time windows under uncertainty (Fernández Cuesta et al., 2017; Seimetz Chagas et al., 2023; Timas and Mohammadi, 2025). However, these studies have shown a marked lack of integration between health logistics and maritime transport, especially in the case of remote islands with small-volume shipments. Therefore, this study aims to fill this gap in the literature by introducing the challenges facing vaccine distribution to remote islands within the framework of pioneer shipping.

The increasing focus on health equity demands that health logistics models accomplish more than cost minimisation and improvement in equality. Addressing the distribution of vaccines to remote islands and marginalised populations calls for a multi-criteria decision-making framework (Al Theeb et al., 2024). Owing to the combinatorial nature and dynamic constraints of the VRP, traditional optimisation methods can become computationally expensive or impractical for real-time applications. RL has recently emerged as a promising alternative for solving complex routing problems, enabling agents to learn optimal routing strategies through interactions with the environment (Phiboonbanakit et al., 2021). By exploring broader operational research models, this study proposes a variant of the RL model to solve the PVRP by incorporating not only the clustering process but also vaccine considerations into the attention mechanism.

This study aimed to model the utilisation of vessels for distributing vaccines, which is a cold-chain-based product. Vaccines are time- and temperature-sensitive; they may be damaged if not stored properly, thus necessitating fast transportation. Vessels used in pioneer shipping carry passengers and other goods in addition to vaccines. It is thus important to balance the shipping performance between the distribution of vaccines, transport of passengers, and transport of other goods. Therefore, the model was constructed within the framework of an optimisation problem for minimising not only the number of damaged vaccines but also the monetary cost of unserved passengers and other goods. In addition, we incorporate the multiplication of the number of vaccines and time required to reach the destination port (see Equation (1)), which emphasises that faster delivery of vaccines is preferred. Therefore, this objective function practically aims to minimise the time to ship the vaccine to the destination port while concurrently reducing the unserved demand of passengers and other goods as well as the number of damaged vaccines. Equation (1) also accounts for wave heights that affect the shipment duration, which could damage the vaccines.

• $α1,α2,α3$⁠: monetary values for an unserved passenger, other goods, and damaged vaccines, respectively.

• $X1,X2,X3$⁠: total number of unserved passengers, other goods, and damaged vaccines, respectively.

• $λd$⁠: number of vaccines shipped to port d.

• $td(ς)$⁠: time required for shipping vaccines to port d, which is influenced by the wave height $(ς)$

This study also considered the utilisation of several vessels that can serve different routes simultaneously. The routes are first grouped within a cluster, such that the vessel serves only the ports within that cluster. To improve connectivity, we also assign the “hub” to connect seaports among different clusters. We designate $H$ as a set of hubs, namely $H={1,2,…,h,…,H}$⁠, $D$ as a set of destination ports, where $D={1,2,…,d,…,D}$⁠, and $K$ as a set of vessels, namely $K={1,2,…,k,…,K}$⁠, where each vessel services a specific cluster route r. The destination ports are then clustered into the route r, namely $Dr={1,2,…,i,…,Dr}$⁠. Let us assume that $Pijkr$ is a binary variable, which has a value of 1 if a vessel k sails from port i to j on a route r (i.e. $∀i,j∈{H,Dr}$⁠) and 0 otherwise.

Constraints (2) and (3) construct the shipping system between the hub and destination port. We assume $τhr$ is a binary variable with a value of 1 if a vessel moves from the hub $h$ to the destination port within the cluster route r, and value 0 otherwise. Constraints (2) and (3) state that if node $i∈Dr$ is selected as the destination port, then vessel k from hub h must visit $i$⁠; if it is not selected as the destination port, no vessel from any of the hubs may visit it.

where ψ is a large number.

Constraint (4) shows that each port is visited exactly once by a single vessel and allocated to a specific route cluster. Equation (5) ensures that each vessel that visits a port also departs from that port. Constraint (6) ensures that no destination port in one cluster may be visited by a vessel assigned to a different cluster, and Constraint (7) ensures that no vessel is allowed to travel from one seaport directly back to the same seaport.

Constraint (8) guarantees that the number of passengers (g = 1), goods (g = 2), and vaccines (g = 3) transported from all destination ports, for all routes to the destination port j, must equal the total demand at port j $(Sjg$⁠), and is tightened with Constraints (9) to (11). Please note $yijkg$ denotes the number of passengers or goods or vaccines shipped by vessel k from port i to port j.

Constraint (12) states that the difference in the load carried by a vessel immediately before and after visiting a seaport must equal the demand at the seaport. Constraint (13) ensures that the load carried by a vessel is less than or equal to the capacity (⁠$uk$⁠) of the vessel. These constraints are tightened by Constraint (14), which guarantees that the load carried by a vessel immediately before visiting a seaport does not exceed the capacity of the vessel minus the demand at the last visited seaport; Constraint (15) which ensures that the load carried by a vessel immediately before visiting a port is at least equal to the demand; and Constraint (16), which states that no cargo is allowed to be transported to and from the same seaport on a single route.

The intermediate variable $tj$ represents a time stamp. In Constraint (17), the arrival time at a port is the time following the voyage time between the origin and destination ports and the loading/unloading activity duration at port j (⁠$Zj$⁠). Please note that $eij$ represents the distance between ports i and j, and $zk(ς)$ denotes the velocity of vessel k, which is influenced by the wave height $(ς)$⁠. Equation (18) imposes the condition that the time of the vaccines when it arrives at the destination seaport j should be less than the maximum voyage duration of the vaccines (T). Equations (19) and (20) indicate the values of the timestamps at the hub and destination ports. Equation (21) refers to the non-negativity characteristic of the decision variables, and Constraints (22) and (23) indicate the binary decision variables.

The optimisation problem is solved by proposing a variant of RL, specifically by integrating the attention mechanism concept (Veličkovi'c et al., 2018). The attention mechanism facilitates the provisioning of the context values (i.e. attention values) for each destination to be visited. These values can be subsequently utilised to prioritise the port order within the routing problem. The context values are determined by multiplying the attention features by the weight matrix. We specifically used the number of passengers, amount of goods, weight of vaccines, and average shipping time of the seaport as the features (i.e. feat), as shown in Figure 1. The average shipping time considers the vessel speed, which is affected by the wave height. The incorporation of this shipping time in the attention feature indicates that the decision relating to the vessel routing considers the essential factor of shipping, namely the wave height. To make each feature comparable, we normalised the values in the range of 0–1 using a sigmoid function.

We introduce a share matrix (W) to calculate the importance of node i to node j (⁠$importij)$ by multiplying feat by its share matrix value W and the adjacency matrix (see Figure 2 (c) – (f)). A larger value of $importij$ represents a greater importance of node i to node j. Hence, it attracts greater attention in terms of number of visits. The attention coefficients (⁠$∅ij$⁠) aim to normalise the importance value of node i to node j with their sum (see Figure 2 (h)). We then invoke the leaky ReLu-based function shown in Equation (24).

where $j′$ is a neighbour of port i (i.e. $Ni′$⁠). The self-attention value of each port i (⁠$li$⁠) is then calculated using Equation (25) and is further used to determine the priority list of customers to be visited (Algorithm 1). The RL agent is employed to seek the value of the share matrix that affects the self-attention mechanism.

In addition to vessel routing, we trained the RL to cluster the seaports in each route. Pioneer shipping facilitates the simultaneous operation of several routes, with each route serviced by a vessel. We then trained the RL agent to determine the cluster of ports for each route to be serviced. The clustering procedure operates under the premise of port closeness, whereby a closer port possesses a greater likelihood of being assigned within a route. By using an approach similar to routing, the agent is trained to decide the weight of port closeness. We define the closeness of a port as the difference between the minimum and maximum distances to reach another seaport. A low difference indicates closeness to the other port. The agent weighs this closeness to cluster the seaports evenly along each route (see Figure 3).

We employed the deep deterministic policy gradient (DDPG) algorithm to determine the shared matrix for routing and weight of port closeness (see Figure 3(c)). The DDPG agent was trained to maximise the cumulative reward, which is the sum of the rewards obtained until the end of the episode (M) (Equation (26)). Note that $γm$ denotes the discounted rewards at episode m. Discounted rewards imply that immediate rewards are more valuable than later rewards. Please note that $rv+m$ denotes the reward obtained at step v of episode m.

In the context of the vessel routing problem, the reward relates to the objective function (1). However, owing to the inherent characteristic of the RL agent, which seeks to maximise the rewards, we reformulate the negative of the objective function as a reward to be maximised, as shown in Equation (27).

The action a (⁠$a∈A)$ taken by the agent is related to the value of the share matrix for deciding the routing (see Figure 2) and the weights of closeness for deciding the cluster route (see Figure 3). The actions subsequently affect the environmental conditions, which pertain to the shipping process, including the arrival time at the port, passenger count, cargo tonnage, quantity of vaccinations shipped, and percentage of vaccine damage. The agent will establish a particular strategy known as a policy (⁠$π$⁠) to achieve a greater return. A policy $π(s,a)$ establishes a probability distribution for each action (⁠$a∈A)$ and state $(s∈S)$ or maps the probability of an action to a specific state. Let us denote the state at step v as $sv$⁠. Assume that the agent selects an action based on the policy and obtains an expected return from a given state, as defined in Equation (28).

The DDPG approach uses an actor–critic RL agent that employs neural networks to seek an optimal policy that maximises the expected return. The actor and critic networks are randomly initialised using $θμ$ and $θQ′$⁠, respectively. During each episode, the actor associates the state $sv$ with the action $av$ at each step and subsequently transmits it to the environment. This state is also utilised for output estimation and loss value calculation. The environment provides the agent with the next state $sv+1$ and the estimated reward $rv$ as feedback. This feedback is used to update the actor and critic networks. The replay memory holds a collection of states, actions, rewards, and the next state. Once the data are gathered, a small set of experiences is randomly selected from the replay memory. The target Q-value for each state–action pair in this set is then calculated using Equation (29). This target Q-value is also utilised to update the critic network by minimising the loss, as described by Equation (30). The Q-value represents the mapping of the state–action pair with its corresponding return value.

During optimisation, the actor and critic networks aim to maximise the expected return by iteratively updating their parameters. The next essential step in optimising the actor network, particularly the policy function, involves calculating the gradient of the expected Q-value with respect to the policy parameters $θμ$⁠. This computation improves the expected cumulative rewards. The general procedure for applying the DDPG is outlined in Algorithm 1 (see Appendix).

The proposed model was tested by applying it to an actual pioneer shipping network in East Nusa Tenggara, which comprises 10 destination ports (see Figure 4). The distances between the seaports were calculated using the ship distance calculator (https://www.bednblue.com/sailing-distance-calculator), summarised in Table 1.

Considering the safety of the vaccine product as the highest priority, we posit that the penalty for damage to the vaccine product should be the highest. Furthermore, the penalty for undelivered cargo, which often consists of primary products, is larger than that for the passenger count. We set the penalty value (i.e. $αg)$ as follows: $α1$ = 50, $α2$ = 75, $α3$ = 100. Owing to insufficient data, we used assumptions regarding the demand for passengers, goods, and vaccine products. Because of the limited need for pioneer shipping, we assumed that the demand will adhere to a uniform distribution, characterised by specific minimum and maximum values. Although this assumption may not be universally applicable to pioneer shipping, it can be regarded as a component of the problem solved by RL agents. Considering the characteristics of RL that are appropriate for complex and dynamic problems, this assumption should be addressed in future studies to train the agent on a broader range of problems. The demand matrices for passengers, goods, and vaccination products are presented in Tables 2–4, which were created randomly under the premise of a uniform distribution. We established the minimum values as 2, 1, and 0.2 for passengers, commodities, and vaccinations, respectively, and the maximum values as 50, 2, and 1 for the same categories. Each route is assumed to be serviced by a single vessel that can carry up to 500 passengers and 50 tonnes of freight. Additionally, we assume the vessel can sail at a maximum speed of 15 knots.

We also conducted a hyperparameter tuning process based on grid search, in which we tuned the learning rates for actors and critics in the range of 0.001–0.1 and batch size in the range of 64–1024. By using the grid search process, the hyperparameter tuning required up to 125 trials. Considering the hyperparameters used in the existing literature (Osa et al., 2019), the hyperparameters used in DDPG are summarised in Table 5. It is also interesting to note that the DDPG incorporates the neural network architecture that influences the RL performance. In the case of the actor network, we set three hidden layers [256, 256, $|A|$], with ReLU activations for the first and second layers and a hyperbolic tangent for the last hidden layer. The last hidden layer of actor networks then provides the action to be decided by the agent (A). For the critic network, we set hidden layers [256, 512], each with ReLU activations.

Pambudi et al. (2022) performed comprehensive reviews of the shelf life of vaccines, which varies based on the type of vaccine and equipment available. We can infer that in the case of a vessel equipped with a storage facility that maintains a temperature range of 2–8 °C, many vaccines will survive for an extended period of weeks. However, in the case of a power failure, the vaccine must be moved to a vaccine carrier (e.g. an ice pack), where it may survive only for 24 h (Department of Health and Family Welfare, 2016).

In this numerical example, we assumed that the vaccine can survive a maximum of 36 h of travel, and the cold storage on the vessel is available for only 12 h after travel. Table 6 summarises the results of the RL solution, obtained by varying the number of routes. When only one route is available, with each vessel visiting each port within a single loop, there is a possibility that up to 51% of the vaccine products may be damaged. The numerical results also imply that increasing the number of available routes can reduce the percentage of damage to the vaccine products. It is also interesting to note that in the case of the four available routes, where the average travel time exceeds 36 h, the damage percentage is zero, which implies that the RL procedures can provide an effective strategy to prioritise the transportation of vaccine products.

Table 7 shows the network performance with different numbers of routes. Because the vessel also carries passengers and other goods in addition to vaccine products, increasing the number of routes improves the service performance for passengers and freight owing to the decrement in shipping time. The shipping plan with four routes can increase the performance of passenger and freight travel hours by up to 35% when compared with that with just one route.

Although the increase in the route options improved vaccine delivery using the vessel, the risk linked to wave-height uncertainty remained, which can extend the voyage duration. We tested the proposed RL for vessel routing using three routes with 600 different sets of randomly generated wave variations. Specifically, we generated different wave heights for each pair of seaports by using a random number and inverse cumulative distribution functions (CDFs), as shown in Figure 5. The CDF uses the data collected annually around the seaports, which includes 2,812,542 significant waves from September 2023 to September 2024 (NASA, 2023). We evaluated several distributions for fitting the wave data, and the Burr distribution was found to provide the best fit based on the Akaike information criterion value. The randomly generated waves $(ς)$ were then used to adjust the vessel speed using Equation (31), as demonstrated by Lewis (1955).

Figure 6 illustrates the effect of wave uncertainty on the travel time. When the wave height is not considered, the average travel time among all seaports is approximately 55.57 h (for the four-route scenario, please see Table 6). The outcome is entirely different when considering wave uncertainty, where 50% of the average travel time among all seaports exceeds 57 h. This situation increases the probability of vaccine product damage to 19%. Specifically, it means that within 600 random wave events, 114 events cause damage to the vaccine product during the voyage. Furthermore, it is noteworthy that the temporal limitation of vaccine viability significantly influences the number of required routes. Table 8 indicates that under conditions of very limited time (i.e. less than 36 h), the four routes are insufficient to prevent vaccination damage. The uncertainty of trip time, combined with the available survival duration of the vaccine, constitutes a critical constraint for utilising pioneering transportation methods for vaccine distribution.

Remote islands pose a significant challenge in terms of vaccine distribution, as the mode of transportation is unreliable. Practically, the remote islands can be reached only through air and sea transport. Air transport can deliver the vaccine the fastest and thus minimise the risk of vaccine damage. However, it requires specific infrastructure and higher shipping costs. We interviewed the staff handling vaccine distribution in the archipelagic province of Indonesia. Although the delivery time of an aeroplane is significantly faster (i.e. 12 times faster than the direct port-to-port shipment), we found that air transport is approximately 10 times more costly than sea transport. In addition, sea transport provides alternatives for delivering the vaccine, that is, directly from port to port or by following an available shipping route (e.g. pioneer shipping route). Both shipping processes require a longer time than that required for air transport, necessitating a storage system to be installed on the vessel. The direct port-to-port route provides a shorter travel time than that obtained by following an available shipping route, but it requires additional vessels for direct shipping. On the contrary, the delivery via the available shipping route requires fewer vessels but involves a longer journey duration.

Considering the possibility of delivering the vaccine via a vessel, the following key points must be addressed to ensure a safe delivery process:

1. Prepare and plan the vaccine transport.

1. Delay consideration

1. Storage equipment

1. Temperature monitoring

Based on the above analysis of the mode characteristic, the schematic decision process for managing vaccine distribution in remote areas is depicted in Figure 7. We categorise vaccines into two main types: emergency and routine vaccines. Emergency vaccines must be delivered as soon as possible, as they are crucial for controlling situations such as a disease outbreak. In contrast, the vaccine delivery can be priorly scheduled for routine vaccines. Emergency vaccines must be delivered by air transport or direct port-to-port shipping processes, whereas routine vaccines can be delivered using the available shipping routes on vessels equipped with vaccine refrigeration equipment. If such equipment is unavailable, it is preferable to use port-to-port direct shipping procedures.

This paper discussed the utilisation of pioneer shipping for transporting cold-chain goods, such as vaccines. Pioneer shipping refers to the shipping service that ensures connectivity to remote islands. A variant of RL is proposed for deciding the shipping cluster of routes as well as the route for transporting the vaccine product using a vessel. Numerical testing was conducted using an actual pioneer shipping route. The numerical results indicated that a single loop (i.e. one route) can cause 51% damage to the vaccine products, whereas increasing the number of routes can help prevent damage during the voyage owing to the reduced shipping time. Additionally, increasing the routes may reduce the average travel time between all seaports by up to 46%, which benefits not only the vaccine product but also passengers and other goods being shipped.

From the numerical example, we found that wave uncertainty significantly increases the average travel time and disrupts the shipping of vaccine products. Therefore, owing to their time-sensitive nature, air transport is the most effective option for delivering emergency vaccines. If air transport is unavailable, the direct port-to-port route offers a shorter travel time than that when following an available shipping route. The pioneer shipping route is mostly suitable for routine or scheduled vaccines. However, it must be ensured that the vessel is equipped with vaccine refrigeration equipment. If such equipment is unavailable, it is preferable to use the direct port-to-port route.

Further research can focus on developing a method based on multimodal perspectives, as the end consumer of cold-chain products is usually located far from seaport locations. Additionally, extending the model to address stochastic optimisation would be intriguing, as it would enable a thorough examination of the uncertainty within the optimal framework. Furthermore, additional training of RL agents to address a broader spectrum of problems should be undertaken to ensure the applicability of RL-based methods.

The authors would like to acknowledge the financial support from the National Research and Innovation Agency (BRIN) under the Research and Innovation for Advanced Indonesia (RIIM) Batch IV program in collaboration with the Education Fund Management Agency (LPDP). We also express our gratitude to ITB for its support through the Research “Unggulan” Program.

The supplementary material for this article can be found online.