Deep reinforcement learning for portfolio allocation: a comparative study on five Vietnamese equities

Deep reinforcement learning (RL) has emerged as a powerful framework for portfolio allocation, enabling agents to learn adaptive investment policies directly from price data instead of relying on predefined heuristics or static optimization rules. By combining RL with deep neural networks, DRL agents approximate non-linear, high-dimensional mappings between market states and optimal portfolio weights, supporting robust decision-making under uncertainty (Jiang et al., 2017; Deng et al., 2016).

Several DRL algorithms have been widely applied in portfolio management, including advantage actor–critic (A2C), proximal policy optimization (PPO) (Schulman et al., 2017a), deep deterministic policy gradient (DDPG), twin delayed DDPG (TD3) (Fujimoto et al., 2018) and SAC (Haarnoja et al., 2018). These models differ by learning strategy (on-vs. off-policy), exploration mechanism and stability trade-off yet share the goal of maximizing long-term return while adapting to stochastic market dynamics. Recent extensions such as truncated quantile critics (TQC) and RecurrentPPO further improve stability and temporal awareness through distributional learning and recurrent architectures (Bühler et al., 2019; Schulman et al., 2017b).

Despite these strong theoretical foundations, applying DRL in financial markets remains challenging. Market data are inherently noisy, non-stationary and subject to regime shifts, which can lead to unstable convergence and sensitivity to hyperparameters (Velay et al., 2023). In addition, transaction costs, limited sample sizes and high cross-asset correlations may distort policy learning and reduce out-of-sample reliability. For this reason, comparing DRL agents with traditional benchmarks – such as equal-weight and minimum-variance portfolios – provides an essential reference for assessing whether adaptive methods genuinely outperform passive diversification strategies (Ledoit and Wolf, 2004).

Empirical studies suggest that while DRL models can capture short-term dependencies and non-linear interactions among asset returns, their performance is highly sensitive to market structure and liquidity conditions (Espiga-Fernández et al., 2024; Rezaei and Nezamabadi-Pour, 2025). These challenges are particularly pronounced in emerging and frontier markets, where thinner liquidity, stronger retail participation and frequent structural changes introduce additional instability. At the same time, such environments provide a valuable test bed for evaluating the robustness and generalization of modern DRL algorithms under realistic constraints.

Motivated by these considerations, this study addresses the need for a controlled and reproducible evaluation of DRL-based portfolio strategies in an emerging-market setting. Specifically, this paper conducts a unified comparison of seven DRL algorithms – A2C, PPO, DDPG, TD3, SAC, TQC and RecurrentPPO – applied to five Vietnamese equities over the period 2018–2025. By maintaining a consistent experimental pipeline, incorporating realistic transaction costs and performing multiple random-seed replications, the study aims to isolate algorithmic behavior from implementation-specific variations and provide transparent, reproducible evidence on model performance.

The main contributions of this study are threefold. First, this study develops a unified and fully controlled experimental framework for evaluating multiple DRL algorithms under identical market conditions, enabling a fair and transparent comparison across on-policy, off-policy, deterministic, stochastic and distributional approaches. Second, it provides empirical evidence on how key algorithmic design features – such as entropy regularization, clipped policy updates and recurrent state representations – affect return dynamics and risk-adjusted performance in an emerging and frontier market setting characterized by non-stationarity and limited liquidity. Third, the study ensures reproducibility and practical relevance by incorporating realistic transaction costs and conducting multiple independent training runs with different random seeds, thereby isolating algorithmic behavior from implementation-specific variations and offering robust and replicable insights into DRL-based portfolio performance. Compared with prior studies that typically focus on a single algorithm or heterogeneous experimental setups, this framework provides a more controlled and comparable evaluation environment, particularly suited for emerging-market conditions.

The selected equities – SBT.VN, BID.VN, CTG.VN, HPG.VN and VCB.VN – represent key sectors of the Vietnamese economy, including finance, manufacturing and consumer goods. These stocks are among the most actively traded and consistently listed constituents of the Ho Chi Minh Stock Exchange, offering relatively long and reliable data histories compared with other Vietnamese equities. This manageable yet diverse universe supports robust time-series modeling while preserving the complexity of a multi-asset environment. Moreover, differences in liquidity and market depth across these assets reflect real frontier-market conditions, making them suitable for testing the adaptability and stability of DRL-based portfolio strategies. The emphasis of this study is not on algorithmic novelty but on isolating the effect of design choices under a unified and controlled experimental pipeline.

The remainder of the paper is organized as follows: Section 2 reviews prior research on DRL in portfolio management, emphasizing risk sensitivity, stability and market adaptability. Section 3 describes the data, feature construction and benchmark portfolios. Section 4 presents the methodological framework, including the environment design, reward formulation and training–evaluation setup. Section 5 reports empirical results on convergence, comparative performance and robustness under different market conditions. Finally, Section 6 concludes and discusses implications for future research.

RL has gained traction in financial portfolio management for its ability to learn optimal allocation strategies through repeated interaction with dynamic markets (Sutton and Barto, 2018). DRL extends this framework by integrating neural networks, enabling agents to approximate complex value functions and policies in high-dimensional, non-linear environments (Jiang et al., 2017; Deng et al., 2016). Recent developments such as A2C, PPO (Schulman et al., 2017b), DDPG, TD3 (Fujimoto et al., 2018) and SAC (Haarnoja et al., 2018) have established DRL as a viable approach for continuous-action portfolio optimization.

Although these advances are significant, most agents remain prone to overfitting historical patterns and performing poorly under regime shifts or structural breaks (Velay et al., 2023). To enhance robustness, researchers have proposed hybrid or risk-sensitive variants that penalize volatility and drawdowns (Rezaei and Nezamabadi-Pour, 2025). Entropy-based methods such as SAC encourage exploration and produce smoother learning, while actorcritic families (A2C, PPO) balance bias and variance in policy updates. Off-policy models (DDPG, TD3) improve sample efficiency through experience replay but often become unstable when market noise dominates. These issues motivate comparative studies evaluating multiple algorithms under identical market conditions and transaction-cost structures.

Recent literature also introduces advanced architectures such as TQC and RecurrentPPO to address heavy-tailed rewards and partial observability (Bühler et al., 2019; Schulman et al., 2017b). TQC extends SAC by truncating extreme critic quantiles, improving value-estimate stability in volatile markets, while RecurrentPPO employs long short-term memory (LSTM) units to preserve temporal memory, which is useful when markets exhibit serial dependence or delayed reactions. Empirical results show both families deliver more consistent out-of-sample performance, making them promising for multi-asset trading environments with strong autocorrelation and heteroscedasticity.

Benchmarking and risk management remain essential for financial validation. Conventional baselines such as equal-weight (EW) and minimum-variance (MinVar) portfolios provide clear yardsticks for evaluating DRL agents. MinVar typically relies on covariance-matrix estimation; the Ledoit–Wolf shrinkage estimator (Ledoit and Wolf, 2004) is preferred for high-dimensional assets because it yields well-conditioned, positive-definite matrices even with limited data. Comparing DRL agents against EW and MinVar benchmarks helps determine whether adaptive methods truly add value beyond passive diversification.

Another related strand of research combines price forecasting models with classical portfolio optimization, such as LSTM-based return prediction followed by Markowitz mean–variance allocation (Hochreiter and Schmidhuber, 1997; Markowitz, 1952). Such forecast-then-optimize approaches are intuitive and useful, especially when the main objective is to improve expected-return estimation. However, they separate the prediction stage from the allocation stage, which may lead to error propagation when forecast errors are passed into the optimizer. In contrast, DRL learns allocation policies directly from market interaction and can incorporate rebalancing behavior, transaction costs and dynamic risk–return trade-offs within a single decision-making framework (Jiang et al., 2017). In this sense, DRL differs from classical mean–variance allocation and covariance-based benchmarks, including those using shrinkage covariance estimators (Ledoit and Wolf, 2004), because it learns portfolio decisions endogenously rather than solving a separate optimization problem from predicted returns. Therefore, the contribution of DRL is not only improved prediction accuracy but also the ability to learn adaptive portfolio decisions under changing market conditions.

Within emerging markets such as Vietnam, DRL applications are still limited but promising. Vietnamese equities feature moderate liquidity, sectoral concentration and heavy retail participation, producing frequent non-stationarity and volatility clustering. Studies on Vietnamese stocks indicate these characteristics create an ideal testbed for adaptive algorithms capable of responding to rapid structural changes (Saberironaghi et al., 2025). However, most prior work focuses on forecasting rather than allocation. A gap remains in systematically assessing multiple DRL algorithms under realistic constraints – transaction costs, frequent rebalancing and stochastic volatility – within reproducible setups. This study addresses that gap through a unified multi-agent backtest on five major Vietnamese equities (SBT.VN, BID.VN, CTG.VN, HPG.VN and VCB.VN) from 2018 to 2025, comparing seven DRL algorithms (A2C, PPO, DDPG, TD3, SAC, RecurrentPPO and TQC) against EW and MinVar benchmarks. By emphasizing reproducibility, consistency across ten random-seed runs and transparent evaluation metrics, this work contributes to methodological refinement in DRL portfolio research and empirical understanding of frontier-market dynamics.

Recent evidence further shows that Southeast Asian and Vietnamese equity markets remain structurally different from mature markets in terms of liquidity dynamics, investor composition and volatility behavior. The Association of Southeast Asian Nations (ASEAN) markets experienced pronounced liquidity changes around major shocks, while Vietnam's capital market continues to be characterized by strong retail participation and market-depth constraints (Thi Bich, 2025; World Bank, 2024). These structural features strengthen the case for adaptive DRL frameworks, especially recurrent architectures such as RecurrentPPO, which are designed to retain temporal information and respond to unstable market signals.

The empirical universe comprises five Vietnamese equities actively traded on the Ho Chi Minh Stock Exchange (HOSE): SBT.VN, BID.VN, CTG.VN, HPG.VN and VCB.VN. These assets represent diverse industry exposures including finance, manufacturing and consumer sectors.

The five stocks were selected based on three criteria: active trading and data availability over the full sample period, sectoral diversification across finance, manufacturing and consumer-related industries and representation of large and liquid Vietnamese equities commonly associated with the VN30 investable universe. This selection ensures that the experiment uses assets with sufficient market depth and reliable price histories while maintaining a controlled portfolio dimension for cross-algorithm comparison.

The five equities were selected not only for their liquidity but also for their continuous data availability, sectoral representation and suitability for a controlled comparative experiment. Specifically, the sample includes major financial, industrial and consumer-related firms, allowing the agents to learn from assets with different return dynamics while maintaining a balanced and reliable panel. This relatively compact universe also helps reduce noise from thinly traded or discontinuously listed stocks, which is particularly important in emerging-market DRL applications.

The analysis is conducted on a focused set of highly liquid Vietnamese equities to ensure data consistency and model stability. While this design enables controlled experimentation, future work should extend the framework to larger and more diverse asset universes. This focused design is intentional to maintain a controlled experimental setting; however, future work should extend the framework to larger and more diverse asset universes to validate generalizability.

Daily open, high, low, close, volume (OHLCV) data are obtained through a local-first process: the system loads pre-downloaded comma-separated values (CSV) or Parquet files when available and reverts to Yahoo Finance only when local data are missing. The full sample covers January 1, 2018, to September 30, 2025. The period from January 1, 2018 to January 1, 2023 is used for training, while January 1, 2023, to September 30, 2025, is reserved for out-of-sample evaluation. Trading calendars across assets are aligned to produce a balanced panel, with forward–backward filling applied for missing records due to market holidays.

For each asset, a consistent set of technical indicators is computed using the ta-library:

1. RSI(14): Relative strength index over a 14-day window, indicating overbought or oversold conditions;

2. MACD and MACD_Signal: Moving average convergence–divergence and its signal line for trend-reversal detection;

3. SMA(20) and EMA(20): Simple and exponential moving averages with a 20-day window to smooth short-term volatility;

4. Bollinger bands (window 20, 2σ): capture relative volatility and price extremes and

5. OBV: On-balance volume, representing directional trade-volume flow.

All indicators are forward- and backward-filled within each ticker to preserve temporal continuity. Dynamic covariance estimation via the Ledoit–Wolf shrinkage estimator (Ledoit and Wolf, 2004) is applied only to construct the MinVar benchmark and is excluded from the reinforcement learning state to isolate the contribution of technical features.

Two deterministic benchmarks are computed for comparison:

1. Equal-weight (EW): simple average of daily returns across all assets.

2. Minimum-variance (MinVar): daily rebalanced using the Ledoit–Wolf shrinkage covariance estimated from 252-day trailing returns underweight bounds (0, 0.1).

The feature set comprises eight technical indicators per asset: RSI(14), MACD, MACD_signal, SMA(20), EMA(20), Bollinger upper band, Bollinger lower band and OBV. The indicators are computed separately for each asset to avoid look-ahead bias and merged into a synchronized panel. Turbulence and covariance terms are excluded from the RL state, as the Ledoit–Wolf covariance estimator (Ledoit and Wolf, 2004) is applied only to the MinVar benchmark.

A custom gymnasium environment, StockPortfolioEnv, is developed to simulate multi-asset portfolio trading with transaction costs and daily rebalancing through continuous actions. The environment is fully differentiable and compatible with continuous-control DRL algorithms such as PPO, SAC, TD3 and RecurrentPPO.

The environment follows the same general logic as widely used financial DRL benchmarking frameworks such as FinRL, where trading environments, agents and backtesting modules are organized to support reproducible portfolio-learning experiments (Liu et al., 2020, 2021). However, the present environment is deliberately kept compact and problem-specific, focusing on five Vietnamese equities, continuous portfolio weight allocation, proportional transaction costs and a unified comparison of seven DRL agents under identical data and evaluation settings.

1. State space: The environment state at time t is represented as

where V_t denotes current portfolio value, $pt∈RN$ the vector of asset prices and $ht∈RN$ current asset holdings. Technical indicators (RSI, MACD, SMA, EMA, Bollinger bands and OBV) provide temporal and trend-related information for each asset. No turbulence or covariance features are included in this study to isolate the impact of technical indicators.

1. Action space: Each agent outputs a continuous vector a_t ∈ [−1, 1]^N, which is mapped to portfolio weights w_t via the softmax normalization:

This transformation guarantees w_i,t ≥ 0 and ∑_iw_i,t = 1, ensuring a long-only and fully invested portfolio. Therefore, the agent continuously adjusts allocation proportions across assets rather than issuing discrete buy/hold/sell commands.

1. Transaction costs: A fixed cost of 10 basis points (0.1%) is applied to each buy or sell operation, calculated on the traded notional value.

2. Portfolio dynamics: After executing an action at time t, the portfolio value evolves as

where Δv_i,t represents the net value traded in asset i and C_t the total transaction cost.

1. Reward: The immediate reward is defined as the normalized daily portfolio return:

which is implemented without additional penalty terms or risk adjustments to preserve comparability across agents. This design isolates return-driven learning and ensures that differences in performance are attributable to algorithmic characteristics rather than reward engineering.

The use of a return-based reward is an intentional design choice. By avoiding additional risk-adjustment terms in the reward function, the experiment preserves a controlled comparison across algorithms and isolates the effect of algorithmic design rather than reward engineering. However, this simplified reward specification may also encourage some agents to pursue higher-return but higher-volatility strategies, since drawdowns and volatility are evaluated ex post rather than penalized directly during training.

All agents thus operate under a continuous control policy, learning to rebalance allocation weights smoothly at each step while accounting for transaction costs and market feedback. To further interpret this setup, the use of continuous action spaces enables fine-grained portfolio adjustments and smoother rebalancing behavior, which are particularly important in financial markets with transaction costs and rapidly changing conditions. This design choice supports more realistic trading behavior compared to discrete allocation schemes, where abrupt changes in portfolio weights may lead to excessive turnover.

In addition to proportional transaction costs, real-world trading in emerging markets may also be affected by liquidity constraints, slippage, etc. These frictions are not explicitly modeled in the present environment, which it assumes execution at observed prices plus a fixed transaction cost. Therefore, the reported results can be interpreted as conservative evidence under simplified execution assumptions rather than as a full representation of real trading impact.

All algorithms in this study operate within a continuous action space, suitable for fine-grained portfolio rebalancing rather than discrete buy/hold/sell actions. Specifically, A2C, PPO, SAC, DDPG, TD3, TQC and RecurrentPPO output a real-valued action vector representing the target allocation weights across N assets. These actions are first normalized via a softmax transformation to ensure the portfolio remains fully invested and long-only. Hence, each element of the action vector determines the proportion of wealth allocated to a specific stock rather than discrete trading decisions. Although the environment internally computes transaction costs for both increasing (“buy”) and decreasing (“sell”) positions, the agent does not explicitly select between discrete actions such as buy, hold or sell; instead, it continuously adjusts portfolio weights at each step to optimize long-term return.

In summary, all evaluated DRL algorithms use continuous control policies: on-policy agents (A2C, PPO and RecurrentPPO) learn smooth stochastic policies parameterized by Gaussian distributions, while off-policy agents (DDPG, TD3, SAC and TQC) employ deterministic or entropy-regularized continuous actions for high-resolution rebalancing in continuous time.

Seven DRL algorithms are evaluated: A2C, PPO, SAC, DDPG, TD3, TQC and RecurrentPPO. All are implemented in Stable-Baselines3, with sb3_contrib extensions where available. A2C and PPO are on-policy actor–critic models, while DDPG, TD3 and SAC are off-policy methods using replay buffers and target networks. TQC and RecurrentPPO extend these designs by improving value distribution accuracy and capturing temporal dependencies. Each algorithm operates in continuous action spaces and adapts to volatile financial conditions (Espiga-Fernández et al., 2024; Liu et al., 2022; Rezaei and Nezamabadi-Pour, 2025; Bühler et al., 2019; Schulman et al., 2017b).

These algorithms are selected to represent a diverse set of learning paradigms, including on-policy and off-policy training and deterministic and stochastic policies, as well as distributional and recurrent extensions.

This diversity enables a systematic and controlled comparison of learning stability, exploration behavior and robustness under non-stationary and noisy financial environments.

All agents are trained on the in-sample window for 150,000 timesteps using a vectorized environment (DummyVecEnv) to parallelize rollouts. Random seeds are fixed for Python, NumPy and PyTorch to ensure reproducibility.

To improve reproducibility, Table 1 summarizes the main hyperparameter settings used for each DRL algorithm. These parameters were kept fixed across all experimental replications, except for the random seeds, to ensure that performance differences mainly reflect algorithmic behavior rather than tuning variation. All models were implemented using the same Stable-Baselines3-based pipeline, identical training and testing windows and the same transaction-cost setting.

Although the hyperparameters were not exhaustively optimized, they were selected from commonly used Stable-Baselines3 configurations and kept consistent across runs to preserve a fair comparative design.

Each DRL agent follows the actor–critic paradigm with distinctive update objectives and stability mechanisms designed for continuous control.

The inclusion of multiple algorithmic variants allows a systematic comparison of learning stability, exploration behavior and robustness under non-stationary financial environments.

A2C estimates both the policy gradient and the state-value function synchronously, providing simplicity and stability but limited sample efficiency under non-stationary market conditions (Mnih et al., 2016).

where $ri(θ)=πθ(ai∣si)πθold(ai∣si)$⁠. The clipping mechanism constrains policy updates to prevent large deviations, improving training stability and robustness in continuous-action environments (Schulman et al., 2017a,b).

DDPG combines deterministic policy gradients with replay-buffer experience reuse, enabling efficient learning in high-dimensional continuous spaces but susceptible to overestimation bias and instability under noise (Lillicrap et al., 2015; Fujimoto et al., 2018).

reducing DDPG's overestimation bias via twin critics and delayed updates (Rezaei and Nezamabadi-Pour, 2025).

Entropy regularization promotes exploration and robustness to regime shifts (Rezaei and Nezamabadi-Pour, 2025).

TQC extends SAC by truncating upper quantiles of the value distribution to prevent overestimation (Bühler et al., 2019):

Incorporates an LSTM layer into PPO to capture temporal dependencies (Schulman et al., 2017b):

enhancing continuity in time-correlated markets.

Each agent trains for 150,000 timesteps and is evaluated deterministically on the out-of-sample period. Training and testing repeat ten times with seeds 42 + r, r = 1,…, 10, to average stochastic variability. Performance metrics include cumulative and annualized return, annualized volatility, Sharpe ratio and maximum drawdown. All results are logged and aggregated across runs.

Two benchmark strategies are used:

1. Equal-weight portfolio: average of daily returns across all assets and

2. Minimum-variance portfolio: daily rebalanced using PyPortfolioOpt with 252-day rolling covariance and Ledoit–Wolf shrinkage (Ledoit and Wolf, 2004).

While a buy-and-hold VN-Index provides a reference for overall market growth, this study focuses on cross-sectional portfolio allocation within a fixed asset universe. Therefore, equal-weight and minimum-variance portfolios are used as benchmarks to ensure a consistent comparison with DRL agents under identical constraints.

Model performance is assessed using standard measures:

1. Cumulative return: $∏t=1T(1+rt)−1$

2. Annualized return: (1 + Cumulative Return)^252∕T − 1

3. Annualized volatility: $252σ(rt)$

4. Sharpe ratio: $E[rt−rf]σ(rt)$ (assume r_f = 0)

5. Maximum drawdown: $maxt∈[0,T]maxτ≤tVτ−Vtmaxτ≤tVτ$

Each model is checkpointed every 25,000 steps and evaluated on the trading window. Policies are reloaded without retraining, and out-of-sample performance is measured on the same metrics to visualize convergence for Sharpe ratio, return, volatility and drawdown.

The workflow consists of five stages: (1) data preprocessing and feature extraction; (2) environment setup with rebalancing and transaction costs; (3) DRL training (A2C, PPO, DDPG, TD3, SAC, TQC and RecurrentPPO); (4) benchmark computation and (5) out-of-sample evaluation and convergence analysis.

This section presents the empirical results of the DRL portfolio experiments conducted on five Vietnamese equities (SBT.VN, BID.VN, CTG.VN, HPG.VN and VCB.VN). All models were trained on daily data from 2018 to 2024 and evaluated over the 2023–2025 trading horizon using ten random-seed replications. The analysis focuses on cumulative returns, convergence stability, risk–return trade-offs and comparative robustness against traditional benchmarks. Overall, DRL-based strategies tend to outperform equal-weight and minimum-variance baselines on most evaluation metrics, with a few noted exceptions, indicating superior adaptability under emerging-market conditions.

Figure 1 shows the median cumulative returns of the DRL agents – A2C, PPO, DDPG, TD3, SAC, TQC and RecurrentPPO – together with the equal-weight and minimum-variance benchmarks. All agents achieved positive cumulative growth, with divergence becoming more visible after mid-2024. Among them, PPO exhibits the strongest and most persistent upward trajectory, clearly outperforming all other agents and benchmarks throughout most of the evaluation horizon. TD3 ranks second, displaying comparable growth with slightly higher short-term fluctuations, reflecting its deterministic yet responsive exploration strategy. A2C remains tightly coupled to the equal-weight baseline, suggesting limited flexibility under regime changes. Overall, the results confirm PPO's superior ability to capture persistent market trends and maintain convergence stability, with TD3 emerging as a solid but more volatile alternative.

Figure 2 illustrates the checkpoint-based convergence of the Sharpe ratio for a representative training run. Each colored line corresponds to an agent in Figure 2: PPO (green), TD3 (cyan), RecurrentPPO (light purple), A2C (blue-violet), DDPG (orange), SAC (light orange) and TQC (pink). Across 25,000–150,000 training steps, most algorithms display stable and gradually improving Sharpe ratios, confirming consistent policy learning and numerical reliability.

PPO (green) exhibits the most pronounced upward trajectory, surpassing all others and steadily improving beyond a Sharpe of 1.0 by the end of training. TD3 (cyan) and RecurrentPPO (light purple) follow closely, maintaining smooth convergence with moderate variance. A2C (blue-violet) stabilizes early near 0.9, indicating limited exploration capacity under synchronous updates. In contrast, SAC (light orange) and TQC (pink) show nearly flat curves around 0.85–0.9, suggesting conservative learning dynamics and slower adaptation to changing reward structures. Overall, these convergence patterns reaffirm the stability advantage of PPO and its clipped-surrogate policy design, while TD3 and RecurrentPPO provide robust but slightly less responsive alternatives.

Figure 3 compares the median annual return (blue) and median maximum drawdown (red) across ten replications. All DRL agents achieve annualized returns between 15 and 17% on average, exceeding the benchmarks. PPO delivers the highest annual return (0.1666) and the best downside profile by Calmar (0.9606) and Sortino (1.3120), indicating that it captures trends while containing drawdowns. RecurrentPPO trades some return (0.1618) for superior risk adjustment, achieving the top Sharpe ratio (1.0461). By contrast, SAC and TQC earn lower total returns (0.1537 each) and weaker drawdown efficiency (Calmar 0.7866) despite Sharpe ratios just above 1.0; their Omega (1.1468) and Sortino (1.1343) also trail PPO and even sit close to or below the equal-weight/min-var baselines (Omega 1.1792, Sortino 1.2570). TD3 and DDPG are mid-pack (returns 0.1638; Sharpe 0.8842), while A2C aligns with the baselines on Sharpe (0.8976) but offers no advantage in drawdown control.

To complement the figure, Table 2 reports median annualized return, volatility, Sharpe, Calmar, Omega, Sortino and training stability. Three points stand out from the table: (1) PPO combines the top return (0.1666) with the strongest Calmar (0.9606) and Sortino (1.3120), giving the best return–drawdown trade-off; (2) RecurrentPPO yields the highest Sharpe (1.0461), confirming its risk-adjusted edge despite a slightly lower return; (3) SAC/TQC, while clearing Sharpe ≈ 1.04, underperform on total return and drawdown efficiency (Calmar 0.7866; Omega 1.1468), so they do not dominate the benchmarks. Overall, PPO and RecurrentPPO provide the most attractive balance of profitability and risk control, whereas SAC/TQC are more conservative and less efficient on downside metrics in this setting.

It is worth noting that some agents with relatively high annual returns nonetheless record lower Sharpe ratios. This occurs because Sharpe ratio measures excess return per unit of volatility. Thus, when a model like DDPG or A2C generates higher raw returns accompanied by proportionally higher variance or more frequent drawdowns, the resulting Sharpe ratio declines even if the mean return is strong. In other words, high annual returns in these models stem from aggressive allocation shifts and momentum chasing during short-term rallies, which amplify gains but also increase downside exposure. PPO and RecurrentPPO, by contrast, maintain smoother volatility profiles and smaller tail losses, yielding more stable risk-adjusted outcomes even if their nominal returns are only marginally higher. This distinction illustrates that profitability alone is not sufficient; risk normalization via Sharpe or Calmar ratios provides a more accurate gauge of long-term portfolio efficiency.

In terms of Sharpe ratio leadership, RecurrentPPO (1.0461) stands out as the top performer, followed closely by SAC (1.0406) and TQC (1.0406). PPO, while slightly lower on Sharpe (0.9251), remains the most balanced model overall due to its combination of higher return, superior Calmar and stronger Sortino ratio. This pattern suggests that the Sharpe-optimal agents (RecurrentPPO, SAC and TQC) achieve steadier risk-adjusted growth, whereas PPO achieves the highest absolute gain without significant volatility penalties – an important trade-off between stability and profitability in emerging-market contexts.

Figure 4 displays the distribution of Sharpe ratios across the ten experimental runs. PPO attains the highest median Sharpe ratio (around 0.93–0.94) with the tightest interquartile range, indicating both strong and consistent risk-adjusted performance across runs. A2C performs comparably on median value but exhibits slightly wider dispersion, suggesting moderate volatility sensitivity. RecurrentPPO, SAC, TD3 and TQC show greater variability and lower medians (around 0.85–0.88), implying less consistent out-of-sample efficiency. The baseline strategies (equal-weight and min-variance) maintain stable but lower Sharpe levels near 0.90, confirming their conservative nature. Overall, PPO emerges as the most balanced and reliable performer, combining the highest median Sharpe with the narrowest variability, reflecting robust policy convergence and stable generalization.

Across analyses, PPO is the clear overall leader: it delivers the highest annual return (0.1666) and the strongest downside profile by Calmar (0.9606) and Sortino (1.3120), and it shows the most persistent cumulative-growth trajectory in Figure 1 alongside the smoothest Sharpe-ratio convergence in Figure 2. TD3 ranks second on cumulative growth but with slightly higher short-term fluctuations. RecurrentPPO provides the best risk-adjusted balance in the tabulated median (Sharpe = 1.0461) and maintains smooth learning, though it trades off some total return (0.1618). SAC and TQC register Sharpe values just above 1.0 in the table but underperform on total return (0.1537) and drawdown efficiency (Calmar 0.7866; Sortino 1.1343), leaving them close to or below the EW/MinVar baselines on several downside metrics. A2C and DDPG yield moderate results – with A2C tracking the baselines on Sharpe and DDPG/TD3 mid-pack – yet they offer less adaptability. Taken together, PPO offers the best all-around profile (return, stability and dispersion control); RecurrentPPO excels on risk-adjusted efficiency; TD3 is a solid but more volatile alternative and SAC/TQC behave more conservatively without dominating the benchmarks in this setting.

The empirical results offer several insights into how DRL algorithms behave under the structural and liquidity constraints of the Vietnamese equity market. Although all models generated positive cumulative returns, the differences in performance reveal that algorithmic design choices – particularly entropy regularization, recurrent state encoding and clipped policy updates – determine the balance between profitability and stability. These findings should be interpreted within a controlled experimental setting, where differences in outcomes are primarily attributable to algorithmic design rather than variations in data preprocessing, feature construction or implementation details.

First, PPO shows clear practical advantages as a stable and high-performing on-policy method for portfolio allocation. It achieved the highest annual return (0.1666) and the strongest Calmar (0.9606) and Sortino (1.3120) ratios, illustrating its ability to generate consistent profits with limited downside exposure. Its clipped surrogate objective constrains policy updates and prevents excessive parameter shifts that could destabilize learning under noisy conditions. This stability explains PPO's smooth convergence and moderate volatility despite market disruptions, consistent with the evidence of Schulman et al. (2017a) on the algorithm's steady performance in continuous-control settings. Importantly, Figure 4 also shows PPO with the highest median Sharpe (about 0.93–0.94) and the tightest dispersion across runs, corroborating its reliability. From a practical perspective, this suggests that PPO's policy update mechanism is well-suited for environments characterized by noisy signals and frequent regime shifts, where overly aggressive updates can degrade performance.

This finding is consistent with prior studies that highlight PPO's stability in continuous control tasks but contrasts with some financial applications where entropy-based methods dominate, suggesting that market structure plays a critical role.

Second, RecurrentPPO achieved the highest median Sharpe in the tabulated summary (1.0461) and demonstrated smooth convergence across checkpoints. Its LSTM-based memory allows the policy to retain temporal dependencies, which enhances learning stability in partially observable environments (Schulman et al., 2017b). This feature appears particularly valuable in Vietnam's market context, where autocorrelation and momentum effects often coincide with rapid structural changes. However, its more conservative learning dynamics slightly reduced raw returns compared to PPO, indicating a deliberate trade-off between responsiveness and risk control. The apparent difference between Figure 4 (where PPO's boxplot median is highest) and the table (where RecurrentPPO's median Sharpe is highest) reflects two complementary summaries: the boxplot shows distributional behavior on the 10-run panel, while the table reports the aggregated median statistic across replications. This highlights the importance of evaluating DRL models using both distributional and aggregated metrics, as reliance on a single summary statistic may obscure important aspects of performance variability.

Third, SAC and TQC delivered risk-adjusted scores just above 1.0 but posted lower total returns (0.1537) and weaker downside control (Calmar 0.7866; Sortino 1.1343). Their entropy-regularized and distributional critics improve exploration robustness (Haarnoja et al., 2018; Bühler et al., 2019); yet, in this dataset, they did not translate that robustness into superior return–drawdown efficiency relative to PPO or even EW/MinVar on some downside metrics. TQC's quantile truncation still mitigates overestimation, but the overall profile remains conservative. This outcome suggests that exploration-oriented mechanisms, while theoretically appealing, may lead to overly cautious policies in relatively small or noisy datasets, limiting their ability to exploit profitable opportunities. This contrasts with prior findings where entropy-based methods often outperform in larger or more liquid markets, suggesting that their advantages may diminish under data constraints and market inefficiencies.

Fourth, A2C and DDPG displayed moderate to weaker risk-adjusted outcomes relative to other agents. A2C's synchronous updates limited adaptability to regime changes, while DDPG's deterministic policies were more sensitive to reward noise during turbulent phases (Fujimoto et al., 2018). Nevertheless, both maintained average returns above traditional benchmarks, indicating that even baseline DRL architectures can outperform static portfolios under emerging-market dynamics (Rezaei and Nezamabadi-Pour, 2025). This reinforces the broader insight that adaptive policy learning, even without advanced stabilization mechanisms, can capture structural inefficiencies in less efficient markets.

Finally, benchmarking against equal-weight and minimum-variance portfolios reinforces the effectiveness of the DRL framework. The MinVar benchmark – based on the Ledoit–Wolf shrinkage estimator (Ledoit and Wolf, 2004) – provides stability but lacks adaptability. In contrast, PPO, TD3 and RecurrentPPO adaptively rebalance exposure and learn dynamic risk management, producing higher cumulative growth and better multi-metric balance; SAC/TQC remain more defensive and do not dominate the baselines on drawdown-sensitive measures. This comparison underscores the importance of adaptability in portfolio allocation, particularly in environments where return distributions and correlations evolve over time.

While multiple random seeds are used to ensure robustness across training runs, this study does not perform an extensive sensitivity analysis on hyperparameters or transaction cost assumptions. Future research should examine the extent to which these factors influence performance stability.

A further practical consideration concerns execution quality. Although the environment incorporates proportional transaction costs, it does not explicitly model slippage or liquidity constraints. This omission may matter in the Vietnamese market, where order-book depth can vary across assets and trading sessions. In practice, larger rebalancing trades may be executed at less favorable prices than those observed in daily data, which could reduce realized returns and increase turnover-related losses. Accordingly, the out-of-sample results should be interpreted as reflecting a controlled backtesting framework rather than full market-impact-adjusted execution.

This study does not include a VN-Index benchmark, as the objective is to evaluate allocation strategies within a fixed asset set. Equal-weight and minimum-variance portfolios therefore provide more appropriate comparators by isolating allocation effects rather than market-wide movements.

In summary, DRL-based portfolio allocation can outperform static benchmarks in emerging markets when enhanced with mechanisms that promote exploration stability and temporal awareness. PPO emerges as the most balanced and practically robust algorithm, combining high profitability, steady Sharpe improvements and consistent convergence. RecurrentPPO complements this by offering the strongest risk-adjusted median in the tabulated results, while TD3 provides a responsive but slightly more volatile alternative. SAC and TQC contribute robustness in principle but, in this application, remain conservative and less efficient on downside control. Future studies should extend the evaluation horizon, incorporate transaction-cost and slippage effects and examine cross-market transferability to assess whether these advantages persist across economic contexts. More broadly, the findings highlight the value of controlled and reproducible evaluation frameworks for understanding the interaction between algorithmic design and market structure in DRL-based portfolio management.

This study investigates the application of DRL for multi-asset portfolio optimization in the Vietnamese stock market. Five representative equities – SBT.VN, BID.VN, CTG.VN, HPG.VN and VCB.VN – were selected from the Ho Chi Minh Stock Exchange for their liquidity, sectoral diversity and reliable data coverage. Seven DRL agents – A2C, PPO, DDPG, TD3, SAC, TQC and RecurrentPPO – were trained on daily price data from 2018 to 2024 and evaluated out-of-sample from 2023 to 2025, ensuring both stable and volatile market regimes were represented.

Methodologically, the framework compared stochastic and deterministic actor–critic structures to evaluate how exploration design affects portfolio stability. PPO proved the most balanced algorithm, achieving the highest annual return (0.1666) and superior Calmar and Sortino ratios, sustaining growth under volatility; it also showed the highest median Sharpe and lowest dispersion in the distributional analysis. RecurrentPPO attained the top Sharpe in the tabulated median (1.0461) via LSTM-based temporal awareness, providing consistent learning and risk control while accepting slightly lower returns. TD3 delivered the second-strongest cumulative growth but with more short-term variability. SAC and TQC achieved Sharpe just above 1.0 yet underperformed on total return and drawdown efficiency, reflecting a more conservative posture. Deterministic models (DDPG, TD3) were moderate but steady, while A2C was stable with limited adaptability. These results support arguments that entropy-driven objectives (Haarnoja et al., 2018) and recurrent encoding (Schulman et al., 2017b) enhance robustness in high-volatility, partially observable environments.

Empirically, PPO, RecurrentPPO and TD3 outperform equal-weight and minimum-variance benchmarks on key dimensions; SAC/TQC remains competitive on some risk-adjusted metrics but do not dominate the baselines on drawdown-sensitive measures. The ability of these DRL agents – especially PPO – to internalize dynamic risk management without relying on static covariance assumptions (Ledoit and Wolf, 2004) demonstrates the adaptability of learning-based frameworks to frontier markets. Overall, DRL-based systems represent a viable, data-driven alternative to classical optimization by learning to balance return and risk directly from market interactions.

A limitation of this study is the relatively small asset universe. Although the five-stock setting enables a controlled comparison of DRL algorithms, it does not represent a fully diversified portfolio. The limited number of assets may restrict diversification opportunities and affect the ability of the learned policies to generalize to larger, more heterogeneous investment universes. Future research should extend the framework to broader Vietnamese and ASEAN equity portfolios to test whether the relative ranking of DRL algorithms remains stable.

Another limitation concerns the simplified reward function. Because the agents are trained using daily portfolio returns without explicit volatility or drawdown penalties, some models may adopt more aggressive allocation policies. Future research should examine whether risk-adjusted reward formulations, such as the differential Sharpe ratio, Sortino-based rewards or drawdown-aware penalties, improve downside control and reduce excessive risk-taking in emerging-market portfolio allocation.

Although the present analysis identifies PPO and RecurrentPPO as the leading models, a limitation is that the current results provide limited direct interpretability of their allocation mechanisms. In particular, the study does not fully decompose whether superior performance arises from dynamic rebalancing across assets or from concentration in one or two high-performing stocks. Future research should therefore include portfolio weight dynamics, asset-level attribution and turnover decomposition to better explain why specific DRL agents outperform others.

A further limitation concerns portfolio scalability. The five-equity universe was deliberately selected to create a controlled and reproducible benchmark in which algorithmic behavior can be isolated from excessive asset-level noise and missing-data problems. Nevertheless, the results should not be interpreted as direct evidence of scalability to larger universes such as the VN30 index. When the number of assets increases, the state and action spaces expand rapidly, and the set of feasible portfolio-weight combinations becomes substantially more complex. This curse of dimensionality can reduce sample efficiency, slow convergence, amplify estimation noise and weaken out-of-sample stability. Accordingly, the present results establish a controlled baseline for DRL-based portfolio allocation in Vietnam, while future research should examine whether the relative advantages of PPO, RecurrentPPO and TD3 persist in larger and more diversified portfolios.

Future research should extend the analysis to longer horizons and alternative reward-shaping schemes emphasizing downside protection and recovery dynamics. Incorporating macroeconomic indicators, sentiment signals, or cross-asset correlations could enhance responsiveness to structural shifts. Future work could also incorporate market-level benchmarks (e.g. VN-Index) to evaluate performance relative to overall market trends. Finally, exploring ensemble DRL models, transfer learning across ASEAN markets and more realistic execution assumptions – including slippage, bid–ask spread dynamics and market-impact effects – would strengthen the robustness and generalizability of DRL applications in real-world portfolio management.

Future research should also compare DRL agents with forecast-then-optimize strategies, such as LSTM forecasting combined with Markowitz optimization, to further evaluate whether direct policy learning offers robust advantages over sequential prediction-based portfolio construction.