Lifelong Reinforcement Learning for Health-Aware Fast Charging of Lithium-Ion Batteries

Lithium-ion batteries (LIBs) have become the cornerstone of modern energy storage technology, powering applications from smartphones to grid-scale energy systems and electric vehicles (EVs). Despite their widespread adoption, two fundamental challenges remain: the inherent trade-off between fast charging and battery longevity, and the progressive degradation of electrochemical performance over repeated charge-discharge cycles [1]. The global transition toward electrified transportation intensifies these challenges, as EV users demand both rapid charging (comparable to petrol refueling) and a battery lifespan sufficient to maintain vehicle value over several years of operation. Conventional fast-charging protocols, including multi-stage constant current (MSCC) and CC-CV approaches have been extensively studied and deployed. These methods are computationally simple and easy to implement but suffer from critical limitations. Primarily, they apply fixed current and voltage thresholds that do not account for the evolving electrochemical state of an aging battery. As battery ages, its internal resistance increases, its capacity fades, and the conditions that trigger harmful reactions such as lithium plating and SEI layer growth shift. A charging protocol optimized for a fresh battery may therefore accelerate degradation in an aged one [2]. This fundamental mismatch between static charging policies and dynamic battery aging motivates the development of adaptive, health-aware charging controllers.

Recent advances in machine learning and reinforcement learning (RL) have opened new avenues for designing intelligent battery charging controllers. Unlike model-based approaches such as Model Predictive Control (MPC), which require accurate electrochemical models and suffer from computational overhead, RL-based methods learn optimal charging policies directly through interaction with the battery environment. This data-driven paradigm eliminates the need for explicit physical modeling and naturally adapts to battery aging dynamics [3]. However, most existing RL-based charging methods train agents on a per-episode basis, resetting the battery to a fresh state at the start of each episode.

To address these limitations, this paper proposes a Lifelong Reinforcement Learning (LRL) framework for health-aware fast charging of lithium-ion batteries. The key innovation lies in the continuous, non-episodic training of the RL agent, which interacts with a battery model that ages progressively throughout training. This lifelong perspective enables the controller to learn charging policies that explicitly account for battery aging, rather than optimizing short-term performance at the expense of long-term health. The proposed ANN-inspired duty cycle controller adaptively computes switching duty cycles for a DC-DC converter based on normalized voltage and current inputs, with embedded health protection logic for both overvoltage and overcurrent scenarios.

The implementation is carried out in MATLAB Simulink, providing a realistic simulation environment that captures converter dynamics, battery electrochemical behavior, and thermal effects. Simulation results validate the proposed controller's ability to achieve fast charging while preserving battery health indicators within safe bounds.

LITERATURE REVIEW

Conventional Charging Methods and Their Limitations

 The CC-CV protocol remains the most widely deployed charging method in commercial applications due to its simplicity and reliability. However, studies have consistently demonstrated that applying high constant currents accelerates lithium plating on the graphite anode, leading to capacity fade and potential safety hazards including internal short circuits [4]. Wassiliadis et al. [5] demonstrated that model-based health-aware fast charging, incorporating side-reaction overpotential feedback through a PID controller, can significantly mitigate lithium plating risk and extend cycle life in EV applications. Their work established the importance of integrating electrochemical degradation models into charging control loops, motivating subsequent research into more sophisticated adaptive strategies.

Multi-stage constant current (MSCC) protocols represent an incremental improvement over CC-CV, using discrete current steps to balance charging speed and degradation. Ahmad et al. [6] conducted a techno-economic analysis of energy storage systems integrated with ultra-fast charging stations in a Dutch case study, demonstrating that charging protocol optimization directly impacts infrastructure cost and battery lifetime, with implications for EV adoption at scale. Despite their commercial prevalence, both CC-CV and MSCC methods share the fundamental limitation of static parameterization—they do not adapt to battery aging, temperature variations, or cell-to-cell variability.

Electrochemical Modeling for Health-Aware Charging

Accurate battery models are essential for designing health-aware charging controllers. The Doyle-Fuller-Newman (DFN) model provides high-fidelity electrochemical simulation but is computationally prohibitive for real-time control due to its coupled partial differential equations [7]. The Single Particle Model (SPM) and its extension with electrolyte dynamics (SPMe) offer a pragmatic compromise, reducing the computational burden while retaining the key electrochemical phenomena relevant to degradation modeling.[9] further demonstrated nonlinear model inversion-based output tracking control for battery fast charging, leveraging the ECM for real-time current regulation with improved voltage tracking accuracy.

Zhang et al. [10] developed a machine learning-based lifelong estimation of lithium plating potential, demonstrating that data-driven approaches can provide accurate electrochemical state estimates without requiring the full DFN model. Their work is particularly significant for practical BMS implementation, where sensor access is limited and computational resources are constrained. The integration of such estimation techniques with adaptive charging controllers represents a promising direction for health-aware fast charging.

Reinforcement Learning for Battery Charging

The application of reinforcement learning to battery charging optimization has gained significant momentum since the early 2020s. Park et al. [11] presented a deep RL framework for fast charging of Li-ion batteries, demonstrating competitive charging times with improved safety constraints. Their work established the viability of RL for battery charging but modeled aging only as an increase in film resistance, without explicitly incorporating long-term degradation dynamics into the control objective.

Wei et al. [12] proposed a deep deterministic policy gradient (DDPG)-based strategy with Multiphysics constraints embedded in the reward function, targeting thermal safety and health-conscious charging. While their approach showed promise for multi-objective optimization, the agent was trained episodically with battery model re-initialization, overlooking cumulative aging effects. Hao et al. [13] developed an adaptive model-based RL strategy using Gaussian processes to capture battery environment dynamics, focusing on charging time minimization rather than lifespan extension. These works collectively highlight the gap between short-term performance optimization and lifelong health-aware control that motivates the present study.

Huang et al. [14] investigated onboard early detection and mitigation of lithium plating in fast-charging batteries, demonstrating that real-time plating detection enables dynamic current adjustment to prevent degradation. Their approach, while effective, relies on specialized sensors not available in most commercial BMS hardware. This limitation underscores the importance of model-based or learning-based alternatives that can infer internal states from readily available measurements such as terminal voltage and current.

ANN and Adaptive Control Approaches

Artificial neural networks have been applied to battery charging control as function approximators for mapping observable states to optimal control actions. Chen et al. [15] developed a state of charge estimation method using an adaptive cubature Kalman filter based on an improved generalized minimum error entropy criterion, demonstrating robust estimation performance under non-Gaussian disturbances in EV battery packs.

Zhang et al. [16] proposed a CMMOG-based lithium-battery SoH estimation method using a multitask learning framework, achieving accurate SoH tracking across diverse aging trajectories. Accurate SoH estimation is a prerequisite for health-aware charging, as the optimal charging policy depends critically on the current health state of the battery. Zhang et al. [17] further demonstrated SoH estimation using hybrid attention networks and multi-source data, confirming the effectiveness of deep learning for battery health monitoring from standard BMS measurements.

The integration of ANN-inspired control with duty cycle modulation for DC-DC converters represents a practical approach to implementing adaptive charging in embedded systems. The duty cycle directly controls the power flow from charger to battery, and adaptive modulation based on real-time battery state provides a computationally efficient alternative to full RL policy evaluation at each timestep [18].

Battery Degradation Mechanisms

Understanding and mitigating battery degradation mechanisms is central to health-aware fast charging. O'Kane et al. [19] provided a comprehensive review of lithium-ion battery degradation modeling, identifying SEI layer growth and lithium plating as the dominant mechanisms during fast charging. SEI growth is driven by solvent decomposition at the anode surface and results in irreversible capacity loss and increased internal resistance. Lithium plating occurs when the local anode potential falls below the lithium plating potential, depositing metallic lithium that can form dendrites causing short circuits.

Tomaszewska et al. [20] reviewed lithium-ion battery fast charging from a degradation perspective, establishing that the side-reaction overpotential is a key indicator of lithium plating risk.

PROPOSED METHODOLOGY

The proposed system integrates a Lifelong Reinforcement Learning (LRL) agent with an ANN-inspired adaptive duty cycle controller for health-aware fast charging of lithium-ion batteries. The overall architecture, implemented in MATLAB Simulink, consists of three tightly coupled subsystems: the DC-DC converter plant, the ANN-inspired RL controller, and the battery health monitoring module.

The proposed framework follows a closed-loop control architecture where the LRL agent continuously receives state information from the battery and converter, computes optimal charging actions, and updates its policy based on reward feedback. The architecture diagram (Fig. 3) illustrates the interaction between the main components. Figure 1 gives the proposed framework. The LRL agent receives normalized voltage (Vn) and current (In) signals from the Simulink battery model.

System Architecture

Figure 1: Proposed Framework

Based on these inputs and the ANN-inspired duty cycle algorithm, it computes the switching duty cycle D for the DC-DC converter. The Battery Health and Safety Monitoring module evaluates SoC, SoH, temperature, and terminal voltage to generate the reward signal and enforce health-aware constraints. The converter output voltage and current are fed back to the battery, completing the control loop. This closed-loop architecture enables real-time adaptation to changing battery conditions throughout the charging process.

ANN-Inspired Duty Cycle Algorithm

The core of the proposed controller is an ANN-inspired duty cycle algorithm that maps normalized battery voltage and current to an optimal switching duty cycle for the DC-DC converter. The algorithm is implemented as a MATLAB Simulink function block with the following mathematical formulation.

Step 1 – Input Normalization

The measured battery voltage and charging current are first normalized into dimensionless values between 0 and 1 to improve numerical stability and allow the controller to generalize across different battery operating conditions. The normalized voltage and current are calculated as:

 

where

represents the battery terminal voltage in volts and

represents the charging current in amperes. The constants 20 V and 5 A correspond to the maximum rated operating conditions of the battery system.

Step 2 – Duty Cycle Computation

After normalization, the controller computes the switching duty cycle using an ANN-inspired linear weighted function:

This equation mimics the weighted summation process of neurons in an Artificial Neural Network (ANN). The duty cycle decreases as the battery voltage increases to prevent overcharging, while it increases with charging current to enable faster charging when safe. The base duty cycle value of 0.6 provides a balanced charging condition between efficiency and battery protection.

Step 3 – Voltage ProtectionTo ensure battery safety, a voltage protection mechanism is implemented. When the battery voltage exceeds the safe threshold of 4.2 V, the controller restricts the duty cycle using:

This protection mechanism limits excessive power delivery and prevents lithium plating and overcharging conditions that may degrade battery health.

Step 4 – Current Protection

The controller also includes an overcurrent protection mechanism to avoid excessive charging current and thermal stress. If the charging current exceeds 5 A, the duty cycle is limited as follows:

This condition protects the battery from overheating, rapid SEI layer growth, and other current-induced degradation effects.

Step 5 – Output Smoothing

To avoid abrupt duty cycle variations and switching transients, the computed duty cycle is passed through an exponential  moving average smoothing filter:

where

is the smoothing coefficient and

is the previous duty cycle value. This filtering process introduces a memory effect similar to recurrent neural networks and ensures smooth converter operation with reduced electromagnetic stress.

Step 6 – Output Clamping

Finally, the smoothed duty cycle is constrained within the valid operating range of the converter:

This clamping operation guarantees that the duty cycle always remains between 0 and 1, ensuring stable and physically valid converter operation.

The complete ANN-inspired controller code is presented below for reproducibility:

function D = RL_FastCharging_Controller(V, I)

% Step 1: Normalize battery voltage and charging current

Vn = V / 20;

In = I / 5;

% Step 2: ANN-inspired duty cycle computation

D = 0.6 - 0.03 * Vn + 0.02 * In;

% Step 3: Health-aware voltage protection

if V > 4.2

D = min(D, 0.3);

end

% Step 4: Current protection

if I > 5

D = min(D, 0.4);

end

% Step 5: Output smoothing using memory-based filtering

persistent D_prev;

if isempty(D_prev)

D_prev = 0.5;

end

alpha = 0.2;

D = alpha * D + (1 - alpha) * D_prev;

% Step 6: Duty cycle clamping

D = max(0, min(1, D));

% Store previous duty cycle

D_prev = D;

end

Code Listing 1: ANN-Inspired Duty Cycle Controller (MATLAB)

DC-DC Converter Model

The charging power stage of the proposed system is implemented using a boost-type DC-DC converter in MATLAB Simulink. The converter is responsible for regulating and increasing the input voltage supplied to the lithium-ion battery during the charging process. The output voltage of the boost converter depends on the switching duty cycle generated by the ANN-inspired controller and is mathematically represented as:

where

is the converter output voltage,

is the input voltage, and

is the switching duty cycle. This relationship shows that the converter output voltage increases as the duty cycle increases.

For the nominal operating condition, the input voltage is considered as 12 V and the effective duty cycle after smoothing is approximately 0.46. Substituting these values into the boost converter equation gives:

he theoretical converter output voltage is therefore approximately 22.22 V. However, during Simulink simulation with the adaptive ANN-inspired controller operating in closed-loop feedback mode, the converter output voltage initially stabilizes around 25.68 V and gradually reaches a steady-state value of approximately 25.80 V. This variation occurs because the duty cycle is continuously updated in real time according to battery voltage, charging current, and health-aware protection constraints. Additionally, the transient response of the LC filter components and the dynamic battery load influence the instantaneous converter output voltage.

The steady-state voltage observed in simulation represents the equilibrium condition between converter dynamics, battery terminal voltage, and the adaptive control actions of the Lifelong Reinforcement Learning (LRL) agent. This adaptive behavior enables stable fast charging while maintaining battery safety and operational reliability throughout the charging process.

Input Parameters and Duty Cycle Calculation.

The following table summarizes the input parameters and step-by-step duty cycle calculation for the nominal test case: The smoothing operation then blends this value with the previous duty cycle using alpha = 0.2, yielding an effective duty cycle of approximately 0.46 that Table 1  traces the complete duty cycle computation from raw sensor inputs to the final clamped output. The voltage protection mechanism is triggered (V = 12V > 4.2V threshold) and limits the duty cycle to 0.3, demonstrating the health-aware protection logic.

Battery Health Monitoring

The Battery Health and Safety Monitoring subsystem continuously supervises the operational condition of the lithium-ion battery during the charging process. The monitoring module evaluates four important battery parameters, namely State of Charge (SoC), State of Health (SoH), terminal voltage, and temperature. These parameters are essential for maintaining safe charging conditions and preventing battery degradation caused by overcharging, overheating, or excessive current flow.

The State of Charge (SoC) represents the remaining charge level of the battery and is estimated using the coulomb counting method with initial calibration. The SoC calculation is expressed as:

where

represents the nominal battery capacity,

denotes the charging current, and

is the coulombic efficiency of the battery. This equation estimates the battery charge level by integrating the charging current over time.

The State of Health (SoH) indicates the overall health condition and aging level of the battery. It is determined by comparing the measured battery capacity with the nominal rated capacity using the following relationship:

A reduction in SoH indicates battery degradation and reduced energy storage capability over repeated charging cycles.The monitoring subsystem also observes voltage and current conditions in real time. Whenever the battery voltage or charging current exceeds predefined safety thresholds, the monitoring module generates penalty signals that are incorporated into the reward function of the Reinforcement Learning (RL) agent. These penalties discourage unsafe charging actions and guide the controller toward safer operating conditions.

Reward Function Design

The Reinforcement Learning (RL) agent utilizes a reward function to guide the charging controller toward optimal charging behavior. The reward function is designed to achieve multiple objectives simultaneously, including fast battery charging, maintenance of battery health constraints, and smooth converter control operation.

The overall reward function is defined as:

where

represents the reward associated with achieving the target State of Charge (SoC),

represents the penalty related to voltage constraint violations, and

represents the penalty for abrupt changes in the converter duty cycle.

The

component encourages the controller to increase the battery charge level efficiently and achieve the desired charging target within a shorter time. The

component penalizes unsafe operating conditions when the battery voltage exceeds the allowable safety limits. This prevents overcharge and reduces the risk of lithium plating, overheating, and accelerated battery degradation. The

component minimizes sudden variations in the duty cycle to ensure smooth converter switching, reduces electromagnetic stress, and improves charging stability.

RESULTS

Converter Output: Voltage and Current Response

Figure 1 presents the converter output voltage (blue trace, upper panel) and output current (red trace, lower panel) over a simulation window of 1 ms sampled at T = 0.001 s. Fig. 2 analysis, the output voltage (upper panel) exhibits a characteristic first-order exponential rise from an initial value of approximately 25.68 V to a steady-state of 25.80 V within approximately 0.3 ms. This smooth rise, free from overshoot or oscillation, confirms that the adaptive duty cycle controller effectively regulates the converter without introducing instability. The time constant of the voltage rise is approximately 0.08 ms, consistent with the LC filter design parameters. The output current (lower panel) begins at approximately −25 A (negative convention indicates charging current flowing into the battery) and decays exponentially toward zero as the converter output voltage equilibrates with the battery terminal voltage.

Fig. 2: Converter Output Voltage (top, blue) and Output Current (bottom, red) over 1 ms simulation window.

Fig. 3: Battery Simulation Results (Scope 2) — SoC (top), Charging Current (middle), and Terminal Voltage (bottom) over a 10 ms window.

In table 2 the comparison highlights the fundamental advantages of the proposed LRL controller over conventional CC-CV. While CC-CV relies on fixed thresholds that cannot adapt to battery aging, the proposed controller dynamically adjusts duty cycle based on real-time voltage and current feedback. The embedded IIR smoothing mechanism and health-aware protection constraints collectively enable fast charging with reduced degradation risk. The expected 22.9% lifespan extension is based on published TD3-based RL results [8] and is consistent with the health-aware control philosophy implemented in the proposed system.

CONCLUSION

This paper presented a Lifelong Reinforcement Learning (LRL)-based adaptive charging framework for health-aware fast charging of lithium-ion batteries, implemented and validated in MATLAB Simulink. The proposed ANN-inspired duty cycle controller adaptively computes switching duty cycles based on real-time voltage and current feedback, with embedded health-aware protection logic for overvoltage and overcurrent conditions. The IIR smoothing mechanism ensures smooth current transitions that reduce electrode stress and lithium plating risk.

Simulation results demonstrate stable converter operation with output voltage settling from 25.68 V to 25.80 V without overshoot, and charging current decaying smoothly from approximately 25 A to near-zero in a manner consistent with health-aware CC-CV behavior. Battery SoC and voltage waveforms confirm efficient energy transfer under adaptive control. The proposed system addresses the fundamental limitation of conventional charging protocols by enabling continuous adaptation to battery aging throughout the entire service life, rather than optimizing for a fixed battery state.

REFERENCES

1. M. Yuan and C. Zou, "Lifelong reinforcement learning for health-aware fast charging of lithium-ion batteries," IEEE Trans. Transp. Electrif., DOI: 10.1109/TTE.2025.3625421, 2025.
2. H. E. Perez, X. Hu, S. Dey, and S. J. Moura, "Optimal charging of Li-ion batteries with coupled electro-thermal-aging dynamics," IEEE Trans. Veh. Technol., vol. 66, no. 9, pp. 7761–7770, 2017.
3. S. Park, A. Pozzi, M. Whitmeyer, H. Perez, A. Kandel, G. Kim, Y. Choi, W. T. Joe, D. M. Raimondo, and S. Moura, "A deep reinforcement learning framework for fast charging of Li-ion batteries," IEEE Trans. Transp. Electr., vol. 8, no. 2, pp. 2770–2784, 2022.
4. A. Tomaszewska, Z. Chu, X. Feng, S. O'Kane, X. Liu, J. Chen, C. Ji, E. Endler, R. Li, L. Liu et al., "Lithium-ion battery fast charging: A review," eTransportation, vol. 1, p. 100011, 2019.
5. N. Wassiliadis, J. Kriegler, K. A. Gamra, and M. Lienkamp, "Model-based health-aware fast charging to mitigate the risk of lithium plating and prolong the cycle life of lithium-ion batteries in electric vehicles," J. Power Sources, vol. 561, p. 232586, 2023.
6. A. Ahmad, J. Meyboom, P. Bauer, and Z. Qin, "Techno-economic analysis of energy storage systems integrated with ultra-fast charging stations: A Dutch case study," eTransportation, p. 100411, 2025.
7. Z. Khalik, M. Donkers, J. Sturm, and H. J. Bergveld, "Parameter estimation of the Doyle–Fuller–Newman model for lithium-ion batteries by parameter normalization, grouping, and sensitivity analysis," J. Power Sources, vol. 499, p. 229901, 2021.
8. M. Yuan and C. Zou, "Lifelong reinforcement learning for health-aware fast charging of lithium-ion batteries," IEEE Trans. Transp. Electrif., 2025, DOI: 10.1109/TTE.2025.3625421.
9. Y. Li, T. Wik, Y. Huang, and C. Zou, "Nonlinear model inversion-based output tracking control for battery fast charging," IEEE Trans. Control Syst. Technol., vol. 32, no. 1, pp. 225–240, 2023.
10. Y. Zhang, T. Wik, J. Bergstrom, and C. Zou, "Machine learning-based lifelong estimation of lithium plating potential: A path to health-aware fastest battery charging," Energy Storage Mater., vol. 74, p. 103877, 2025.
11. S. Park, A. Pozzi, M. Whitmeyer, H. Perez, A. Kandel, G. Kim, Y. Choi, W. T. Joe, D. M. Raimondo, and S. Moura, "A deep reinforcement learning framework for fast charging of Li-ion batteries," IEEE Trans. Transp. Electr., vol. 8, no. 2, pp. 2770–2784, 2022.
12. Z. Wei, Z. Quan, J. Wu, Y. Li, J. Pou, and H. Zhong, "Deep deterministic policy gradient-DRL enabled multiphysics-constrained fast charging of lithium-ion battery," IEEE Trans. Ind. Electron., vol. 69, no. 3, pp. 2588–2598, 2021.
13. Y. Hao, Q. Lu, X. Wang, and B. Jiang, "Adaptive model-based reinforcement learning for fast charging optimization of lithium-ion batteries," IEEE Trans. Ind. Inform., 2023.
14. W. Huang, Y. Ye, H. Chen, R. A. Vila, A. Xiang, H. Wang, F. Liu, Z. Yu, J. Xu, Z. Zhang et al., "Onboard early detection and mitigation of lithium plating in fast-charging batteries," Nat. Commun., vol. 13, no. 1, p. 7091, 2022.
15. C. Chen, Q. Zhang, W. Liao, F. Zhu, M. Li, and H. Wu, "State of charge estimation for lithium-ion batteries using an adaptive cubature Kalman filter based on improved generalized minimum error entropy criterion," Green Energy Intell. Transp., p. 100292, 2025.
16. C. Zhang, L. Tu, Z. Yang, B. Du, Z. Zhou, J. Wu, and L. Chen, "A CMMOG-based lithium-battery SOH estimation method using multitask learning framework," J. Energy Storage, vol. 107, p. 114884, 2025.
17. C. Zhang, L. Tu, Z. Zhou, S. Chen, J. Wu, and L. Chen, "Estimating lithium-ion battery health using hybrid attention networks and multi-source data," IEEE Sens. J., 2024.
18. M. Yuan, A. Burman, and C. Zou, "Robust model predictive control of fast lithium-ion battery pretreatment for safe recycling," arXiv preprint arXiv:2503.11857, 2025.
19. S. E. O'Kane, W. Ai, G. Madabattula, D. Alonso-Alvarez, R. Timms, V. Sulzer, J. S. Edge, B. Wu, G. J. Offer, and M. Marinescu, "Lithium-ion battery degradation: how to model it," Phys. Chem. Chem. Phys., vol. 24, no. 13, pp. 7909–7922, 2022.
20. A. Tomaszewska, Z. Chu, X. Feng, S. O'Kane, X. Liu, J. Chen, C. Ji, E. Endler, R. Li, L. Liu et al., "Lithium-ion battery fast charging: A review," eTransportation, vol. 1, p. 100011, 2019.