Zero-Shot Payload Adaptation for Robotic Manipulators via Domain-Randomized Deep Reinforcement Learning
DOI:
https://doi.org/10.22581/0621Keywords:
Domain Randomization, Robotic Manipulator Control, Trajectory Tracking, Zero-Shot Generalization, Payload Variation, Deep Reinforcement LearningAbstract
Reliable robotic manipulator control under unforeseen payload changes has been studied because such variation creates a major challenge for consistent automated operation. Traditional model-based approaches have been viewed as unreliable since parameter uncertainty weakens system response, while adaptive methods have been reported with slow convergence behaviour. A key gap has been described as the lack of one stable non adaptive policy that ensures steady reliability across many physical parameters. This study has been designed to create and verify a robust control policy that supports strong zero-shot generalization for unseen payload conditions. The method has been formed by linking deep reinforcement learning with domain randomization inside a detailed simulator that trains an agent through Proximal Policy Optimization. The central innovation has been presented as the ability to produce generalized control across many randomized dynamic models used during simulated training cycles. Simulation results have been reported with superior trajectory tracking and stable system motion across unseen payloads, showing reduced error values in varied tests. Measured outcomes have shown low mean error of 0.032 rad with only a 20% rise from baseline and steady energy use.
Downloads
References
[1] Z. Liu, Q. Liu, W. Xu, L. Wang, and Z. Zhou, “Robot learning towards smart robotic manufacturing: A review,” Robot. Comput.-Integr. Manuf., vol. 77, p. 102360, Oct. 2022, doi: 10.1016/j.rcim.2022.102360.
[2] W. Zhang et al., “Research progress and development trend of surgical robot and surgical instrument arm,” Int. J. Med. Robot., vol. 17, no. 5, p. e2309, Oct. 2021, doi: 10.1002/rcs.2309.
[3] S. Thakar, P. Rajendran, A. M. Kabir, and S. K. Gupta, “Manipulator Motion Planning for Part Pickup and Transport Operations From a Moving Base,” IEEE Trans. Autom. Sci. Eng., vol. 19, no. 1, pp. 191–206, Jan. 2022, doi: 10.1109/TASE.2020.3020050.
[4] P. Asgharian, A. M. Panchea, and F. Ferland, “A Review on the Use of Mobile Service Robots in Elderly Care,” Robotics, vol. 11, no. 6, p. 127, Nov. 2022, doi: 10.3390/robotics11060127.
[5] P. Gümbel and K. Dröder, “Precision optimized process design for highly repeatable handling with articulated industrial robots,” CIRP Ann., vol. 73, no. 1, pp. 25–28, 2024, doi: 10.1016/j.cirp.2024.03.008.
[6] A. Milecki and P. Nowak, “Review of Fault-Tolerant Control Systems Used in Robotic Manipulators,” Appl. Sci., vol. 13, no. 4, p. 2675, Feb. 2023, doi: 10.3390/app13042675.
[7] G. Zepeda, R. Kelly, and C. Monroy, “A Theoretical Conversion of Torque Controllers for Handling a Position Servo-actuated Robot Manipulator Model,” Int. J. Control Autom. Syst., vol. 22, no. 4, pp. 1301–1312, Apr. 2024, doi: 10.1007/s12555-022-1243-9.
[8] Z. Jiang, X. Zhang, and G. Liu, “Trajectory tracking control of a 6-DOF robotic arm based on improved FOPID,” Int. J. Dyn. Control, vol. 13, no. 4, p. 137, Apr. 2025, doi: 10.1007/s40435-025-01620-x.
[9] J. Cvejn, “The variable-inertia modified computed-torque control of robot manipulators,” Multibody Syst. Dyn., vol. 57, no. 2, pp. 157–179, Feb. 2023, doi: 10.1007/s11044-022-09862-9.
[10] Y. Tao, S. Chen, H. Liu, J. Wan, H. Wei, and T. Wang, “Robot hybrid inverse dynamics model compensation method based on the BLL residual prediction algorithm,” Robotica, vol. 43, no. 1, pp. 350–367, Jan. 2025, doi: 10.1017/S0263574724001905.
[11] K. V. Sancak and Z. Y. Bayraktaroglu, “Nonlinear Computed Torque Control of 6-Dof Parallel Manipulators,” Int. J. Control Autom. Syst., vol. 20, no. 7, pp. 2297–2311, July 2022, doi: 10.1007/s12555-021-0198-6.
[12] X. Shen, K. Zhou, R. Yu, and B. Wang, “Design of Adaptive RBFNN and Computed-torque Control for Manipulator Joint Considering Friction Modeling,” Int. J. Control Autom. Syst., vol. 20, no. 7, pp. 2340–2352, July 2022, doi: 10.1007/s12555-021-0146-5.
[13] C. Feng, Z. Chen, W. Jin, and W. Guo, “Motion Control and Simulation Analysis of a Manipulator Based on Computed Torque Control Method,” in Intelligent Robotics and Applications, vol. 14267, H. Yang, H. Liu, J. Zou, Z. Yin, L. Liu, G. Yang, X. Ouyang, and Z. Wang, Eds., in Lecture Notes in Computer Science, vol. 14267. , Singapore: Springer Nature Singapore, 2023, pp. 10–22. doi: 10.1007/978-981-99-6483-3_2.
[14] Y. Qin, Z. Yin, Q. Yang, and K. Zhang, “Dynamics Parameter Identification of Articulated Robot,” Machines, vol. 12, no. 9, p. 595, Aug. 2024, doi: 10.3390/machines12090595.
[15] J. Li, Y. Liang, and Z. Wu, “Tracking control via time-varying feedback for an uncertain robotic system with both output constraint and dead-zone input,” ISA Trans., vol. 154, pp. 147–159, Nov. 2024, doi: 10.1016/j.isatra.2024.08.028.
[16] Z. Anjum, S. Samo, A. Nighat, A. U. Nisa, M. A. Soomro, and R. Alayi, “Design and Modeling of 9 Degrees of Freedom Redundant Robotic Manipulator,” J. Robot. Control JRC, vol. 3, no. 6, pp. 800–808, Dec. 2022, doi: 10.18196/jrc.v3i6.15958.
[17] M. Xie, T. Chen, S. Ni, and C. Feng, “Flexible payload transportation using cooperative space manipulators with statics compensation,” ISA Trans., vol. 157, pp. 329–339, Feb. 2025, doi: 10.1016/j.isatra.2024.12.001.
[18] T. Xu et al., “An online payload identification method based on parameter difference for industrial robots,” Robotica, vol. 42, no. 8, pp. 2690–2712, Aug. 2024, doi: 10.1017/S026357472400105X.
[19] F. Aghili, “Impedance Control for Manipulators Handling Heavy Payloads,” 2024, doi: 10.48550/ARXIV.2409.14639.
[20] S. Talukder and R. Kumar, “Robust Stability of Neural-Network-Controlled Nonlinear Systems With Parametric Variability,” IEEE Trans. Syst. Man Cybern. Syst., vol. 53, no. 8, pp. 4820–4832, Aug. 2023, doi: 10.1109/TSMC.2023.3257269.
[21] X. Liang, Z. Yao, Y. Ge, and J. Yao, “Reinforcement learning based adaptive control for uncertain mechanical systems with asymptotic tracking,” Def. Technol., vol. 34, pp. 19–28, Apr. 2024, doi: 10.1016/j.dt.2023.05.016.
[22] N. P. Lawrence et al., “Machine learning for industrial sensing and control: A survey and practical perspective,” Control Eng. Pract., vol. 145, p. 105841, Apr. 2024, doi: 10.1016/j.conengprac.2024.105841.
[23] M. Cai, E. Aasi, C. Belta, and C.-I. Vasile, “Overcoming Exploration: Deep Reinforcement Learning for Continuous Control in Cluttered Environments From Temporal Logic Specifications,” IEEE Robot. Autom. Lett., vol. 8, no. 4, pp. 2158–2165, Apr. 2023, doi: 10.1109/LRA.2023.3246844.
[24] A. Belherazem, R. Salim, A. Laidani, and M. Chenafa, “Vibration Control of a Two-Link Flexible Manipulator,” Autom. Control Comput. Sci., vol. 58, no. 3, pp. 346–358, June 2024, doi: 10.3103/S0146411624550017.
[25] V. S. Gupta and Y. G. Joshi, “Modelling and Study of Dynamics and Control of a Two-Link Serial Manipulator,” in Recent Developments in Mechanics and Design, D. K. Singh, S. Hegde, and A. Mishra, Eds., in Lecture Notes in Mechanical Engineering. , Singapore: Springer Nature Singapore, 2024, pp. 1–11. doi: 10.1007/978-981-97-3874-8_1.
[26] P. M. Wensing and J.-J. E. Slotine, “Coriolis Factorizations and their Connections to Riemannian Geometry,” 2023, arXiv. doi: 10.48550/ARXIV.2312.14425.
[27] S. Zhou, X. Li, M. Yin, D. Shang, and Z. Li, “Dynamic modeling and vibration suppression for double flexible manipulator with a tip payload,” J. Mech. Sci. Technol., vol. 37, no. 9, pp. 4441–4454, Sept. 2023, doi: 10.1007/s12206-023-0803-0.
[28] J. Haack, F. Stark, S. Vyas, F. Kirchner, and S. Kumar, “Adaptive Model-Base Control of Quadrupeds via Online System Identification using Kalman Filter,” 2025, arXiv. doi: 10.48550/ARXIV.2506.13432.
[29] J. N. Clelland, T. J. Klotz, and P. J. Vassiliou, “Dynamic Feedback Linearization of Control Systems with Symmetry,” 2021, doi: 10.48550/ARXIV.2103.05078.
[30] M. Ming, Z. Feng, J. Ling, and X. Xiao, “Disturbance observer-based model prediction control with real-time modified reference for a piezo-actuated nanopositioning stage,” Trans. Inst. Meas. Control, vol. 42, no. 4, pp. 813–822, Feb. 2020, doi: 10.1177/0142331219878048.
[31] H. Ye and Y. Song, “Adaptive Control With Guaranteed Transient Behavior and Zero Steady-State Error for Systems With Time-Varying Parameters,” IEEECAA J. Autom. Sin., vol. 9, no. 6, pp. 1073–1082, June 2022, doi: 10.1109/JAS.2022.105608.
[32] S. Gong, Z. Xu, L. Cheng, and X. Huang, “Self-organizing model reference adaptive control for aircraft with enhanced persistent excitation,” Aerosp. Sci. Technol., vol. 145, p. 108875, Feb. 2024, doi: 10.1016/j.ast.2024.108875.
[33] Y. Ding, C. Guo, Y. Luo, H. Cai, X. Zhao, and R. Qu, “Research on PID self-tuning control based on recursive least squares parameter identification for the fast steering mirror-based optoelectronic tracking system,” Sens. Actuators Phys., vol. 386, p. 116323, May 2025, doi: 10.1016/j.sna.2025.116323.
[34] M. M. Mustafa, C. D. Crane, and I. Hamarash, “Adaptive-Sliding Mode Trajectory Control of Robot Manipulators with Uncertainties,” 2024, arXiv. doi: 10.48550/ARXIV.2408.03102.
[35] T. Duong, A. Altawaitan, J. Stanley, and N. Atanasov, “Port-Hamiltonian Neural ODE Networks on Lie Groups For Robot Dynamics Learning and Control,” 2024, arXiv. doi: 10.48550/ARXIV.2401.09520.
[36] D. M. Le, O. S. Patil, C. F. Nino, and W. E. Dixon, “Accelerated Gradient Approach For Deep Neural Network-Based Adaptive Control of Unknown Nonlinear Systems,” IEEE Trans. Neural Netw. Learn. Syst., vol. 36, no. 4, pp. 6299–6313, Apr. 2025, doi: 10.1109/TNNLS.2024.3395064.
[37] A. A. Malikopoulos, “Separation of learning and control for cyber–physical systems,” Automatica, vol. 151, p. 110912, May 2023, doi: 10.1016/j.automatica.2023.110912.
[38] C. Tang, B. Abbatematteo, J. Hu, R. Chandra, R. Martín-Martín, and P. Stone, “Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes,” 2024, arXiv. doi: 10.48550/ARXIV.2408.03539.
[39] C. Rizzardo, F. Chen, and D. Caldwell, “Sim-to-real via latent prediction: Transferring visual non-prehensile manipulation policies,” Front. Robot. AI, vol. 9, p. 1067502, Jan. 2023, doi: 10.3389/frobt.2022.1067502.
[40] E. Inanc, Y. Gurses, A. Habboush, Y. Yildiz, and A. M. Annaswamy, “Neural Network Adaptive Control with Long Short-Term Memory,” 2023, arXiv. doi: 10.48550/ARXIV.2301.02316.
[41] K. Guo and Y. Pan, “Composite adaptation and learning for robot control: A survey,” Annu. Rev. Control, vol. 55, pp. 279–290, 2023, doi: 10.1016/j.arcontrol.2022.12.001.
[42] R. Raileanu, M. Goldstein, D. Yarats, I. Kostrikov, and R. Fergus, “Automatic Data Augmentation for Generalization in Deep Reinforcement Learning,” 2020, arXiv. doi: 10.48550/ARXIV.2006.12862.
[43] E. Korkmaz, “Adversarial Robust Deep Reinforcement Learning Requires Redefining Robustness,” 2023, arXiv. doi: 10.48550/ARXIV.2301.07487.
[44] P. Trentsios, M. Wolf, and D. Gerhard, “Overcoming the Sim-to-Real Gap in Autonomous Robots,” Procedia CIRP, vol. 109, pp. 287–292, 2022, doi: 10.1016/j.procir.2022.05.251.
[45] G. Tiboni, K. Arndt, and V. Kyrki, “DROPO: Sim-to-Real Transfer with Offline Domain Randomization,” 2022, arXiv. doi: 10.48550/ARXIV.2201.08434.
[46] M. Wang et al., “Meta-Reinforcement Learning Based on Self-Supervised Task Representation Learning,” 2023, arXiv. doi: 10.48550/ARXIV.2305.00286.
[47] Z. Yin, C. Ye, H. An, W. Lin, and Z. Wang, “Robot Manipulation Skills Transfer for Sim-to-Real in Unstructured Environments,” Electronics, vol. 12, no. 2, p. 411, Jan. 2023, doi: 10.3390/electronics12020411.
[48] Y. Liu et al., “Efficient End-to-End Detection of 6-DoF Grasps for Robotic Bin Picking,” 2024, arXiv. doi: 10.48550/ARXIV.2405.06336.
[49] Z. Jiang, G. Tian, Y. Cui, T. Liu, Y. Gu, and Y. Wang, “Digital Twin System for Home Service Robot Based on Motion Simulation,” 2023, arXiv. doi: 10.48550/ARXIV.2309.05993.
[50] P. D. Ngo, M. Tejedor, and F. Godtliebsen, “Data-Driven Robust Control Using Reinforcement Learning,” Appl. Sci., vol. 12, no. 4, p. 2262, Feb. 2022, doi: 10.3390/app12042262.
[51] P. Corke, Robotics, Vision and Control, vol. 118. in Springer Tracts in Advanced Robotics, vol. 118. Cham: Springer International Publishing, 2017. doi: 10.1007/978-3-319-54413-7.
[52] R. Kirk, A. Zhang, E. Grefenstette, and T. Rocktäschel, “A Survey of Zero-shot Generalisation in Deep Reinforcement Learning,” 2021, doi: 10.48550/ARXIV.2111.09794.
[53] M. Raoufi, H. Habibi, A. Yazdani, and H. Wang, “Robust Prescribed Trajectory Tracking Control of a Robot Manipulator Using Adaptive Finite-Time Sliding Mode and Extreme Learning Machine Method,” Robotics, vol. 11, no. 5, p. 111, Oct. 2022, doi: 10.3390/robotics11050111.
[54] A. Kloss, G. Martius, and J. Bohg, “How to train your differentiable filter,” Auton. Robots, vol. 45, no. 4, pp. 561–578, May 2021, doi: 10.1007/s10514-021-09990-9.
[55] P. D. Nguyen, N. H. Nguyen, and H. T. Nguyen, “Adaptive control for manipulators with model uncertainty and input disturbance,” Int. J. Dyn. Control, vol. 11, no. 5, pp. 2285–2294, Oct. 2023, doi: 10.1007/s40435-023-01115-7.
[56] F. Jacob, E. H. Grosse, S. Morana, and C. J. König, “Picking with a robot colleague: A systematic literature review and evaluation of technology acceptance in human–robot collaborative warehouses,” Comput. Ind. Eng., vol. 180, p. 109262, June 2023, doi: 10.1016/j.cie.2023.109262.
[57] J. Arents and M. Greitans, “Smart Industrial Robot Control Trends, Challenges and Opportunities within Manufacturing,” Appl. Sci., vol. 12, no. 2, p. 937, Jan. 2022, doi: 10.3390/app12020937.
[58] H. Hu, K. Zhang, A. H. Tan, M. Ruan, C. G. Agia, and G. Nejat, “A Sim-to-Real Pipeline for Deep Reinforcement Learning for Autonomous Robot Navigation in Cluttered Rough Terrain,” IEEE Robot. Autom. Lett., vol. 6, no. 4, pp. 6569–6576, Oct. 2021, doi: 10.1109/LRA.2021.3093551.
[59] S. Zhang, D. Zhai, Y. Xiong, J. Lin, and Y. Xia, “Safety‐critical control for robotic systems with uncertain model via control barrier function,” Int. J. Robust Nonlinear Control, vol. 33, no. 6, pp. 3661–3676, Apr. 2023, doi: 10.1002/rnc.6585.
Published
Issue
Section
License
Copyright (c) 2026 Mehran University Research Journal of Engineering and Technology
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
How to Cite
Similar Articles
- Kamran Kazi, Arbab Nighat, Farida Memon, Tarique Rafique, Azam Rafique Memon, Robust outdoor trajectory mapping using CNN features and loop closure optimization , Mehran University Research Journal of Engineering and Technology: Vol. 44 No. 3 (2025): July Issue
- Zahid Hussain Khaskheli , Qasim Ali, M Shehram Shah , Din Muhammad , Farast Ali, Hina Ali, Maria Khalid, Zoha Waheed, Umer Gaffar, Improving Human Action Recognition Through Cross-View Camera-Aware Augmentation , Mehran University Research Journal of Engineering and Technology: Vol. 45 No. 3 (2026): July Issue
- Chandar Kumar, Dur Muhammad Soomro, Najeeb Ur Rehman Malik, Data Driven Learning Based Robust Control for Nonlinear Electromechanical Systems: A Case Study on SRV02 , Mehran University Research Journal of Engineering and Technology: Vol. 44 No. 4 (2025): October Issue
- Ayesha Rafique , Rabia Noor Enam, Madiha Abbasi , Noreen Akram, Application of deep learning models for pest detection and identification , Mehran University Research Journal of Engineering and Technology: Vol. 44 No. 2 (2025): April Issue
- Syed Noor Hussain Shah, Muhammad Shahan Ibad, Sumaira Johar, Omar Bin Samin, Afsheen Khalid, Sami Ur Rahman, Enhancing Malaria Diagnosis: A Hybrid Deep Learning Approach Integrating CNN-Swin Transformer, and Grad-CAM , Mehran University Research Journal of Engineering and Technology: Vol. 45 No. 2 (2026): April Issue
- Syed Shabeeb Raza, Muhammad Imran Saleem, Hassan Ahmed Khan, Shaheena Noor, Ali Yousuf Khan, Eman Razzaq, A Comparative Analysis of Software-Defined Networking (SDN) Security Policies With and Without Reinforcement Learning Automation , Mehran University Research Journal of Engineering and Technology: Vol. 45 No. 2 (2026): April Issue
- Muhammad Zubair, Syed Muhammad Sajjad Haider, Ans Rehman, Muhammad Saqib Javed, Aashir Waleed, Rimsha Chauhdary, PackMLP: A Patch Based MLP Model for Intelligent Inventory Forecasting and Automation with A-LIQ QR Scanning Integration , Mehran University Research Journal of Engineering and Technology: Vol. 45 No. 1 (2026): January Issue
- Yang Haoxuan, Faizan Qamar, Syed Hussain Ali Kazmi, Khairul Akram Zainol Ariffin, Kashif Nisar, Hybrid Model for Data-Driven Channel Estimation in mmWave Massive MIMO Systems with Compressed Sensing and Deep Learning , Mehran University Research Journal of Engineering and Technology: Vol. 45 No. 2 (2026): April Issue
- Maryam Arain, Sheeraz Ahmed Memon, Attiya Baqai, Mohsin Memon, Kundan Kumar, Sajid Hussain, Zuhaib Siddiqui, Rapid Detection and Classification of Escherichia Coli, Salmonella Typhi, and Staphylococcus Aureus using Smartphone-Based Deep Learning System , Mehran University Research Journal of Engineering and Technology: Vol. 45 No. 3 (2026): July Issue
- Gift Mbuzi, Abdur Rashid Sangi, Baha Ihnaini, Anil Carie, Sruthi Sivarajan, Satish Anamalamudi, Data-driven insights into groundwater quality: machine and deep learning approaches , Mehran University Research Journal of Engineering and Technology: Vol. 44 No. 3 (2025): July Issue
You may also start an advanced similarity search for this article.