In the present work, three materials-based nanoscale TiO2 compound materials were directly fabricated via hydrothermal synthesis methods to be used in gamma-ray shielding applications as ceramics and paints. X-ray diffraction and energy dispersive X-ray techniques were utilized to characterize the formation of TiO2, Mo–TiO2, and Co–TiO2 nanoparticles. Additionally, transmission electron microscopy was utilized in order to study the distribution and particle sizes of the synthesized composites. The study affirms a homogeneous distribution for the fabricated nanocomposites and clarified that the particle sizes of the fabricated composites ranged between 10 nm and 15 nm. Additionally, the Monte Carlo simulation studies the role of Mo and Co in the enhancement of the fabricated composites-based TiO2 nano-sheets. The simulation study illustrates that the Co–TiO2 composites have a linear attenuation coefficient of 0.845 cm−1 which is greater than the linear attenuation coefficient of the TiO2 composite (0.733 cm−1) and Co–TiO2 composite (0.765 cm-1) at gamma-ray energy of 0.1 MeV. Additionally, the half-value thickness, lead equivalent thickness, and exposure build-up factors for Mo–TiO2 composites are less than that calculated for both TiO2 and Co–TiO2 composites along the studied gamma-ray energy interval between 0.015 MeV and 15 MeV. The enhanced shielding capabilities of the synthesized nanocomposites indicate their potential for gamma radiation shielding applications, demonstrating that tailoring the composition and microstructure of ceramic nanomaterials can optimize attenuation properties for contemporary medical physics applications. The improved gamma ray blocking potential of these engineered nanocomposites highlights their promise for shielding against harmful gamma radiation from modern sources. © 2024 Elsevier Ltd.