This study investigated the combustion process in a hydrotreated vegetable oil (HVO)–hydrogen dual-fuel operation using simultaneous imaging of the OH* and CH* chemiluminescence in a rapid compression and expansion machine (RCEM). In this operation, hydrogen served as the primary fuel, ignited by a small quantity of pilot fuel. CH* chemiluminescence was primarily detected in the pilot fuel combustion regions, whereas OH* chemiluminescence was detected in both the pilot fuel and hydrogen combustion regions, enabling the separation of pilot ignition and hydrogen flame propagation. The combustion mechanism was found to proceed through four distinct stages: autoignition of the pilot fuel, combustion of the mixture in the lean pilot fuel region, propagation of the hydrogen–air premixture flame, and flame propagation toward the wall and squish area. Furthermore, the effects of the pilot injection parameters on the combustion characteristics were systematically evaluated by varying the injection quantity, injection pressure, and nozzle specifications (hole diameter and number of holes). Increasing the pilot injection quantity improved the degree of constant volume of combustion but intensified the combustion near the wall, potentially increasing the cooling loss. Reducing the injection pressure shifted the autoignition location toward the center of the piston bowl, potentially reducing cooling loss but prolonging the combustion duration. With smaller injection quantities, fewer nozzle holes resulted in a higher second heat release rate peak, owing to the increased space for hydrogen flame propagation. Conversely, with larger injection quantities, a greater number of nozzle holes led to a shorter combustion duration while maintaining the combustion away from the wall.