Abstract: To address the multi variable constraints and response lag in the low load wet dry state conversion process of ultra supercritical once through boilers, a full condition control oriented digital twin is constructed, and a reinforcement learning environment with multiple continuous actions is established. A deep reinforcement learning control method integrating expert policy warm start and risk adaptive safety shielding is proposed. Simulations are carried out under eight working conditions including 10% 81% boiler rated load (BRL) state transition, variable rate load ramping, combined disturbance and load reversal. The results show that the expert baseline reduces the average load error by 36.08% compared with nominal feedforward control; the pre trained agent outperforms nominal feedforward control; the short trained Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm suffers cross scene performance degradation. After introducing safety shielding, the average load and pressure errors decrease by 31.70% and 29.00% respectively compared with the unshielded TD3. No hard constraint violations of pressure and water storage tank level occur throughout the whole process. Safety shielding can effectively improve the operational stability of the reinforcement learning method for wet dry state conversion control.