Hmmm if it was a direct flame you could use something like this https://www.amazon.com/dp/B00AFVLVI4/ref=cm_sw_r_cp_api_glc_fabc_avQ9Fb9V0DQVA

which is just different chemicals that burn different colors. Not sure how to safely make flames shoot out though

Amazon link for the lazy: Mystical Fire Flame Colorant Vibrant Long-Lasting Pulsating Flame Color Changer for Indoor or Outdoor Use 0.882 oz Packets 6 Pack https://www.amazon.com/dp/B00AFVLVI4/ref=cm_sw_r_cp_api_i_YgOKCbFGACT9E