I have the same issue using RPI zero and justboom dac zero.
I think that noise comes from the power supply of RPI zero.
Maybe isolation of the power supply is not good enough, especially the headphone circuit.
I built a headphone amp for zero before.
It used 5v form RPI Zero, it had the same problem.