That's the biggest hurdle if someone wanted to make a convincing fake. Once the model is trained it takes far less processing to use, but copying someone's voice is a completely separate process. Then you need the venue to appear legitimate.

Any convincing deepfake will still be a Hollywood level production.