You can click Cancel Recording as it's starting.

You can also click Stop Recording while it's recording.

You can either pause or end recording and that will remove the Fathom notetaker from the call.