You can't edit the job without disabling.
I was able to figure this out on my own:
I just edited the job, and went to the mailbox selection screen, then just continued forward and saved it.
The user doesn't show because it's actually connecting to your office 365 and showing a list of active users.
Not sure if this is a bug or a feature, but the ex-user is off the list and I'm not getting any more warnings.