We had a similar came-and-went cat who we referred to as Scruffy -- he never did make up enough that I really thought of him as ours (and since he was an adult unaltered tom when we met, I didn't really want him living in the house), but he did make up enough that I was able to take him and have him neutered.