But, then, what do you do with them?
If one mouse can find a hole/way to get in your house, so can others. And if you release the captured critters just outside, they are sure to find the way back in.
Putting the critter in someone else's gardens might work, if they're far enough away - maybe.
Keep in mind, also, that city mice won't survive well if taken out to the countryside. They are too 'light in the loafers' to fit in, and the rough and rowdy bogtrotter mice will torment them no end - maybe even beat them up from time to time - too cruel.