Ooh, found not one but *two* Japanese language exchange groups in my area: Japan Exchange Society and Bay Area Japanese Language Meetup group. The former appears to have more regular meetings. The latter's more a social networking site; so far I've seen one guy who says he does iaido and one who has an icon of a chibi Ultraman in a cowboy hat. (Sho-man, not Seven.) I sense Good Sign.
(What? I figure since I'm not getting socialization at work any more I might as well get out and start networking.)