SSH Allow Password For One User, Rest Only Allow Public Keys

Is it possible with ssh to allow passwords from a certain user, but deny using passwords for everybody else?

Basically I want to allow password auth for user justin but everybody else must use public keys.

PasswordAuthentication no

Seems to be global though, no way to specify by user.