From Mark Suster on how to hire:
Yes, you need to call their references. Expect them to sing their praises. If a person doesn’t list the most positive references to begin with then you know they’re not worth hiring. But you have to find a way to call people not on their list. You need to be careful and respectful of them because it’s possible that their boss doesn’t know they’re interviewing. But you have to find a way to get some info. Often you can find people if you learn to become a LinkedIn Ninja.