From FirstRound Capital:
Intuit Co-founder Scott Cook is very particular about who he hires. And after 30 years, he has it down to a science — including how to get the truth out of a reference check.
The reality is that people want to be nice, so almost everyone will start out by saying how great the candidate is. Cook’s advice is to completely ignore this opening feedback. When they’re done, ask this: “Among all the people you’ve seen in this position, on a zero to 10 scale, where does this person rank?” If the person says seven, immediately ask why they aren’t a nine or a ten. “Then you’ll finally start learning what this person really thinks,” Cook says.
He also ends every call by asking the person for other people who could speak to the candidate’s performance. The further you get away from provided references, the more valuable the data.