Will we have the info we need at the end of the day?
Experience of folks on the loop -- can only afford one or, at the very most, two inexperienced interviewer per loop
Make changes if needed -- don't just "accept" the loop
If two groups involved, are there enough from both teams -- 2 people, one of which is a lunch interview won't cut it
Read the resume, does it match the job? Are there others it matches more closely
Managing the loop:
If you don't see feedback within an hour, ask the loop where the heck it is ... people learn to get the info out fast but you sometimes have to ask
Publicly whack those that send in crappy feedback and do it immediately:
"Nice guy" ... "really enjoyed talking to him" ... make sure the feedback has content. Don't accept 5 or 10 lines that say almost nothing
If it's a dev interview, insist on seeing code for all but lunch interview and sometimes the last one of the day ... if no code, then complain to the loop ... helps everyone learn
Complain early so things can be put back on track
Read all feedback and learn who can interview and who could use some advice ... get them the advice privately
Must take a position:
Tentative hire is just wrong ... call it publicly ... folks learn to never do this
Insist on a position ... must start with "hire" or "no hire" ... nothing else works
If uncertain, no hire
Calibrate interviews to individual being hired:
Insist on deep knowledge from senior folks but expect only core skills and potential from university hires ... make sure that all interviewers are asking the right level of questions for the person/position under discussion ... if not, correct quickly so the error doesn't continue
Adapt loop:
We paid big bucks to get this candidate in ... we shouldn't be bringing someone in that doesn't have good potential ... if they have good potential then don't send home immediately
If failing as dev ask yourself what they have done well with and where they might do better
Consider redirecting to PM or test role if failing as dev ... some of our best PM and Test team members were once Devs.
Jump in early ... if failing, call GPM or test manager if appropriate, get on the phone with recruiting and get other possibilities lined up ... it's your job to adapt the loop if it's not working
If someone really is failing, under no circumstances send home without 3 interviews ... I personally like to see four and, if there other borderline cases, I continue the loop ... it's a good investment
The AA interview often doesn't happen due to poor performance of candidate during the day ... try to see anyone who is borderline ... you may learn that they actually do meet the bar and it's more information to educate other folks on the loop ... unless clearly not going to get hired under any circumstances, do the AA
The interview
The best predictor of future performance is past performance. Very seldom does someone do OK at school, not excel at past jobs, yet suddenly become a superstar. If someone is a "hard worker", find a real example of this. If someone is an "excellent problem solver", find real examples. If they aren't there, it's simply not true.
Be casual and friendly and help the candidate relax. You need details fast -- you only have an hour -- so the more relaxed they are, the more successful you'll both be
If you aren't sure you want to hire them, and you usually won't be, don't waste much if anytime describing the project, how things work or other introductory stuff and focus the whole hour on learning what you need to learn. Candidates don't mind being challenged.
I look for Motivation, tools of the trade, raw IQ:
You'll get raw IQ from design questions and generally from the interactions during the interview
Tools of the trade: I'll survey standard computer science and education questions to make sure a candidate knows common algorithms, has appropriate education or equivalent work experience. Be particularly careful of candidates that have a masters in computer science but an undergraduate in something like English lit. When you see that, make very sure that they have the appropriate tools of the trade, standard algorithms, knows the basics about compilers, operating systems, multi-threaded systems, etc.
Motivation: comes from discussions of past project, how they tackle problems, how they have worked through tough problems, do they give up, did they actually ship or switch projects early, etc., etc.
I almost always do a pass through education and significant experience making sure I believe it and looking for interesting problems with which they have experience for a design questions. Having done that, I'll chose an example from a past project. I'll ask them for details about this and first assure myself that they actually did the work (you do remember recent work if you actually did it yourself and were a driver) and we'll debate a few design decisions to see how candidate deals with technical conflict or disagreement. Just going with whatever I'm saying without good technical reason isn't good enough. Then, I'll change a design parameter and ask what they'll do for V2. What if it won't fit into memory, I need multi-user concurrent access, I want to reduce response time by a factor of 10, etc., etc. This gives a good opportunity to see how they work. Keep firing questions and mods and see how they adapt.
Ask a coding question by going back to the survey through the education and choosing a problem that isn't too hard but they don't already know the answer -- you can only afford 20 min on coding so it can't be too hard. I ask them to code the solution as quickly as possible, make it syntactically correct, and of course the code itself must be correct
Always ask coding questions in 100% of all interviews prior to AA and, where it's not already clear prior to the AA, do it then too.
All the above is a dev interview but others aren't wildly different
For a SDE/T interview, I'll ask a simpler coding question but still want to see code. I substitute design questions with testing questions. I ask them to test a software component with which they are familiar, like a compiler symbol table, and instruct them to give me as many tests as possible as quickly as possible. If you don't get on the order of 15 or 20 pretty quickly, worry but keep trying. If you can't get them there, they don't make the bar.
For a PM interview, I'll not ask a coding question, will still ask the same design question, and I'll go looking for customer focus in and around design questions. Many questions on how to chose features, when to push a feature, challenge with schedule/feature trade-offs, challenge with late bugs and whether or not to fix. Basically, scenarios from various parts of the dev cycle to see how they react. Ask questions from their past to understand how they will work with other groups in conflict. Could they evangelize a feature with other groups, work with other groups that aren't happy with what we are doing, reduce conflict, etc. Present design sketches and ask them to criticize. Expect them to argue a point carefully, accurately, forcefully but not painfully. How to influence a dev team without direct leadership position.
For an architect interview, make 100% sure that the candidate has "been in the trenches" and has actually written production software and shipped production software. An architect needs credibility with the dev team and it absolutely won't work if they haven't "been there". Ask for their principles of development and expect to learn from them. Challenge them with feature/schedule trade-off and insist that they establish themselves as practical, able to argue hard for the right answer, but they have to be practical. Many questions in common with a PM loop but with a higher bar for success. How do they deal with conflict. How do the influence a dev team?
Learning from the loop:
Introduce new team members to interviewing by putting them on a loop with good interviewers. They'll read the feedback from everyone else and consider there own and will learn ... don't introduce more than one or, at the outside two, at a time
Go chat with new folks about what you learned and what they learned ... helps them calibrate and learn to look deep
Those that aren't giving good feedback need to be reminded of their responsibilities
There are some that just do crappy interviews and it never gets better ... don't put them on loops
Work with the HR team on who you actually want on loops ... I like to get involved in who is on the loop, sometimes tuned to candidate and sometimes to the work load we're currently under. At bare min, make sure that the loop is appropriate and pass on changes to HR so they can chose the right folks in the future
Rarely, but occasionally, I'll send a "here's what we could do better" note to the loop or "it was a great loop and here's why"
Very rarely, when it's a borderline case, I won't make a decision until I've spoken to everyone on the loop ... once I held a meeting since the feedback ranged from "I would rather quit than work with this person" to "we absolutely have to have them"
Keep an eye on how your hires do and learn from the result
