Strategies for Interviewing and Hiring: Part 2 of 2
CULTURE FIT, OR: TECHNICAL SKILLS MATTER, SOFT SKILLS ALSO MATTER
“Culture fit” is a term that is used to describe engineers that have the right personality for a given company. In the hiring process, “lack of culture fit” is used to turn away engineers who are good enough at coding but just don’t seem right for the company. But Bartram explains that “lack of culture fit” usually means “lack of enthusiasm for what a company does.” Enthusiasm is everything.
And on the flip side, engineers can likely relate to this. Would you really want to work somewhere doing something that you weren’t excited about? Some people can get by on that “it’s just a day job” mentality, but most people will find it unfulfilling after a while. Companies are looking for engineers who are excited about solving the tough problems of today. If you’re enthusiastic, how will you show it? Bartram suggests researching the company ahead of time and coming up with a list of questions to ask about to show your excitement.
Triplebyte does extensive technical screening, so when candidates who go through their process are rejected, it’s often for culture fit reasons. Responding to that, Bartram went back and interviewed all the companies extensively about what they look for. “What we observed is that [culture fit] is an extremely overloaded term. It doesn’t mean any one thing.”
A lot of it comes down to core company philosophy and organizing those perspectives as well as needs/wants. The Triplebyte approach has a ripple effect of success for all parties involved:
“Company A just has a strong belief that if you don’t program in compiled language, that’s the sign that you’re not serious. Company B thinks if you programmed in a compiled language, you’re stodgy and old. I think about 50% of interview failures are due to avoidable fit, avoidable mismatches. What we found is that if we can measure this stuff, we can get the interview pass rate basically doubled. Over the last two months, about 53% of our interviews have resulted in a job offer at a company, and that’s roughly twice the rate at those own companies and we’re able to get that rate just by basically measuring these things.”
There are a variety of nontechnical reasons to not hire a candidate that Bartram covers. Maybe the company thought you were:
- Arrogant
- Unkind
- Likely to clash with the current team
- Propelled by a different programming philosophy than their team
Bartram further breaks down culture fit into two parts:
- Soft skills: communication ability, ability to work on a team, general positive attitude, and excitement for what the company does.
- The specific, intentional choices the company makes about what traits and what types of employees they want to have.
“Facebook’s famous early motto of ‘move fast and break things.’ You can view that as a specific intentional choice that they want to bias towards sort of productive hackers as opposed to other approaches to engineering. Stripe is an example. Stripe has a particular focus on friendliness to a degree much further than basically any other company. Stripe wants to be an extremely friendly, compassionate place and they will reject people who everyone else would hire just because they’re only a 6 out of 10 from this rather than an 8 out of 10.” – Ammon Bartram
“The interesting thing is that companies themselves aren’t very crisp about the differences. It’s a catchall phrase that’s often pretty poorly defined,” said Bartram, and that’s where the weird science of technical interviews and human interaction collide. Part of it is “acknowledging that there are skills outside of technical brilliance that matter. The difficulty then is coming up with how do you actually measure those skills in a way you’re not being led astray by confounding factors?”
MEASURES OF SUCCESS: HOW TO CONDUCT AN INTERVIEW
Triplebyte has found that sometimes the most straightforward engineering problems can be the most predictive and concrete measures of progress, and that an engineer’s interviewing process can be a huge indicator of their working and problem-solving styles.
Bartram talked through how Triplebyte evaluates interviews:
“We like having like paired metrics. One metric to that is a raw measure of how far they got. The second measure, that’s a measure of, for example, process quality. We also like recording subjective numbers and objective numbers. An objective measure of quality will be; did they write tests? Did they create constants to record these values. Did they generalize the problem such that this aspect can be changed and it will still work? A long list of things like that we record yes, no, for these things. Did they do these things? We’re expressing some judgments, because there are some great programmers who might not do some of those things. In balance, we find that people who run a better process get more check marks than people who don’t. That’s these sort of the objective measure of code quality. The objective measure of progress is just how many of these tests passed within the time that they’re working.”
One experiment Triplebyte did was with take-home projects. People could do both a regular interview and then also a longer take-home project. “Looking at the difference in quality between the work people did during an hour or two when they’re on the spot versus when they had their time to think about it, it was really quite enlightening,” Bartram said. “Just people who we had written off as bad would then turn around and do really impressive professional-level work.”
“I think if you watch yourself, if you really sort of introspect to look at your own decision making process, when you’re trying to do an interview for a [nontechnical role], I think you’ll notice that — the thing you’re thinking 30 seconds in is what you’re going to think at the end. But when asking technical questions, the impression actually does change quite a lot over time.” – Ammon Bartram
Here’s Triplebyte’s perspective on interview questions.
The Bad:
Questions that require sort of a single leap of insight are very bad questions, because the answer ends up being extremely noisy. Questions where the answer can be given away easily are also bad.
Example of a bad interview question:
- “Imagine you’re walking up a flight of stairs and at every step you – I think a small step up once – Up one step of the stairs, or a large step up two steps, and the question is imagine you’re on the nth step of the flight of stairs, how many unique paths could you have taken from the bottom to arrive on the Nth step? If you think about it, it ends up that the answer is the Fibonacci sequence, and you can prove that to yourself and it’s kind of cool observation, but if you ask some people are going to know the answer and they might just say – Other people might know the answer and they’ll sort of pretend, they’ll reason it through and make it look like they’re really smart and figured it out. Others might get flustered and feel frustrated and not come up with the answer. That’s the interview, there’s very little you can do to sort of help. You tell someone, ‘Hey, it’s Fibonacci sequence.’ Suddenly you’ve given away and there’s no signal there.”
The Good:
A better approach is a question where the answer is 45 minutes of programming, and so there’s very little that could have been done sort of practicing the programming ahead of time to give them an advantage. You can measure the candidate moving through the process and give some of the noise people get stuck by giving some clues and not totally invalidate the question.
Example of a good interview question:
- “Let’s say matrix multiplication, ‘Please write a function that measures this and produces the answer.’ That’s a somewhat involved process. Knowing the answers shows slightly familiarity with linear algebra, but it still is fundamentally a series of steps that can be thought through. If a candidate gets stuck on one step, you can get them a little bit of a clue to help them move forward and they can still sort of get past that point and demonstrate competence later on.”
Whiteboarding might really get your motor running, practicing at home with challenging algorithms while wearing your comfy socks. Or it might be the bane of your existence, a necessary evil on the path of success. Bartram sees both sides of the whiteboarding controversy:
“What we found is just there are many many different ways to be good. Someone can be a great programmer and if you ask them at working on a whiteboard, their actual arguments for why a whiteboard is a good way to run a programming interview. This is something that a lot of people have very strong belief that whiteboards are terrible and there is a slight argument. The argument is giving an engineer a whiteboard in an interview, you are removing the constraint that their code actually compile and run and so they don’t get these lose time worrying about the exact syntax of an API call. They can just focus on the core logic. I think ultimately that falls down — If that’s what you care about, you should probably just have the programmer write code on a computer and then tell them that doesn’t need to run. But there is that argument, and so the more I’ve done this, the more I’ve just sort of come to challenge all of the beliefs that everyone has. The answer is that it’s some weighted combination. The true signal is just some weighted combination of all the different parts that everyone has in their head.”
Kyle Polich from the Data Skeptic podcast has some other perspectives for interviewers:
“Most people are not trained in how to interview properly. They go into a room and seem to ask questions along the lines of, ‘Does this person know the same things I know?’ Whereas they should actually be asking, ‘Can this person do the job?’
False-positives and false-negatives are another point of controversy within the hiring realm. Bartram reveals, “It does make sense to optimize for fewer false-positives and more false-negatives, but I think companies still underestimate their actual false-negative rate, and you can look at that by looking at the agreement rate between companies.”
There are a lot of different ways to conduct interviews, but Triplebyte’s innovative approach considers both the parameters that the hiring companies set forth, as well as their own special blend of unbiased interview conditions.
ON HIRING AND BEING HIRED
In conclusion, the right way to measure skill is not clear-cut and yet Triplebyte is doing a tremendous job in paving a new way forward on the hiring path. Instead of asking people you’re hiring the same questions that you were once asked, there are newer and more efficient ways to round out your team, as well as better ways to connect with the right team for you.
It’s not easy being a job-seeker or a hiring manager. But with a lot of preparation, clear goals and expectations, open communication, and an open mind, you’ll be on your way to finding your next great career path or hire. Just remember that a lot of people are going through the same frustrations and challenges, and a company like Triplebyte is making the whole process a little easier to find that needle in the haystack person or opportunity. Look beyond the old questions, and maybe even whiteboarding, and look ahead to the dawn of a more efficient and enjoyable interviewing era. Now let’s get back to building great software!
Triplebyte is a company that connects engineers with top tech companies. We’re running an experiment and our hypothesis is that Software Engineering Daily listeners will do well above average on the quiz. Go to triplebyte.com/sedaily and take the multiple-choice quiz, and in a few episodes we’ll share some stats about how you all did. Try it yourself at triplebyte.com/sedaily.
