When I was running DevOps/SRE consulting I did up to a 2hr deep dive into how to set up a codebase in VCS, establish CI/CD, design deployment, and manage the lifecycle. That gave a lot of freedom to ask specific questions related to the clients I need staff to support and also to get a feel for what people did and did not know. You start with some trivia questions and can organically move around based on the candidate to probe their strengths. That requires having a good interview team; what I’ve found is that if you train your team to interview the way you’d handle a normal design session things work out really well. I’ve got a lot of concrete problems to drop in based on real issues we’ve faced (if you’ve had to support EoL’d Ruby in k8s you can commiserate with a surprising number of folks). The length was dependent on the salary being asked for which determined title. Someone at an architect or manager level needed to know way more than someone just hitting senior status.

Now that I do general engineering stuff I’m trying to figure out something along those lines. I’m currently a fan of code reading some gnarly legacy shit we have to support and either some short data pipelining, API optimization, or component creation depending on what I’m trying to hire for. If you spend an hour with someone trying to code together you can get a decent sense of how they will work day-to-day in terms of how they handle confusion, talking through problems, and basic language knowledge. I try to remove gotchas and keep things in the candidate’s wheelhouse (One time I failed an interview because someone asked me to use a library I’d never seen and I spent an hour flailing because it needed a trailing slash but didn’t have good error messages so I could not figure out why anything wasn’t working. Huge fucking waste of everyone’s time; didn’t evaluate anything other than if I knew that library that wasn’t important.).

I don’t do take homes. I do realize that limits the candidate pool to people that are comfortable collaborating in a potentially stressful situation. I sometimes worry about that. It’s very important for a strong remote team to work well together even when things are awkward or stressful, so there’s a bit of value doing it that way imo.

Frankly I don’t care about interview questions leaking. Coming out of DevOps, it’s very obvious when someone actually understands what’s going on in the interview and when they’re copying and pasting. I’m very happy to have people take code from Stack Overflow during interviews; that’s real fucking life and shows me their research skills. Candidates I will hire can explain or improve the code. I’ve had to reject plenty of sysadmins that do not understand any of what they’re copying and cannot adapt to minor changes that I’ll add just to see how someone handles it. I worked at a company whose questions were on Glassdoor. We had plenty of people that came in prepared for those. We hired several that could roll with the punches (what if you change the inputs, how do you test this case, what’s going to break) and dropped plenty that clearly had no fucking clue.