Data Scientist Interview Questions for AI Training Work

I run a power analysis based on the minimum effect size that would actually matter for the business decision, the expected baseline variance, and the desired statistical power, typically 80%. Skipping this step is how teams end up running underpowered experiments that can't reliably detect real effects, then wrongly concluding a change had no impact.

Correlation just means two variables move together, which can be driven by a third confounding factor rather than either variable causing the other. With observational data, I look for natural experiments, instrumental variables, or matching techniques to approximate a controlled comparison, since a true randomized experiment usually isn't available and correlation alone can't support a causal claim.

For data with clear seasonal and trend patterns and limited history, a statistical model like ARIMA or exponential smoothing is often more reliable than machine learning, which typically needs more data to outperform simpler methods. I'd benchmark both against a holdout period rather than assuming the more complex option is automatically better.

I'd use an inherently interpretable model where possible, or apply a post-hoc explanation method like SHAP values on top of a more complex model if the added performance is worth the extra explanation layer. The explanation needs to be accurate to how the model actually makes decisions, not just a plausible-sounding story layered on afterward.

I'd use a line chart with a shaded confidence interval band around the point forecast, since showing the range of plausible outcomes prevents stakeholders from over-trusting a single point estimate. A forecast presented as one precise number tends to get treated as certain, which misrepresents what the model actually knows.

I'd present the tradeoff explicitly: run for two weeks and accept a much wider margin of error, extend the timeline, or narrow the test to a larger effect size that would still be detectable in two weeks. Running the test anyway without flagging the power issue risks a false negative getting treated as a confirmed result.

I'd flag it immediately rather than assuming it's fine because the model happens to still run without errors. A model validated for one use case can perform poorly in a different context even if nothing about the code changed, and using it without re-validation risks decisions being made on predictions that were never actually tested for that purpose.

I'd push for a more specific, decision-oriented question before starting any analysis, since 'understand customers better' isn't answerable in a way that leads to action. I'd ask what decision this analysis is meant to inform, and work backward from there to figure out what data and methods actually address that decision.

An analysis showed a popular feature wasn't actually driving the retention improvement the team believed it was, once other factors were controlled for. I presented the analysis with the methodology clearly laid out rather than softening the conclusion, since the team needed the accurate picture to make good decisions about where to invest next, even though it wasn't the answer they expected.

Instead of describing model architecture, I framed the explanation around what inputs mattered most to the prediction and what that meant for the business, using a concrete example prediction rather than abstract terminology. Leadership needed to trust the output enough to act on it, which didn't require them to understand the mechanics underneath.