Deep Learning Engineer Interview Questions for AI Training Work

Underfitting shows up as both training and validation loss staying high and close together, while a data quality problem often shows as the model struggling despite having enough capacity and training time. I check the training loss curve first, since a model that can't even fit the training data well points to capacity or optimization issues rather than a data problem.

Normalizing or standardizing inputs is close to non-negotiable, since unscaled features can cause unstable gradients and slow convergence. Beyond that, I check for label quality, class balance, and whether any features leak information about the target, since these issues are much cheaper to catch before training than after a model has already learned to exploit them.

I look at the gap between training and validation performance and how it changes with more data. If more data or better regularization keeps improving validation performance, the current architecture likely has enough capacity. If the model plateaus even with a lot of data and careful tuning, that's a stronger signal the architecture itself is the limiting factor.

Quantization, converting weights to lower precision, is usually the first lever since it often costs little accuracy for a meaningful size and speed improvement. Pruning less important weights and knowledge distillation into a smaller student model are further options when quantization alone doesn't hit the target, though each adds validation work to confirm accuracy holds up.

I use a smaller-scale proxy, like a subset of the data or fewer training epochs, to narrow the hyperparameter search space before committing to expensive full runs on the most promising candidates. Learning rate and batch size tend to matter most, so I prioritize tuning those first rather than searching the full hyperparameter space with equal effort across all of them.

That pattern is a textbook sign of overfitting, so I'd add or strengthen regularization, like dropout or weight decay, and consider early stopping at the point where validation loss was lowest. I'd also check whether the training set is simply too small relative to model capacity, since more data is often the more durable fix than regularization alone.

I'd try a less aggressive compression setting first and measure exactly where the accuracy loss becomes unacceptable, rather than assuming compression and accuracy trade off linearly. If that's not enough, I'd consider a fundamentally smaller architecture trained from the start with the deployment constraint in mind, since architectures designed for efficiency from the outset often compress better than large models forced to shrink after the fact.

I'd suspect the test set doesn't represent real-world input distribution and would sample actual production inputs to compare against the test set statistically. A gap between test set performance and real-world perception almost always traces back to a mismatch between the two distributions, not a flaw in the model that the test set would have caught anyway.

I had normalized a feature using statistics computed on the full dataset including the validation set, which subtly leaked information and inflated validation performance. I caught it when production performance didn't match, recomputed normalization using only training statistics, and the corrected numbers were a more honest, if less impressive, picture of the model's real performance.

A more accurate model exceeded the latency budget for a real-time application, so I chose a smaller architecture with a modest accuracy tradeoff after confirming the accuracy drop didn't cross a threshold that meaningfully affected the end user experience. Meeting the latency requirement reliably mattered more than squeezing out the last few points of accuracy.