Train Test Split and Metrics

Train Test Split and Metrics

In Hands-on first Machine Learning Algorithm from Scratch, we trained a linear regression model to predict house prices. We also made a simple 90% / 10% split using NumPy slicing.

Now we are going to slow down and make that idea rigorous.

A model is not useful because it memorizes training rows. It is useful when it performs well on new rows it has not seen before. Train/test splitting and metrics are how we measure that.

By the end of this lesson, you should be able to answer:

• What data should the model learn from?
• What data should be kept hidden until evaluation?
• Which metric should we use for a regression model?
• How do we avoid accidentally cheating during evaluation?