Introduction

Machine learning is the latest hot technology, but machine intelligence can only be achieved if the models and data sets are accessible — and that requires a lot of data. One way to get machine learning right is with Scikit-Learn, a Python library that makes it easy to work with Machine Learning algorithms. In this blog post, we share our tips for getting started with Scikit-Learn and building intelligent machines.

This article walks you through how to efficiently fit your data using scikit-learn’s grid search implementation and discusses some best practices when using it. The post also covers some specific features when working with classification problems, such as setting up hyperparameters for classification (e.g. C and gamma in Logistic Regression), using metrics to evaluate your model, and converting metrics to class probabilities.

We start with a brief overview of scikit-learn from the perspective of a machine learning beginner and then dive into Grid Search as recommended by the Scikit-Learn documentation. Finally, we conclude by discussing best practices for grid search on classification problems.

Scikit-Learn Overview: Machine Learning for Beginners

If you’re a beginner to machine learning and want to get started you can first check out the best AI Training in Kochi.

Problematic terminology in ML

Machine Learning (ML) is a large field. There are many opinions about what it is, what it’s good for, and how best to use it. In this article we will try to focus on one part of the field – supervised classification – which falls under one of the larger umbrella terms, Machine Learning.

A lot of science has been done on the subject, but there are a lot of opinions as well. What we refer to as “classification” is called many names in ML: classifications, clustering, regression, decision trees, and many more. They all refer to the same thing: finding patterns in data that produce results that we can interpret. For example, predicting whether someone will buy a product using a classifier is called classification.

What is wrong with this “hierarchy of specializations” is that it can make it hard for beginners to understand how machine learning works. We’ll try to avoid the term “classification” as much as possible in this article, and instead use the more general term “prediction”.

Learn to predict

Another way of saying that is: “you can predict things by looking at data”. Machine learning attempts to find patterns in data that you can use to predict future outcomes. In this sense, machine learning is also very similar to statistical modeling over databases, but with a lot of differences. For example, when we do regression over a database, we hope that our model will make it even possible to predict values outside of the chosen range or set. With machine learning you are limited by your data and your knowledge about what your data might signal.

An example:

Suppose you have data that has been modeled with a normal distribution. That means that the data will have a bell curve, which looks like this:

Outliers are regions of your data that are not part of the “normal” distribution, i.e. they lie outside the curve. An anomaly is a statistical property where something unexpected occurs because the data does not conform to expectations. By looking at our normal curve above, you might ask if there are any anomalies in it (hint: one definitely is). In machine learning, an anomaly refers to an example that was not predicted by our algorithm or model.

The figure below contains two anomalies in a data set. Can you spot them?

There are two anomalies. Can you find them? (hint: they are red).

In the previous example, finding anomalies was easy because there is only 1 normal distribution in our data set. If you have more than 1 normal distribution in your data set, it becomes harder to spot their “abnormalities”. That’s why we introduced the term anomaly detection algorithm: it finds that stuff that doesn’t conform to our expectations – and we expect normal distributions.

Classification

As we described above, most classification problems can be broken down into following steps:

Prediction: “what will this data point (or result) look like?” How do we make predictions? We learn from past data and try to predict future results in some way. For example, we might try to predict the price of an Apple product in the future by considering how it is priced now and how it was priced last year and the year before that. If we were really good and had enough historical data, we could even look back at the situation from years before that. This kind of prediction is called regression – but it’s not machine learning. We’ve found the ideal price.

How do you make predictions when you don’t know the data? Sometimes you can look at the distance between data points and apply some function to them to get an idea of their future value. For example, if we want to predict the price of an Apple product in the future, it seems reasonable that we might try to find a data point that was positioned close to our current price (and visa versa). If we were really good and had enough historical data, we could even look back at the situation from years before that. This kind of prediction is called regression – but it’s not machine learning. We’ve found a “good” position for our new item.

Scikit-Learn is one of the most used libraries for machine learning in Python. If you want to know more about Machine Learning and Artificial Intelligence you can check out Artificial Intelligence Training in Kochi.