The goal of this day is to learn to use Keras to build Neural Networks and train them on small data sets.
classification & regression
The goal of this day is to learn to use Keras to build Neural Networks and train them on small data sets.
classification & regression
'2.4.3'
## Historical
## Rules
The correction will provide the code and output because it is not straightforward to reproduce results using Keras. There are many source of randomness. Even if all the seeds are fixed to a constant they may be other source of randomness. https://machinelearningmastery.com/reproducible-results-neural-networks-keras/
The goal of this exercise is to learn to set up the optimization for a regression neural network. There's no code to run in that exercise. In W2D2E3, we implemented a neural network designed for regression. We will be using this neural network:
The goal of this exercise is to learn to set up the optimization for a regression neural network. There's no code to run in that exercise. In W2D2E3, we implemented a neural network designed for regression. We will be using this neural network:
As a reminder, the main difference between a regression and classification neural network's architecture is the output layer activation function.
1. Fill this chunk of code to set up the optimization part of the regression neural network:
1. Fill this chunk of code to set up the optimization part of the regression neural network:
```
```python
model.compile(
optimizer='adam',
loss='',#TODO1
metrics=[''] #TODO2
)
```
Hint:
- Mean Squared Error (MSE) and Mean Absolute Error (MAE) are common loss functions used for regression problems. Mean Absolute Error is less sensitive to outliers. Different loss functions are used for classification problems. Similarly, evaluation metrics used for regression differ from classification.
Hint:
- Mean Squared Error (MSE) and Mean Absolute Error (MAE) are common loss functions used for regression problems. Mean Absolute Error is less sensitive to outliers. Different loss functions are used for classification problems. Similarly, evaluation metrics used for regression differ from classification.
https://keras.io/api/metrics/regression_metrics/
## Correction:
## Correction
1. This question is validated if the chunk of code is:
1. This question is validated if the chunk of code is:
```
```python
model.compile(
optimizer='adam',
loss='mse',
metrics=['mse']
)
```
All regression metrics or losses used are correct. As explained before, the loss functions are chosen thanks to nice mathematical properties. That is why most of the time the loss function used for regression is the MSE or MAE.
https://keras.io/api/losses/regression_losses/
https://keras.io/api/metrics/regression_metrics/
All regression metrics or losses used are correct. As explained before, the loss functions are chosen thanks to nice mathematical properties. That is why most of the time the loss function used for regression is the MSE or MAE.
The goal of this exercise is to learn to train a neural network to perform a regression on a data set.
The data set is Auto MPG Dataset and the go is to build a model to predict the fuel efficiency of late-1970s and early 1980s automobiles. To do this, provide the model with a description of many automobiles from that time period. This description includes attributes like: cylinders, displacement, horsepower, and weight.
The data set is Auto MPG Dataset and the goal is to build a model to predict the fuel efficiency of late-1970s and early 1980s automobiles. To do this, provide the model with a description of many automobiles from that time period. This description includes attributes like: cylinders, displacement, horsepower, and weight.
- Drop the columns: **model year**, **origin**, **car name**
- Split train test without shuffling the data. Keep 20% for the test set.
- Scale the data using Standard Scaler
- Drop the columns: **model year**, **origin**, **car name**
- Split train test without shuffling the data. Keep 20% for the test set.
- Scale the data using Standard Scaler
2. Train a neural network on the train set and predict on the test set. The neural network should have 2 hidden layers and the loss should be **mean_squared_error**. The expected **mean absolute error** on the test set is maximum 10.
*Hint*: inscrease the number of epochs
**Warning**: Do no forget to evaluate the neural network on the **SCALED** test set.
*Hint*: increase the number of epochs
**Warning**: Do no forget to evaluate the neural network on the **SCALED** test set.
## Correction
## Correction
1. This question is validated if the input DataFrames are:
1. This question is validated if the input DataFrames are:
X_train_scaled shape is (313, 5) and the first 5 rows are:
@ -102,7 +102,7 @@ X_train_scaled shape is (313, 5) and the first 5 rows are:
The output neuron has to be `Dense(1)` - by defaut the activation funtion is linear. The loss has to be **mean_squared_error** and the **input_dim** has to be **5**. All variations on the others parameters are accepted.
The output neuron has to be `Dense(1)` - by default the activation function is linear. The loss has to be **mean_squared_error** and the **input_dim** has to be **5**. All variations on the others parameters are accepted.
*Hint*: To get the score on the test set, `evaluate` could have been used: `model.evaluate(X_test_scaled, y_test)`.
*Hint*: To get the score on the test set, `evaluate` could have been used: `model.evaluate(X_test_scaled, y_test)`.
# Exercise 3 Multi classification - Softmax
The goal of this exercise is to learn to a neural network architecture for multi-class data. This is an important type of problem on which to practice with neural networks because the three class values require specialized handling. A multi-classification neural network uses as output layer a **softmax** layer. The **softmax** activation function is an extension of the sigmoid as it is designed to output the probabilities to belong to each class in a multi-class problem. This output layer has to contain as much neurons as classes in the multi-classification problem. This article explains in detail how it works. https://developers.google.com/machine-learning/crash-course/multi-class-neural-networks/softmax
The goal of this exercise is to learn to a neural network architecture for multi-class data. This is an important type of problem on which to practice with neural networks because the three class values require specialized handling. A multi-classification neural network uses as output layer a **softmax** layer. The **softmax** activation function is an extension of the sigmoid as it is designed to output the probabilities to belong to each class in a multi-class problem. This output layer has to contain as much neurons as classes in the multi-classification problem. This article explains in detail how it works.
Let us assume we want to classify images and we know they contain either apples, bears, candies, eggs or dogs (extension of the example in the link above).
1. Create the architecture for a multi-class neural network with the following architecture and return `print(model.summary())`:
Let us assume we want to classify images and we know they contain either apples, bears, candies, eggs or dogs (extension of the example in the link above).
- 5 inputs variables
- hidden layer 1: 16 neurons and sigmoid as activation function
- hidden layer 2: 8 neurons and sigmoid as activation function
- output layer: The number of neurons and the activation function should be adapted to this multi-classification problem.
1. Create the architecture for a multi-class neural network with the following architecture and return `print(model.summary())`:
- 5 inputs variables
- hidden layer 1: 16 neurons and sigmoid as activation function
- hidden layer 2: 8 neurons and sigmoid as activation function
- output layer: The number of neurons and the activation function should be adapted to this multi-classification problem.
## Correction
## Correction
1. This question is validated if the code that creates the neural network is:
1. This question is validated if the code that creates the neural network is:
The goal of this exercise is to learn to optimize a multi-classification neural network. As learnt previously, the loss function used in binary classification is the log loss - also called in Keras `binary_crossentropy`. This function is defined for binary classification and can be extended to multi-classfication. In Keras, the extended loss that supports multi-classification is `binary_crossentropy`. There's no code to run in that exercise.
1. Fill the chunk of code below in order to optimize the neural network defined in the previous exercise. Choose the adapted loss, adam as optimizer and the accuracy as metric.
```
```python
model.compile(loss='',#TODO1
optimizer='', #TODO2
metrics=['']) #TODO3
```
## Correction
## Correction
1. This question is validated if the chunk of code is:
The goal of this exercise is to learn to use a neural network to classify a multiclass data set. The data set used is the Iris data set which allows to classify flower given basic features as flower's measurement.
The goal of this exercise is to learn to use a neural network to classify a multiclass data set. The data set used is the Iris data set which allows to classify flower given basic features as flower's measurement.
Preliminary:
- Split train test. Keep 20% for the test set. Use `random_state=1`.
- Scale the data using Standard Scaler
Preliminary:
- Split train test. Keep 20% for the test set. Use `random_state=1`.
- Scale the data using Standard Scaler
1. Use the `LabelBinarizer` from Sckit-learn to create a one hot encoding of the target. As you know, the output layer of a multi-classification neural network shape is equal to the number of classes. The output layer expects to have a target with the same shape as its output layer.
1. Use the `LabelBinarizer` from Sckit-learn to create a one hot encoding of the target. As you know, the output layer of a multi-classification neural network shape is equal to the number of classes. The output layer expects to have a target with the same shape as its output layer.
2. Train a neural network on the train set and predict on the test set. The neural network should have 1 hidden layers. The expected **accuracy** on the test set is minimum 90%.
*Hint*: inscrease the number of epochs
**Warning**: Do no forget to evaluate the neural network on the **SCALED** test set.
*Hint*: increase the number of epochs
**Warning**: Do no forget to evaluate the neural network on the **SCALED** test set.
## Correction
## Correction
1. This question is validated if the output of the first ten values of the train labels are:
1. This question is validated if the output of the first ten values of the train labels are:
```
```console
array([[0, 1, 0],
[0, 0, 1],
[0, 1, 0],
@ -231,20 +232,13 @@ array([[0, 1, 0],
2. This question is validated if the accuracy on the test set is bigger than 90%. To evaluate the accuracy on the test set you can use: `model.evaluate(X_test_sc, y_test_multi_class)`.
Here is an implementation that gives 96% accuracy on the test set.
Here is an implementation that gives 96% accuracy on the test set.
brad-gh
3 years ago
committed by