feat: structure week3 day1,2 and 3

one_exercise_per_file/week03/day01/ex01/audit/readme.md

@@ -0,0 +1,8 @@
+1. This question is validated if this code:
+
+    ```
+    neuron = Neuron(0,1,4)
+    neuron.feedforward(2,3)
+    ```
+
+ returns **0.9990889488055994**.

one_exercise_per_file/week03/day01/ex01/readme.md

@@ -0,0 +1,52 @@
+# Exercise 1 The neuron 
+
+The goal of this exercise is to understand the role of a neuron and to implement a neuron. 
+
+An artificial neuron, the basic unit of the neural network, (also referred to as a perceptron) is a mathematical function. It takes one or more inputs that are multiplied by values called “weights” and added together. This value is then passed to a non-linear function, known as an activation function, to become the neuron’s output.
+
+As desbribed in the article, **a neuron takes inputs, does some math with them, and produces one output**.
+
+Let us assume there are 2 inputs. Here are the three steps involved in the neuron: 
+
+1. Each input is multiplied by a weight
+    - x1 -> x1 * w1
+    - x2 -> x2 * w2
+2. The weighted inputs are added together with a biais b
+    - (x1 * w1) + (x2 * w2) + b
+3. The sum is passed through an activation function
+    - y = f((x1 * w1) + (x2 * w2) + b)
+
+    - The activation function is a function you know from W2DAY2 (Logistic Regression): **the sigmoid**
+
+Example: 
+
+x1 = 2 , x2 = 3 , w1 = 0, w2= 1, b = 4
+
+1. Step 1: Multiply by a weight
+    - x1 -> 2 * 0 = 0 
+    - x2 -> 3 * 1 = 3 
+2. Step 2: Add weigthed inputs and bias
+    - 0 + 3 + 4 = 7 
+3. Step 3: Activation function
+    - y = f(7) = 0.999
+---
+1. Implement a the function feedforward of the class `Neuron` that takes as input the inputs (x1, x2) and that uses the attributes: the weights and the biais to return y: 
+
+
+    ```
+    class Neuron:
+    def __init__(self, weight1, weight2, bias):
+        self.weights_1 = weight1
+        self.weights_2 = weight2
+        self.bias = bias
+
+    def feedforward(self, x1, x2):
+        #TODO
+        return y
+
+
+    ```
+
+Note: if you are confortable with matrix multiplication, feel free to vectorize the operations as done in the article. 
+
+https://victorzhou.com/blog/intro-to-neural-networks/

one_exercise_per_file/week03/day01/ex02/audit/readme.md

@@ -0,0 +1 @@
+1. This question is validated the output is: **0.9524917424084265**

one_exercise_per_file/week03/day01/ex02/readme.md

@@ -0,0 +1,47 @@
+# Exerice 2 Neural network
+
+The goal of this exercise is to understand how to combine three neurons to form a neural network. A neural newtwork is nothing else than neurons connected together. As shown in the figure the neural network is composed of **layers**:
+
+  - Input layer: it only represents input data. **It doesn't contain neurons**.
+  - Output layer: it represents the last layer. It contains a neuron (in some cases more than 1).
+  - Hidden layer: any layer between the input (first) layer and output (last) layer. Many hidden layers can be stacked. When there are many hidden layers, the neural networks is deep.
+
+Notice that the neuron **o1** in the output layer takes as input the output of the neurons **h1** and **h2** in the hidden layer. 
+
+In exercise 1, you implemented this neuron.
+![alt text][neuron]
+
+[neuron]: images/w3_day1_neuron.png "Plot"
+
+Now, we add two more neurons: 
+
+- h2, the second neuron of the hidden layer
+- o1, the neuron of the output layer
+
+
+![alt text][nn]
+
+[nn]: images/w3_day1_neural_network.png "Plot"
+
+1. Implement the function `feedforward` of the class `OurNeuralNetwork` that takes as input the input data and returns the output y. Return the output for these neurons:
+
+    ```
+    neuron_h1 = Neuron(1,2,-1)
+    neuron_h2 = Neuron(0.5,1,0)
+    neuron_o1 = Neuron(2,0,1)
+    ```
+
+    ```
+    class OurNeuralNetwork:
+        
+        def __init__(self, neuron_h1, neuron_h2, neuron_o1):
+            self.h1 = neuron_h1
+            self.h2 = neuron_h2
+            self.o1 = neuron_o1
+
+        def feedforward(self, x1, x2):
+        # The inputs for o1 are the outputs from h1 and h2
+        # TODO
+            return y
+
+    ```

one_exercise_per_file/week03/day01/ex03/audit/readme.md

@@ -0,0 +1,2 @@
+1. This question is validated if the output is: **0.5472899351247816**.
+

one_exercise_per_file/week03/day01/ex03/readme.md

@@ -0,0 +1,14 @@
+# Exercise 3 Log loss
+
+The goal of this exercise is to implement the Log loss function. As mentioned last week, this function is used in classification as a **loss function**. It means that the better the classifier is, the smaller the loss function is. W2D1, you implemented the gradient descent on the MSE loss to update the weights of the linear regression. Similarly, the minimization of the Log loss leads to finding optimal weights. 
+
+Log loss: - 1/n * Sum[(y_true*log(y_pred) + (1-y_true)*log(1-y_pred))]
+
+1. Create a function `log_loss_custom` and compute the loss for the data below: 
+
+    ```
+    y_true = np.array([0,1,1,0,1])
+    y_pred = np.array([0.1,0.8,0.6, 0.5, 0.3])
+    ```
+    Check that `log_loss` from `sklearn.metrics` returns the same result
+https://scikit-learn.org/stable/modules/generated/sklearn.metrics.log_loss.html

one_exercise_per_file/week03/day01/ex04/audit/readme.md

@@ -0,0 +1,8 @@
+1. This question is validated if the output is: 
+    ```
+    Bob: 0.7855253278357536
+    Eli: 0.7771516558846259
+    Tom: 0.8067873659804015
+    Ryan: 0.7892343955586032
+    ```
+2. This question is validated if the logloss for the 4 students is **0.5485133607757963**.

one_exercise_per_file/week03/day01/ex04/readme.md

@@ -0,0 +1,22 @@
+# Exercise 4 Forward propagation
+The goal of this exerice is to compute the log loss on the output of the forward propagation. The data used is the tiny data set below. 
+
+
+| name   |   math |   chemistry |   exam_success |
+|:-------|-------:|------------:|---------------:|
+| Bob    |     12 |          15 |              1 |
+| Eli    |     10 |           9 |              0 |
+| Tom    |     18 |          18 |              1 |
+| Ryan   |     13 |          14 |              1 |
+
+
+The goal if the network is to predict the success at the exam given math and chemistry grades. The inputs are `math` and `chemistry` and the target is `exam_sucess`.
+
+1. Compute and return the output of the neural network for each of the students. Here are the weights and biases of the neural network: 
+
+    ```
+    neuron_h1 = Neuron(0.05, 0.001, 0)
+    neuron_h2 = Neuron(0.02, 0.003, 0)
+    neuron_o1 = Neuron(2,0,0)
+    ```
+2. Compute the logloss for the data given the output of the neural network with the 4 students. 

one_exercise_per_file/week03/day01/ex05/audit/readme.md

@@ -0,0 +1,12 @@
+1. This question is validated if the output is **7**.
+
+2. This question is validated if the outputs are:
+
+    ```
+    Bob: 14.918863163724454
+    Eli: 14.83137890625537
+    Tom: 15.086662606964074
+    Ryan: 14.939270885974128
+    ```
+
+3. This question is validated if the MSE is **10.237608699909138**

one_exercise_per_file/week03/day01/ex05/readme.md

@@ -0,0 +1,64 @@
+# Exercise 5 Regression 
+
+The goal of this exercise is to learn to adapt the output layer to regression. 
+As a reminder, one of reasons for which the sigmoid is used in classification is because it contracts the output between 0 and 1 which is the expected output range for a probability (W2D2: Logistic regression). However, the output of the regression is not a probability. 
+
+In order to perform a regression using a neural network, the activation function of the neuron on the output layer has to be modified to **identity function**. In mathematics, the identity function is: **f(x) = x**. In other words it means that it returns the input as so. The three steps become: 
+
+
+1. Each input is multiplied by a weight
+    - x1 -> x1 * w1
+    - x2 -> x2 * w2
+2. The weighted inputs are added together with a biais b
+    - (x1 * w1) + (x2 * w2) + b
+3. The sum is passed through an activation function
+    - y = f((x1 * w1) + (x2 * w2) + b)
+    - The activation function is **the identity**
+    - y = (x1 * w1) + (x2 * w2) + b
+
+All other neurons' activation function **doesn't change**.
+
+1. Adapt the neuron class implemented in exercise 1. It now takes as a parameter `regression` which is boolean. When its value is `True`, `feedforward` should use the identity function as activation function instead of the sigmoid function. 
+
+
+    ```
+    class Neuron:
+    def __init__(self, weight1, weight2, bias, regression):
+        self.weights_1 = weight1
+        self.weights_2 = weight2
+        self.bias = bias
+        #TODO
+
+    def feedforward(self, x1, x2):
+        #TODO
+        return y
+
+    ```
+
+    - Compute the output for:
+
+        ```
+        neuron = Neuron(0,1,4, True)
+        neuron.feedforward(2,3)
+        ```
+
+
+2. Now, the goal of the network is to predict the physics' grade at the exam given math and chemistry grades. The inputs are `math` and `chemistry` and the target is `physics`.
+
+| name   |   math |   chemistry |   physics |
+|:-------|-------:|------------:|---------------:|
+| Bob    |     12 |          15 |              16 |
+| Eli    |     10 |           9 |              10 |
+| Tom    |     18 |          18 |              19 |
+| Ryan   |     13 |          14 |              16 |
+
+
+Compute and return the output of the neural network for each of the students. Here are the weights and biases of the neural network: 
+
+    ```
+    #replace regression by the right value
+    neuron_h1 = Neuron(0.05, 0.001, 0, regression)
+    neuron_h2 = Neuron(0.002, 0.003, 0, regression)
+    neuron_o1 = Neuron(2,7,10, regression)
+    ```
+3. Compute the MSE for the 4 students. 

one_exercise_per_file/week03/day01/readme.md

@@ -0,0 +1,22 @@
+# W3D1  Piscine AI - Data Science 
+
+
+# Table of Contents:
+
+
+# Introduction
+
+Deep learning is a huge domain. We will focus on Artificial Neural Networks. The goal is to understand how do the neural networks train and train them on data. Understand the challenges of training a neural network
+Architectures as RNN, LSTM (learn sequences, used in TS and NLP),CNN used a lot in image processing are well know algorithms in deep learning but won't be covered by the AI branch. Once you have a good understanding of ANN feel free to extend your knowledge to new architectures. 
+
+
+## Rules
+
+## Ressources 
+https://victorzhou.com/blog/intro-to-neural-networks/
+
+
+https://srnghn.medium.com/deep-learning-overview-of-neurons-and-activation-functions-1d98286cf1e4#:~:text=What%20is%20a%20neuron%3F,to%20become%20the%20neuron's%20output.
+
+Reproduire cet article sans back prop 
+https://towardsdatascience.com/machine-learning-for-beginners-an-introduction-to-neural-networks-d49f22d238f9

one_exercise_per_file/week03/day02/ex01/audit/readme.md

@@ -0,0 +1 @@
+1. This question is validated if the output is: `<tensorflow.python.keras.engine.sequential.Sequential object at  xxx`

one_exercise_per_file/week03/day02/ex01/readme.md

@@ -0,0 +1,5 @@
+# Exercise 1 Sequential
+
+The goal of this exercise is to learn to call the object `Sequential`. 
+
+1. Put the object Sequential in a variable named `model` and print the variable `model`.

one_exercise_per_file/week03/day02/ex02/audit/readme.md

@@ -0,0 +1,56 @@
+1. This question is validated if the fields`batch_input_shape`,`units` and `activation` match this output: 
+
+    ```
+    {'name': 'dense_7',
+    'trainable': True,
+    'batch_input_shape': (None, 5),
+    'dtype': 'float32',
+    'units': 8,
+    'activation': 'sigmoid',
+    'use_bias': True,
+    'kernel_initializer': {'class_name': 'GlorotUniform',
+    'config': {'seed': None}},
+    'bias_initializer': {'class_name': 'Zeros', 'config': {}},
+    'kernel_regularizer': None,
+    'bias_regularizer': None,
+    'activity_regularizer': None,
+    'kernel_constraint': None,
+    'bias_constraint': None}
+    ```
+
+2. This question is validated if the fields`units` and `activation` match this output: 
+
+    ```
+    {'name': 'dense_8',
+    'trainable': True,
+    'dtype': 'float32',
+    'units': 4,
+    'activation': 'sigmoid',
+    'use_bias': True,
+    'kernel_initializer': {'class_name': 'GlorotUniform',
+    'config': {'seed': None}},
+    'bias_initializer': {'class_name': 'Zeros', 'config': {}},
+    'kernel_regularizer': None,
+    'bias_regularizer': None,
+    'activity_regularizer': None,
+    'kernel_constraint': None,
+    'bias_constraint': None}
+    ```
+3. This question is validated if the fields`units` and `activation` match this output: 
+
+    ```
+    {'name': 'dense_9',
+    'trainable': True,
+    'dtype': 'float32',
+    'units': 1,
+    'activation': 'sigmoid',
+    'use_bias': True,
+    'kernel_initializer': {'class_name': 'GlorotUniform',
+    'config': {'seed': None}},
+    'bias_initializer': {'class_name': 'Zeros', 'config': {}},
+    'kernel_regularizer': None,
+    'bias_regularizer': None,
+    'activity_regularizer': None,
+    'kernel_constraint': None,
+    'bias_constraint': None}
+    ```

one_exercise_per_file/week03/day02/ex02/readme.md

@@ -0,0 +1,22 @@
+# Exercise 2 Dense
+
+The goal of this exercise is to learn to create layers of neurons. Keras proposes options to create custom layers. The neural networks build in these exercises do not require custom layers. `Dense` layers do the job. A dense layer is simply a layer where each unit or neuron is connected to each neuron in the next layer. As seen yesterday, there are three main types of layers: input, hidden and output. The **input layer** that specifies the number of inputs (features) is not represented as a layer in Keras. However, `Dense` has a parameter `input_dim` that gives the number of inputs in the previous layer. The output layer as any hidden layer can be created using `Dense`, the only difference is that the output layer contains one single neuron. 
+
+1. Create a `Dense` layer with these parameters and return the output of `get_config`: 
+
+    - First hidden layer connected to 5 input variables. 
+    - 8 neurons 
+    - sigmoid as activation function 
+
+
+2. Create a `Dense` layer with these parameters and return the output of `get_config`: 
+
+    - Hidden layer (not the first one)
+    - 4 neurons 
+    - sigmoid as activation function 
+
+3. Create a `Dense` layer with these parameters and return the output of `get_config`: 
+
+    - Output layer
+    - 1 neuron 
+    - sigmoid as activation function 

one_exercise_per_file/week03/day02/ex03/audit/readme.md

@@ -0,0 +1,11 @@
+1. This question is validated if the code that creates the neural network is: 
+
+    ```
+    model = keras.Sequential()
+    model.add(Dense(8, input_shape=(5,), activation= 'sigmoid'))
+    model.add(Dense(4, activation= 'sigmoid'))
+    model.add(Dense(1, activation= 'linear'))
+
+    ```
+
+The first two layers could use another activation function that sigmoid (eg: relu)

one_exercise_per_file/week03/day02/ex03/readme.md

@@ -0,0 +1,10 @@
+# Exercise 3 Architecture
+
+The goal of this exercise is to combine the layers and to create a neural network. 
+
+1. Create a neural network for regression with the following architecture and return `print(model.summary())`: 
+
+    - 5 inputs variables
+    - hidden layer 1: 8 neurons and sigmoid as activation function
+    - hidden layer 2: 4 neurons and sigmoid as activation function
+    - output layer: 1 neuron. Find the adapted activation function

one_exercise_per_file/week03/day02/ex04/audit/readme.md

@@ -0,0 +1,62 @@
+1. This question is validated if the output of `model.get_config()['layers']` matches the fields `batch_input_shape`, `units` and `activation`.
+
+```
+[{'class_name': 'InputLayer',
+  'config': {'batch_input_shape': (None, 30),
+   'dtype': 'float32',
+   'sparse': False,
+   'ragged': False,
+   'name': 'dense_134_input'}},
+ {'class_name': 'Dense',
+  'config': {'name': 'dense_134',
+   'trainable': True,
+   'batch_input_shape': (None, 30),
+   'dtype': 'float32',
+   'units': 10,
+   'activation': 'sigmoid',
+   'use_bias': True,
+   'kernel_initializer': {'class_name': 'GlorotUniform',
+    'config': {'seed': None}},
+   'bias_initializer': {'class_name': 'Zeros', 'config': {}},
+   'kernel_regularizer': None,
+   'bias_regularizer': None,
+   'activity_regularizer': None,
+   'kernel_constraint': None,
+   'bias_constraint': None}},
+ {'class_name': 'Dense',
+  'config': {'name': 'dense_135',
+   'trainable': True,
+   'dtype': 'float32',
+   'units': 5,
+   'activation': 'sigmoid',
+   'use_bias': True,
+   'kernel_initializer': {'class_name': 'GlorotUniform',
+    'config': {'seed': None}},
+   'bias_initializer': {'class_name': 'Zeros', 'config': {}},
+   'kernel_regularizer': None,
+   'bias_regularizer': None,
+   'activity_regularizer': None,
+   'kernel_constraint': None,
+   'bias_constraint': None}},
+ {'class_name': 'Dense',
+  'config': {'name': 'dense_136',
+   'trainable': True,
+   'dtype': 'float32',
+   'units': 1,
+   'activation': 'sigmoid',
+   'use_bias': True,
+   'kernel_initializer': {'class_name': 'GlorotUniform',
+    'config': {'seed': None}},
+   'bias_initializer': {'class_name': 'Zeros', 'config': {}},
+   'kernel_regularizer': None,
+   'bias_regularizer': None,
+   'activity_regularizer': None,
+   'kernel_constraint': None,
+   'bias_constraint': None}}]
+```
+You should notice that the neural network is struggling to learn. By luck the initialization of the weights might have led to an accuracy close of 90%. But when I trained the neural network, with `batch_size=300` on the data here is the ouptput of the last epoch (50):
+
+`Epoch 50/50
+2/2 [==============================] - 0s 1ms/step - loss: 0.6559 - accuracy: 0.6274`
+
+2. This solution is validated if the the accuracy at epoch 50 is higher than 95%.

one_exercise_per_file/week03/day02/ex04/readme.md

@@ -0,0 +1,28 @@
+# Exercise 4 Optimize
+
+The goal of this exercise is to learn to train the neural network. Once the architecture of the neural network is set there are two steps to train the neural network: 
+
+- `compile`:  The compilation step aims to set the loss function, to choose the algoithm to minimize the chosen loss function and to choose the metric the model outputs.
+
+  - The **optimizer**. We’ll stick with a pretty good default: the Adam gradient-based optimizer. Keras has many other optimizers you can look into as well.
+  - The **loss function**. Depending on the problem to solve: classification or regression Keras proposes different loss functions. In classification Keras distinguishes between `binary_crossentropy` (2 classes) and `categorical_crossentropy` (>2 classes), so we’ll use the latter. 
+  - The **metric(s)**. A list of metrics. Depending on the problem to solve: classification or regression Keras proposes different loss functions. For example for classification the metric can be the accuracy. 
+
+
+- `fit`: Training a model in Keras literally consists only of calling fit() and specifying some parameters. There are a lot of possible parameters, but we’ll only manually supply a few:
+  - The **training data**, commonly known as X and Y, respectively.
+  - The **number of epochs** (iterations over the entire dataset) to train for.   
+  - The **batch size** (number of samples per gradient update) to use when training.
+
+  This article gives more details about **epoch** and **batch size**: https://machinelearningmastery.com/difference-between-a-batch-and-an-epoch/
+
+1. Create the following neural network (classification): 
+    - Set the right number of inputs variables
+    - hidden layer 1: 10 neurons and sigmoid as activation function.
+    - hidden layer 2: 5 neurons and sigmoid as activation function.
+    - output layer: 1 neuron and sigmoid as activation function.
+    - Choose the accuracy metric, the adam optimizer, the adapted loss and epoch smaller than 50. 
+
+    Import the breast cancer data set from `sklearn.datasets` using `load_breast_cancer` and train the neural network on the data set.  
+
+2. Scale the data using `StandardScaler` from `sklearn.preprocessing`. Train the neural network again. 

one_exercise_per_file/week03/day02/readme.md

@@ -0,0 +1,26 @@
+# W3D2  Piscine AI - Data Science 
+
+
+# Table of Contents:
+
+
+# Introduction
+Keras backend TF
+The goal of this day is to learn to use Keras to build Neural Networks. 
+
+There are two ways to build Keras models: sequential and functional.
+
+The sequential API allows you to create models layer-by-layer for most problems. It is limited in that it does not allow you to create models that share layers or have multiple inputs or outputs. The exercises focuses on the usage of the sequential API. 
+
+'2.4.3'
+
+## Historical
+
+
+
+## Rules
+
+The correction will provide the code and output because it is not straightforward to reproduce results using Keras. There are many source of randomness. Even if all the seeds are fixed to a constant they may be other source of randomness. https://machinelearningmastery.com/reproducible-results-neural-networks-keras/
+A developper
+## Ressources 
+https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/

one_exercise_per_file/week03/day03/ex01/audit/readme.md

@@ -0,0 +1,13 @@
+1. This question is validated if the chunk of code is: 
+
+```
+model.compile(
+  optimizer='adam',
+  loss='mse',
+  metrics=['mse'] 
+)
+```
+All regression metrics or losses used are correct. As explained before, the loss functions are chosen thanks to nice mathematical properties. That is why most of the time the loss function used for regression is the MSE or MAE. 
+
+https://keras.io/api/losses/regression_losses/
+https://keras.io/api/metrics/regression_metrics/

one_exercise_per_file/week03/day03/ex01/readme.md

@@ -0,0 +1,26 @@
+# Exercise 1 Regression - Optimize 
+
+The goal of this exercise is to learn to set up the optimization for a regression neural network. There's no code to run in that exercise. In W2D2E3, we implemented a neural network designed for regression. We will be using this neural network: 
+
+    ```
+    model = keras.Sequential()
+    model.add(Dense(8, input_shape=(5,), activation= 'sigmoid'))
+    model.add(Dense(4, activation= 'sigmoid'))
+    model.add(Dense(1, activation= 'linear'))
+
+    ```
+As a reminder, the main difference between a regression and classification neural network's architecture is the output layer activation function.
+
+1. Fill this chunk of code to set up the optimization part of the regression neural network: 
+
+```
+model.compile(
+  optimizer='adam',
+  loss='',#TODO1
+  metrics=[''] #TODO2
+)
+```
+Hint: 
+- Mean Squared Error (MSE) and Mean Absolute Error (MAE) are common loss functions used for regression problems. Mean Absolute Error is less sensitive to outliers. Different loss functions are used for classification problems. Similarly, evaluation metrics used for regression differ from classification. 
+
+https://keras.io/api/metrics/regression_metrics/

one_exercise_per_file/week03/day03/ex02/audit/readme.md

@@ -0,0 +1,59 @@
+1. This question is validated if the input DataFrames are: 
+
+X_train_scaled shape is (313, 5) and the first 5 rows are:
+
+|    |   cylinders |   displacement |   horsepower |   weight |   acceleration |
+|---:|------------:|---------------:|-------------:|---------:|---------------:|
+|  0 |     1.28377 |       0.884666 |     0.48697  | 0.455708 |       -1.19481 |
+|  1 |     1.28377 |       1.28127  |     1.36238  | 0.670459 |       -1.37737 |
+|  2 |     1.28377 |       0.986124 |     0.987205 | 0.378443 |       -1.55992 |
+|  3 |     1.28377 |       0.856996 |     0.987205 | 0.375034 |       -1.19481 |
+|  4 |     1.28377 |       0.838549 |     0.737087 | 0.393214 |       -1.74247 |
+
+The train target is: 
+
+|    |   mpg |
+|---:|------:|
+|  0 |    18 |
+|  1 |    15 |
+|  2 |    18 |
+|  3 |    16 |
+|  4 |    17 |
+
+
+X_test_scaled shape is (79, 5) and the first 5 rows are:
+
+|     |   cylinders |   displacement |   horsepower |    weight |   acceleration |
+|----:|------------:|---------------:|-------------:|----------:|---------------:|
+| 315 |   -1.00255  |      -0.554185 |    -0.5135   | -0.113552 |      1.76253   |
+| 316 |    0.140612 |       0.128347 |    -0.5135   |  0.31595  |      1.25139   |
+| 317 |   -1.00255  |      -1.05225  |    -0.813641 | -1.03959  |      0.192584  |
+| 318 |   -1.00255  |      -0.710983 |    -0.5135   | -0.445337 |      0.0830525 |
+| 319 |   -1.00255  |      -0.840111 |    -0.888676 | -0.637363 |      0.813262  |
+
+The test target is: 
+
+|     |   mpg |
+|----:|------:|
+| 315 |  24.3 |
+| 316 |  19.1 |
+| 317 |  34.3 |
+| 318 |  29.8 |
+| 319 |  31.3 |
+
+2. This question is validated if the mean absolute error on the test set is smaller than 10. Here is an architecture that works: 
+
+```
+# create model
+model = Sequential()
+model.add(Dense(30, input_dim=5, activation='sigmoid'))
+model.add(Dense(30, activation='sigmoid'))
+model.add(Dense(1))
+# Compile model
+model.compile(loss='mean_squared_error',
+                optimizer='adam', metrics='mean_absolute_error')
+```
+
+The output neuron has to be `Dense(1)` - by defaut the activation funtion is linear. The loss has to be **mean_squared_error** and the **input_dim** has to be **5**. All variations on the others parameters are accepted. 
+
+*Hint*: To get the score on the test set, `evaluate` could have been used: `model.evaluate(X_test_scaled, y_test)`. 

one_exercise_per_file/week03/day03/ex02/readme.md

@@ -0,0 +1,17 @@
+# Exercise 2 Regression example
+
+The goal of this exercise is to learn to train a neural network to perform a regression on a data set.
+The data set is Auto MPG Dataset and the go is to build a model to predict the fuel efficiency of late-1970s and early 1980s automobiles. To do this, provide the model with a description of many automobiles from that time period. This description includes attributes like: cylinders, displacement, horsepower, and weight.
+
+https://www.tensorflow.org/tutorials/keras/regression
+
+
+1. Preprocess the data set as follow:
+    - Drop the columns: **model year**, **origin**, **car name**
+    - Split train test without shuffling the data. Keep 20% for the test set.
+    - Scale the data using Standard Scaler
+
+
+2. Train a neural network on the train set and predict on the test set. The neural network should have 2 hidden layers and the loss should be **mean_squared_error**. The expected **mean absolute error** on the test set is maximum 10.
+*Hint*: inscrease the number of epochs 
+**Warning**: Do no forget to evaluate the neural network on the **SCALED** test set. 

one_exercise_per_file/week03/day03/ex03/audit/readme.md

@@ -0,0 +1,9 @@
+1. This question is validated if the code that creates the neural network is: 
+
+    ```
+    model = keras.Sequential()
+    model.add(Dense(16, input_shape=(5,), activation= 'sigmoid'))
+    model.add(Dense(8, activation= 'sigmoid'))
+    model.add(Dense(5, activation= 'softmax'))
+
+    ```

one_exercise_per_file/week03/day03/ex03/readme.md

@@ -0,0 +1,12 @@
+# Exercise 3 Multi classification - Softmax
+
+The goal of this exercise is to learn to a neural network architecture for multi-class data. This is an important type of problem on which to practice with neural networks because the three class values require specialized handling. A multi-classification neural network uses as output layer a **softmax** layer. The **softmax** activation function is an extension of the sigmoid as it is designed to output the probabilities to belong to each class in a multi-class problem. This output layer has to contain as much neurons as classes in the multi-classification problem. This article explains in detail how it works. https://developers.google.com/machine-learning/crash-course/multi-class-neural-networks/softmax
+
+Let us assume we want to classify images and we know they contain either apples, bears, candies, eggs or dogs (extension of the example in the link above). 
+
+1. Create the architecture for a multi-class neural network with the following architecture and return `print(model.summary())`: 
+
+    - 5 inputs variables
+    - hidden layer 1: 16 neurons and sigmoid as activation function
+    - hidden layer 2: 8 neurons and sigmoid as activation function
+    - output layer: The number of neurons and the activation function should be adapted to this multi-classification problem. 

one_exercise_per_file/week03/day03/ex04/audit/readme.md

@@ -0,0 +1,7 @@
+1. This question is validated if the chunk of code is:
+
+```
+model.compile(loss='categorical_crossentropy',
+              optimizer='adam',
+              metrics=['accuracy'])
+```

one_exercise_per_file/week03/day03/ex04/readme.md

@@ -0,0 +1,11 @@
+# Exercise 4 Multi classification - Optimize 
+
+The goal of this exercise is to learn to optimize a multi-classification neural network. As learnt previously, the loss function used in binary classification is the log loss - also called in Keras `binary_crossentropy`. This function is defined for binary classification and can be extended to multi-classfication. In Keras, the extended loss that supports multi-classification is `binary_crossentropy`. There's no code to run in that exercise.
+
+1. Fill the chunk of code below in order to optimize the neural network defined in the previous exercise. Choose the adapted loss, adam as optimizer and the accuracy as metric.
+
+```
+model.compile(loss='',#TODO1
+              optimizer='', #TODO2
+              metrics=['']) #TODO3
+```

one_exercise_per_file/week03/day03/ex05/audit/readme.md

@@ -0,0 +1,27 @@
+1. This question is validated if the output of the first ten values of the train labels are: 
+
+```
+array([[0, 1, 0],
+       [0, 0, 1],
+       [0, 1, 0],
+       [0, 0, 1],
+       [0, 0, 1],
+       [1, 0, 0],
+       [0, 1, 0],
+       [1, 0, 0],
+       [0, 1, 0],
+       [0, 0, 1]])
+```
+
+2. This question is validated if the accuracy on the test set is bigger than 90%. To evaluate the accuracy on the test set you can use: `model.evaluate(X_test_sc, y_test_multi_class)`.
+
+Here is an implementation that gives 96% accuracy on the test set. 
+
+```
+model = Sequential()
+model.add(Dense(10, input_dim=4, activation='sigmoid'))
+model.add(Dense(3, activation='softmax'))
+# Compile model
+model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
+model.fit(X_train_sc, y_train_multi_class, epochs = 1000, batch_size=20)
+```

one_exercise_per_file/week03/day03/ex05/readme.md

@@ -0,0 +1,14 @@
+# Exercise 5 Multi classification example
+
+The goal of this exercise is to learn to use a neural network to classify a multiclass data set. The data set used is the Iris data set which allows to classify flower given basic features as flower's measurement. 
+
+Preliminary: 
+    - Split train test. Keep 20% for the test set. Use `random_state=1`. 
+    - Scale the data using Standard Scaler
+
+
+1. Use the `LabelBinarizer` from Sckit-learn to create a one hot encoding of the target. As you know, the output layer of a multi-classification neural network shape is equal to the number of classes. The output layer expects to have a target with the same shape as its output layer.  
+
+2. Train a neural network on the train set and predict on the test set. The neural network should have 1 hidden layers. The expected **accuracy** on the test set is minimum 90%.
+*Hint*: inscrease the number of epochs 
+**Warning**: Do no forget to evaluate the neural network on the **SCALED** test set. 

one_exercise_per_file/week03/day03/readme.md

@@ -0,0 +1,24 @@
+# W3D2  Piscine AI - Data Science 
+
+
+# Table of Contents:
+
+
+# Introduction
+Keras backend TF
+The goal of this day is to learn to use Keras to build Neural Networks and train them on small data sets. 
+
+classification & regression 
+
+'2.4.3'
+
+## Historical
+
+
+
+## Rules
+
+The correction will provide the code and output because it is not straightforward to reproduce results using Keras. There are many source of randomness. Even if all the seeds are fixed to a constant they may be other source of randomness. https://machinelearningmastery.com/reproducible-results-neural-networks-keras/
+A developper
+## Ressources 
+https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/

one_md_per_day_format/piscine/Week3/w3day02.md

@@ -237,7 +237,4 @@ You should notice that the neural network is struggling to learn. By luck the in
 `Epoch 50/50
 2/2 [==============================] - 0s 1ms/step - loss: 0.6559 - accuracy: 0.6274`
 
-2. This solution is validated if the the accuracy at epoch 50 is higher than 95%.
-
-
-AA9FE32D1CA7E292E6A4C27145
+2. This solution is validated if the the accuracy at epoch 50 is higher than 95%.