Quantitative Problems

Suppose that a neural network has been trained to classify students as likely to graduate or not likely to graduate based on various input parameters. There is a single output neuron having four inputs, $x_1,x_2,x_3,$ and one output, $y$. The output $y$ is used to make the classification. The weights of the three inputs of this neuron are $w_1=0.25$, $w_2=0.09$, and $w_3=-<!--\; -\; -->0.31$. The bias of the neuron is $b=-<!--\; -\; -->0.23$. Find the output $y$ if $(x_1,x_2,x_3)=(0.4,-<!--\; -\; -->1.2,0.6)$ and if the activation function is:

1. ReLU
2. Leaky ReLU defined by max⁡ (0.2x,x)
3. Sigmoid
4. Softplus

Which of the four activation functions, applied at the output layer, would be best suited for this classification task? Using this activation function, how would the neural network classify the student?

MSE

Hinge loss