Loading [MathJax]/jax/output/HTML-CSS/jax.js

3/9/17

Classification - Binary

[Foreword]

[Ex]
Email: spam / not spam
Online transaction: fraudulent
Tumor: Malignant / Benign

y {0, 1}

0: negative class ex: not spam, Benign tumor
1: positive class ex: spam, Malignant tumor

[Linear regression is not good]
Assume this is the hypothesis: hθ(x)=θTx
then we set 0.5 as the threshold, that means

hθ(x)0.5, predict y = 1
hθ(x)<0.5, predict y = 0

Here is the schematic diagram
[Plot 1]


When our data obtains the red point only, this threshold works well.
But when adding a purple point around there, the fitted line will get more flatten than before and it's not a good prediction.
Another funny thing is when we using linear regression for a classication problem, the hypothesis may output a value that are much larger or less than 1 and 0, even if the training example y only have 0 and 1.
So, this is the reason we will use another method which is called : Logistic regression.

[Logistic regression]
Because we want 0hθ(x)1, so we using a function g to change the original  hθ(x)

hθ(x)=g(θTx)=11+eθTx

This formula is called "sigmoid function" or "logistic function"
We can use 0.5 as the threshold again through the sigmoid function, Let's take a loot on this plot
[Plot 2]


When hθ(x)0.5 means z0, it'll output y = 1, in contrast hθ(x)<0.5 means z<0,y=0


[Ex 1]
hθ(x)=p(y=1|x,θ)= estimate the probability that y = 1 on input x. So, if we input A patient's tumor size into the sigmoid function and get
hθ(x)=0.7
Then we can tell A there is 70% chance that this tumor being malignant.

[Ex 2]
Assume our data is below this plot and our hθ(x)=θ0+θ1x1+θ2x2+θ3x21+θ4x22,
[Plot 3]

we can get a divide circle when choose the fitted θ by Logistic regression as below plot.
θ=[1,0,0,1,1]T
It means this function will predict y = 1 when  1+x21+x221
[Plot 4]

This green circle is called decision boundary, the blue x means y = 1 and red o means y = 0


No comments:

Post a Comment