Cost Function

[Ex]
Here is our training set and hypothesis as below:
$$\begin{array}{ll} \hfill\mathrm{Size~in~feet^2 (x)}\hfill & \hfill\mathrm{Price($)~in~1000's(y)}\hfill \\ \hline \\ 2104 & 460 \\ 1416 & 232 \\ 1534 & 315 \\ 852 & 178 \\ ... & ... \\ \end{array}$$

In this example, we want to fit a straight line to predict the house price, then we set a simple model.
Hypothesis: $h_\theta(x) = \theta_0 + \theta_1 x$

It's a simple regression problem, you can choose any number for $\theta_0$ and $\theta_1$, so that we can get the $h_\theta(x)$ and it's meaning the value which the model predict to the input x. As a prediction model, we want the difference between $h_\theta(x)$ and y to be small, in other words
is this hypothesis good fit to the data?

We can measure the accuracy of our hypothesis function by using a cost function or called squared error function, in other words it's almost the same as MSE in statistics.

[Def]
$$J(\theta_0, \theta_1) = \dfrac {1}{2m} \displaystyle \sum _{i=1}^m \left ( \hat{y}_{i}- y_{i} \right)^2 = \dfrac {1}{2m} \displaystyle \sum _{i=1}^m \left (h_\theta (x_{i}) - y_{i} \right)^2$$
m: the number of training example
$h_\theta(x)$: form of hypothesis