i will implement linear regression which can be adapted classification easily, i use Matlab by following the Dr. Andrew Ng's class. You can watch the classes online from here.
While implementing i also came across a very nice blog post, actually only dataset differs, in this case i use the original dataset given by the Dr. Ng, more details of the code below can be reached from here (DSPlog )
and the batch gradient descent update rule is
For different values of theta, in this case theta0 and theta1, we can plot the cost function J(theta) in 3d space or as a contour.