CustomValidator ClientValidationFunction OnServerValidate Confusion

I have a form that has a textbox and a CustomValidator on it where I
need to perform some validation on both the client and also on the
server. I have a button on the form that saves the data. If I change
nothing in the field and hit the Save button, then Client side
validation occurs, the page does a postback and then the server side
validation occurs. If I change the data in the textfield, however,
and hit the save button then only the client side validation occurs,
the page does NOT postback and the server side validation does not
occur. At this point, I have the logic set so that it isn't really
even doing any validation (just setting IsValid = true) and it is
still doing this. Anyone have any idea what I am doing wrong?

<asp:TextBox ID="txtEmailAddress" MaxLength="255" runat="server"
Width="400px" CssClass="controlText" />

I have a customer validator set up as follows:

<asp:CustomValidator ID="CustomValidator1"
ClientValidationFunction="ClientSideValidate"
OnServerValidate="ServerValidation" EnableClientScript="true"
runat="server" ErrorMessage="Some Error Message"
ControlToValidate="txtEmailAddress" />

Client side Valdiation is as follows:
function ClientSideValidate(source, args)
{
//perform some validation
args.IsValid = true;

}

Server side Validation logic is as follows:
Sub ServerValidation(ByVal source As Object, ByVal args As
ServerValidateEventArgs)

'Perform some validation logic
args.IsValid = True

End Sub

Form has a button on it to save:

<asp:LinkButton ID="lnkSave" runat="server" ><img src="images/
save.jpg" class="imgButton" alt="Save" /></asp:LinkButton>


AutoEventWireup="false"