Two ratings given in a matter 6 hours apart? I do not see rate abuse. He could easily disagree with what you were saying or disliked it previously for this to happen. Remember, 2 are negative and the others are simply neutral ratings. A pattern for example would be this happening several times and that would be considered rate abuse.