Ahh, I know the solution towards this. You were not voting in-game. To have the Voting xp stay with you, vote while in an infection game is being played. I've had this problem for a while, once I figured out about this though, it has helped me a lot. I hope this has been helpful to you as well. If you have anymore questions, just ask. :D
You can't get the ones you lost back. :/ I know it sucks.. I once had 150 lvls from voting and lost all of them. Just remember to vote while in an infection game. :>
Since this thread has been inactive for so long and the issue has most likely been resolved, I'll be closing this. Make another thread if the issue is still there.