What probably happened is you voted offline or not on infection and you died losing all the lvls you got from the votes when you were offline. This glitch has been around for ages and it happens to everyone.
Alright I think that is what happened. SSMH you are right, infection is the server that glitches the most because the infection plugin for 1.8.1 is still sort of in beta form. So you have to vote online infection to fully receive your "lucky" levels. You do not have to be online for your regular 3 levels of xp. This is just part of the glitch I have learned from voting.
I have 97 levels on infection right now, and I'm still the cfl level or something like that. You will still get the levels, just not the rank.
Voting when you are not on the server will give you the xp levels, it just won't let you rank up with them.
Vote whilst having your minecraft window still open when running infection. Stay on Infection until the chat message comes through. Your levels will then stay with you even when you die.