As mineterria said, it is not the server. The server (should) have enough memory to support everyone/thing on the server. It is either you Internet, or your computer. If you have a really low-end computer, lag is bound to occur. However, one thing you must realize, is that there are 2 types of lag: FPS Lag and Latency Lag. FPS Lag is your FPS dropping, and then if it is a spike, it will come back up in a few seconds. Latency Lag occurs when you Internet connection is not strong enough to support the packages that the server is sending to your client, via the Internet. Try playing on the server with F3 on, and watch you FPS during the spikes, if the FPS drops and comes back up, it is FPS lag, if not, it is latency lag. To help with FPS lag, consider getting a mod such as optifine, and lower you video settings in MC. For Latency lag, you can't do much. Best of luck!
You'd be surprised, there was a survey taken last year, of gamers Internet quality, and most of the time, unless they did gaming most all of their time, they had fairly poor Internet quality.