Okay a few things, after I was attacked over the past couple days I decided to do some work myself and found that it is pretty easy to bring the server to it's knees using just apache bench mark.
So while yes it was being attacked, this also means that something is wrong because it shouldnt...