When Facebook scans your email contact list for friends, it makes note of the friends you add and don't add. You see, the people you don't add stay in Facebook's system and will come back to haunt you later — exes, your backstabbing high school enemy, old bosses. Facebook will then continually remind you about these people with their "Facebook Friend Suggestion Emails."
To remove these contacts and any trace of them on Facebook, just log in to your account and go here. Done and done.