Hover over the Following button next to the user you don't want to follow anymore. It will turn red and say Unfollow. Click Unfollow and the user will be removed from your following list.
If it doesn't turn red or click, refresh the page and try again, or click the user's name to visit their profile page and unfollow them there.
My follower count is wrong:
If you received a message from Twitter alerting you to a new follower, but don't see that person later in your followers list, it's likely that user was removed for spam activity, unfollowed you, or deactivated their account.
I'm having trouble removing a follower (blocking):
The only way to remove someone who's following you is to block them. Click here to learn how to block a user.
It looks like I'm not following someone I know I followed: