cvs does not keep track of directories, only of files in it.
But: Normally, cvs does not remove the directories if they are empty. If you want to let cvs do that, you have to add -P ("prune directories") to the checkout and update commands.
Of course, these are only client-side options. If you want to completely remove it from the server, just remove that directory from the repository structure itself. Node that this is safe as long as the directory is really (and always was) empty - otherwise, I would not recommend that.