Removing Bookmarks from home page

I just recently added a website bookmark to my home screen. Story short, I'm now trying to delete it and when I hold it down, it starts to wiggle for deletion, but there is no little x in the corner of it. All I can do is move it. It doesn't seem to be deletable. Any suggestions on how to get rid of it?