Oh yes, it sounds quite creepy indeed. Teleporting when you look away at it. However, they only attack when you put the cross hairs directly on them. Dont hit them with the cross hairs and they dont do anything.
Also, thats an epic seed. I wish my world has something like that.