Higham Hill Library in danger of being moved