See the [annotation demo](http://matplotlib.org/devdocs/examples/pylab_examples/annotation_demo.html):  Both "points" and "bottom right (points)" are in the wrong location.