As @anntzer described in #9318, integer categoricals (even when strings) aren't using the unit mapping and so are being handled incorrectly:

fig, ax = plt.subplots()
ax.set_xlim([-1,2])
ax.set_ylim([0,2])
ax.plot(["!", "0"], [1, 2])

ETA: Ok, so the bug is that it's doing an inplace substitution, and casting into a string:

vals[vals=='!'] = 0 # casts into string
vals=='0' # matches everything