Previously errors were seen on older python versions, ensure those problems are gone and run the tests on all CI versions.