Motivation

As reported in #9605, we were not handling properly the Max* argument when combined with Delimiter and Prefix, which returns CommonPrefixes.

Here's the description of the Delimiter parameter, which helps understand the behavior:

The delimiter grouping the included keys. A delimiter is a character that you specify to group keys. All keys that contain the same string between the prefix and the first occurrence of the delimiter are grouped under a single result element in CommonPrefixes. These groups are counted as one result against the max-keys limitation. These keys are not returned elsewhere in the response.

The behavior is most List* operations is pretty close, but has subtle changes between them.

Changes

• Split all tests related to List*, especially around pagination, as those are quite verbose and the test_s3 file is becoming extremely huge (>10k lines) into its own file.
• Create new tests validating this behavior for every List* operation except ListParts which doesn't have the CommonPrefix behavior.
  • ListObjects
  • ListObjectsV2
  • ListObjectVersions
  • ListMultipartUploads

EDIT: it was green before rebasing, pipeline is struggling (kclpy): https://app.circleci.com/pipelines/github/localstack/localstack/20034/workflows/a3805d65-c1b9-4708-a79b-f5ceed202830

Testing notes

I've also re-ran external test suites with this change and it's all green there as well.