Symfony version(s) affected: 5.1/5.2

Description
Serializer Groups Annotations not updating when updated, need to cache:clear on each @Groups modification. In dev environment at least (not tested in prod).

Changing a arbitrary .yaml config updates cache and the Group is updated too.

How to reproduce
Create an app in 5.0 5.1-5.2 with Doctrine entities, add Serializer group and call the Serializer on this group :

    /**
     * @ORM\Column(type="string", length=70)
     * @Assert\NotBlank
     * @Groups("movies_get")
     */
    private $title;

Call the URL, all the JSON fields are OK.
Remove or add a Group annotation, reload the URL, the JSON fields are not updated.
Clear the cache in any way, the JSON fields are OK.

Possible Solution
Possible link with : #39988 ?
As the issue seems similar but at a higher level 🤓

Or, when switching from 5.0 to 5.1, Doctrine packages has changes, need code updates, so perhaps the issue comes from here ?

Additional context
If you have any questions, let me know.