@RobertIndie @merlimat @eolivelli Could you take a second look? I believe this PR is more universal than #1 that focuses on the script to build Python client on macOS. This PR also avoids many code changes (include CMakeLists.txt)

Hi @merlimat, here are my points for some of your concerns.

For the most important issue (IMO) you've mentioned

We must not use a random version from C++ client main branch.

The main branch is not guaranteed to be stable. We should only depend the stable version of C++ client for formal releases. For example, git submodule can specify a tag of a formal C++ client release. When we release a Python client, we can specify the tag like

[submodule "pulsar-client-cpp"]
	path = pulsar-client-cpp
	url = https://github.com/apache/pulsar-client-cpp.git
        tag = v3.0.0-pre-release-1

The main branch should also relies on a tag. However, there is no tag in the new created pulsar-client-cpp repo. Should we create one and rely on it? Or I think we can reset to a specific commit in the script here.

When we're building a source tar.gz we would have to include C++ code
The Python wrapper shouldn't depend on any internal code from the C++ library, just the public API

I agree. Just want to remove the dependency in another PR. I will do that in this PR soon.

When building Python client, we shouldn't be building the C++ client lib. Instead we should just fetch the binaries that are already available (eg: RPM, Deb, APK, etc.. )

However, it makes the release or Python client slow.

Since most of the codes will be the same with #1, I marked this PR as draft and will make further changes after #1 is merged.

In future, this PR title might become "Add cpp client as a submodule" or "Add Instructions of building Python client from source"

We now have the freedom of doing releases with much less overhead and we can release new features much quicker than before.

I'm wondering how to manage the releases for C++ client now. If the main branch must depend on a released C++ client, then, if we want to support a new feature, we have to release the C++ client first. Then each C++ feature requires a release. I'm afraid that it's not a good way for release even if the release could be much quicker than before. We might have:

• pulsar-cpp-3.0.0-pre-feature1
• pulsar-cpp-3.0.0-pre-feature2
• pulsar-cpp-3.0.0-pre-feature3
• ...

Adding a git tag for them might be good (but I think it could still create a lot of tags). But should we create each binary uploaded for a single feature? @merlimat

When we release a Python client, we can specify the tag like

Exactly, but then it would be the same as depending on a released version of C++ client. Having the submodule won't help here.

if we want to support a new feature, we have to release the C++ client first.

Yes, if the feature is significant enough to grant a C++ release, then we just do the 2 releases (and also NodeJS and maybe other wrappers).

If the feature is not big enough, we can wait to include other changes in C++ before releasing. We would still be in a completely different situation from where we are today, where the Python release is tied to a huge Pulsar release that includes everything.

If we automate all the steps, we are now in a position to do C++ release with minimal effort, therefore we can do them very frequently.

Having the submodule won't help here.

I have thought that we can depend on an unstable version of pulsar-client-cpp in main branch, then for those want some specific features (or bug fixes), they can build the Python client from source.

But just thinking again, they can also build the unreleased C++ client from source. Then build the unreleased Python client from source based on the C++ client built by themselves. But this way should not be encouraged anyway unless the demands are very urgent.

It looks like Pulsar and BK, but C++ client can be released very quickly, not like BK. So I will close this PR.

FYI, @codelipenghui @Demogorgon314 @shibd

thinking again

Agree. Submodule increase understanding burdens.