Skip to content

Bug: Relative template_dir path causes TemplateNotFound error in monorepo configuration #845

New issue
New issue
Open
Open
Bug: Relative template_dir path causes TemplateNotFound error in monorepo configuration#845
Labels
bugSomething isn't working properlyconfirmedPrevent from becoming stale
Milestone
Monorepo support
@deme3

Description

@deme3
deme3
opened on Feb 23, 2024

The problem

TemplateNotFound error thrown by Jinja2 when generating changelog in a subfolder. The directory structure is as follows:

- main_program
+-- templates
    +-- main_program
        +-- CHANGELOG.md.j2
+-- CHANGELOG.md
+-- pyproject.toml
+-- ...
- lib1
- lib2

To be clear, the templates are discovered by semantic-release, but when they're passed to Jinja, a relative path is used, and Jinja is not able to find them.

Expected behavior

I expect a generated CHANGELOG.md in the same level as pyproject.toml by using the templates provided in the configuration.

Environment

  • Python Version: v3.8.18
  • Pip Version: v23.0.1
  • Semantic Release Version: v9.1.0
pip freeze 
annotated-types==0.6.0
antlr4-python3-runtime==4.13.1
antlr4-tools==0.2
argcomplete==3.2.2
argparse-addons==0.12.0
asn1tools==0.166.0
bcrypt==4.0.1
Beaker==1.12.1
beautifulsoup4==4.12.2
bitstruct==8.17.0
build==1.0.3
cantools==39.0.0
capstone==5.0.0rc2
certifi==2024.2.2
cffi==1.15.1
charset-normalizer==3.3.2
click==8.1.7
colorama==0.4.6
colored-traceback==0.3.0
crccheck==1.3.0
cryptography==40.0.2
cssselect==1.2.0
decli==0.6.1
decorator==5.1.1
diskcache==5.6.1
docutils==0.20.1
dotty-dict==1.3.1
exceptiongroup==1.1.3
filebytes==0.10.2
FormEncode==2.1.0
future==0.18.3
gitdb==4.0.11
GitPython==3.1.42
idna==3.6
importlib-metadata==7.0.1
importlib-resources==6.1.1
iniconfig==2.0.0
install-jdk==1.1.0
intervaltree==3.1.0
jaraco.classes==3.3.1
jeepney==0.8.0
Jinja2==3.1.3
keyring==24.3.0
lark==1.1.8
-e git+<redacted>
lxml==4.9.3
Mako==1.2.4
markdown-it-py==3.0.0
MarkupSafe==2.1.2
mdurl==0.1.2
mikai==4.0.1
more-itertools==10.2.0
msgpack==1.0.5
-e git+<redacted>
nh3==0.2.15
nose==1.3.7
numpy==1.24.4
packaging==23.1
paramiko==3.1.0
Paste==3.7.1
PasteDeploy==3.1.0
PasteScript==3.3.0
pkginfo==1.9.6
pluggy==1.3.0
plumbum==1.8.1
prompt-toolkit==3.0.36
psutil==5.9.5
pwntools==4.9.0
pycparser==2.21
pydantic==2.6.1
pydantic_core==2.16.2
pyelftools==0.29
Pygments==2.16.1
Pylons==1.0.3
PyNaCl==1.5.0
pyparsing==3.1.0
pyproject_hooks==1.0.0
pyserial==3.5
PySide6==6.6.1
PySide6-Addons==6.6.1
PySide6-Essentials==6.6.1
PySocks==1.7.1
pytest==7.4.2
python-can==4.2.2
python-dateutil==2.8.2
python-gitlab==4.4.0
python-semantic-release==9.1.0
pytz==2023.3
PyYAML==6.0.1
questionary==2.0.1
readme-renderer==42.0
repoze.lru==0.7
requests==2.31.0
requests-toolbelt==1.0.0
rfc3986==2.0.0
rich==13.7.0
ROPGadget==7.3
ropper==1.13.8
Routes==2.5.1
rpyc==5.3.1
SecretStorage==3.3.3
shellingham==1.5.4
shiboken6==6.6.1
simplejson==3.19.2
six==1.16.0
smmap==5.0.1
sortedcontainers==2.4.0
soupsieve==2.5
Tempita==0.5.2
termcolor==2.4.0
textparser==0.24.0
tomli==2.0.1
tomlkit==0.12.3
twine==5.0.0
typing_extensions==4.7.1
tzdata==2023.3
unicorn==2.0.1.post1
urllib3==2.2.1
waitress==2.1.2
wcwidth==0.2.9
WebError==0.13.1
WebHelpers==1.3
WebOb==1.8.7
WebTest==3.0.0
wrapt==1.15.0
zipp==3.17.0

Using build==1.0.3 and setuptools=56.0.0.

Configuration

[tool.semantic_release]
assets = []
commit_message = "chore: bumped version to {version}\n\nAutomatically generated by python-semantic-release"
commit_parser = "angular"
logging_use_named_masks = false
major_on_zero = true
tag_format = "main_program-v{version}"
version_variables = [
  "main_program/__init__.py:__version__"
]


[tool.semantic_release.changelog]
template_dir = "main_program/templates"
changelog_file = "CHANGELOG.md"
exclude_commit_patterns = []

Logs

GitHub Attachment to Logs

Additional context

There's something wrong here:

python-semantic-release/semantic_release/changelog/template.py

Lines 77 to 123 in 82bfcd3

def recursive_render(
template_dir: Path,
environment: Environment,
_root_dir: str | os.PathLike[str] = ".",
) -> list[str]:
rendered_paths: list[str] = []
for root, file in (
(Path(root), file)
for root, _, files in os.walk(template_dir)
for file in files
# we slice relpath.parts[1:] to allow the top-level
# template folder to have a dot prefix.
# e.g. to permit ".github/psr-templates" to contain the templates,
# rather than enforcing a top-level, non-hidden directory
if not any(
elem.startswith(".")
for elem in Path(root).relative_to(template_dir).parts[1:]
)
and not file.startswith(".")
):
output_path = (_root_dir / root.relative_to(template_dir)).resolve()
log.info("Rendering templates from %s to %s", root, output_path)
output_path.mkdir(parents=True, exist_ok=True)
if file.endswith(".j2"):
# We know the file ends with .j2 by the filter in the for-loop
output_filename = file[:-3]
# Strip off the template directory from the front of the root path -
# that's the output location relative to the repo root
src_file_path = str((root / file).relative_to(template_dir))
output_file_path = str((output_path / output_filename).resolve())
log.debug("rendering %s to %s", src_file_path, output_file_path)
stream = environment.get_template(src_file_path).stream()
with open(output_file_path, "wb+") as output_file:
stream.dump(output_file, encoding="utf-8")
rendered_paths.append(output_file_path)
else:
src_file = str((root / file).resolve())
target_file = str((output_path / file).resolve())
log.debug(
"source file %s is not a template, copying to %s", src_file, target_file
)
shutil.copyfile(src_file, target_file)
rendered_paths.append(target_file)
return rendered_paths

specifically, line 109

python-semantic-release/semantic_release/changelog/template.py

Lines 107 to 110 in 82bfcd3

log.debug("rendering %s to %s", src_file_path, output_file_path)
stream = environment.get_template(src_file_path).stream()

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working properlyconfirmedPrevent from becoming stale

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions