Skip to content

Lambda: Implement cross-account access #8195

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
May 15, 2023
Merged

Conversation

viren-nadkarni
Copy link
Member

@viren-nadkarni viren-nadkarni commented Apr 25, 2023

Summary

This PR adds support for cross-account access to Lambda functions and layers.

Specifically, following operations:

This is a prerequisite for cross-account IAM policy enforcement.

Related

https://docs.aws.amazon.com/lambda/latest/dg/access-control-resource-based.html#permissions-resource-xaccountinvoke

@viren-nadkarni viren-nadkarni self-assigned this Apr 25, 2023
@github-actions
Copy link

github-actions bot commented Apr 26, 2023

LocalStack Community integration with Pro

2 039 tests   1 758 ✔️  1h 16m 30s ⏱️
       2 suites     281 💤
       2 files           0

Results for commit 8c402e1.

♻️ This comment has been updated with latest results.

@viren-nadkarni viren-nadkarni force-pushed the lambda-cross-accounts branch from dfd04b8 to dcd931b Compare April 27, 2023 07:30
@viren-nadkarni viren-nadkarni force-pushed the lambda-cross-accounts branch from ccbd481 to 37e09df Compare April 27, 2023 11:33
Lambda function retrievals must be made with same context region as the
ARN.
@@ -54,7 +55,7 @@

# Pattern for extracting various attributes from a full or partial ARN or just a function name.
FUNCTION_NAME_REGEX = re.compile(
r"(arn:(aws[a-zA-Z-]*):lambda:)?((?P<region>[a-z]{2}(-gov)?-[a-z]+-\d{1}):)?(?P<account>\d{12}:)?(function:)?(?P<name>[a-zA-Z0-9-_\.]+)(:(?P<qualifier>\$LATEST|[a-zA-Z0-9-_]+))?"
r"(arn:(aws[a-zA-Z-]*):lambda:)?((?P<region>[a-z]{2}(-gov)?-[a-z]+-\d{1}):)?(?:(?P<account>\d{12}):)?(function:)?(?P<name>[a-zA-Z0-9-_\.]+)(:(?P<qualifier>\$LATEST|[a-zA-Z0-9-_]+))?"
Copy link
Member Author

@viren-nadkarni viren-nadkarni Apr 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before this change, the regex used to capture the : as part of the account ID

@coveralls
Copy link

coveralls commented Apr 27, 2023

Coverage Status

Coverage: 82.139% (+0.05%) from 82.092% when pulling 7b1ad5a on lambda-cross-accounts into c28ef9a on master.

@viren-nadkarni viren-nadkarni marked this pull request as ready for review April 27, 2023 13:30
@viren-nadkarni viren-nadkarni added the semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases label Apr 27, 2023
@joe4dev
Copy link
Member

joe4dev commented Apr 28, 2023

@viren-nadkarni Looking forward to cross-account support for Lambda 🚀

Lambda layers are not in the list because the AWS documentation dedicates a separate section to them here: https://docs.aws.amazon.com/lambda/latest/dg/access-control-resource-based.html#permissions-resource-xaccountlayer

lambda:GetLayerVersion is an important feature for shared Lambda layers. Should this be covered within this PR or as a follow-up?

There are a couple of TODOs related to IAM in localstack.services.awslambda.provider.LambdaProvider._validate_layers, including a hard-coded AccessDeniedException IAM exception.

@viren-nadkarni viren-nadkarni force-pushed the lambda-cross-accounts branch from 593408b to 2399a17 Compare May 4, 2023 10:20
@viren-nadkarni viren-nadkarni force-pushed the lambda-cross-accounts branch from 2399a17 to 07ac492 Compare May 4, 2023 10:32
Policy operations don't work across accounts in the old provider
@viren-nadkarni
Copy link
Member Author

viren-nadkarni commented May 5, 2023

@joe4dev: Thanks for highlighting that! I've added cross-account access support for the layer operations along with the tests. The scope is currently limited to retrievals only. Actual IAM enforcement will be added in a later PR.

Copy link
Member

@dfangl dfangl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work making lambda ready for cross account access!
Just some minor nits, it is good to merge once they are addressed (from my side).

In general, it would be nice to be able to perform aws validated cross account tests, what do you think @dominikschubert , is there some way to facilitate this?

layer_version_arn = api_utils.layer_version_arn(
layer_name, context.account_id, context.region, str(version_number)
)
region_name, account_id, layer_n, _ = LambdaProvider._resolve_layer(layer_name, context)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why use layer_n instead of layer_name here suddenly? I actually had to think a second there what that is supposed to mean.

Copy link
Member Author

@viren-nadkarni viren-nadkarni May 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

layer_name is the function argument and can contains either the layer name or layer ARN. This is user-provided and error messages are generated accordingly.

layer_n contains the layer name and is named so to avoid overwriting 🔝

I've added a comment to clarify this in 8c402e1

@@ -3128,14 +3147,12 @@ def delete_layer_version(
if version_number < 1:
raise InvalidParameterValueException("Layer Version Cannot be less than 1", Type="User")

# TODO: test cross-region? (e.g. with functions it doesnt work and raises an exception)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not seem tested now, are we sure this works? Otherwise we might not want to remove that TODO

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We generally allow cross-region access in LocalStack, except in Lambda where it is snapshot tested

Layers currently allows cross-region in LS, so I've removed the comments

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the point of that comment was however to snapshot test if it is possible in aws or not, and apply the restrictions.

@viren-nadkarni
Copy link
Member Author

In general, it would be nice to be able to perform aws validated cross account tests, what do you think @dominikschubert , is there some way to facilitate this?

I'm actually working on this. #8204 will update our test strategy to use a non-standard account ID. As you can see from the failing tests, a lot of our services are not cross-account ready.

I'll be going through all the services and update the cross-accounts logic. This is a huge undertaking and will be done in a series of PRs, starting with #8223 (for sqs) and #8235 (for lambda).

@viren-nadkarni viren-nadkarni merged commit 53ebcc6 into master May 15, 2023
@viren-nadkarni viren-nadkarni deleted the lambda-cross-accounts branch May 15, 2023 06:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants