Add support for Python 3.10

Latest official update for anyone who missed it in the wall of comments:

We have today [2023-02-28] published a Lambda container base image for Python 3.10, for both x86_64 and arm64 architectures. This image is suitable for production workloads.

Instructions for how to use Lambda container base images for Python are given in the ‘Usage’ tab at https://gallery.ecr.aws/lambda/python.

We expect to publish managed runtime support for Python 3.10 within 90 days.

Previous official update:

We recognize the extended delay in providing Python 3.10 support in AWS Lambda, and apologize for the inconvenience this has caused. Our plan is to publish a base container images for Python 3.10 this month [February 2023], with the managed runtime and tooling support (SAM CLI, CFN, CDK, AWS CLI, console, etc) support to follow. We will provide an ETA for the managed runtime when we publish the container image.

Python 3.10 was released on 2021-10-04

Announcement: https://pythoninsider.blogspot.com/2021/10/python-3100-is-available.html
- 3.10.1 announcement: https://pythoninsider.blogspot.com/2021/12/python-3101-is-available.html
- 3.10.2 announcement: https://pythoninsider.blogspot.com/2022/01/python-3102-3910-and-3110a4-are-now.html
- 3.10.3 announcement: https://pythoninsider.blogspot.com/2022/03/python-3103-3911-3813-and-3713-are-now.html
- 3.10.4 announcement: https://pythoninsider.blogspot.com/2022/03/python-3104-and-3912-are-now-available.html
- 3.10.5 announcement: https://pythoninsider.blogspot.com/2022/06/python-3105-is-available_6.html
- 3.10.6 announcement: https://pythoninsider.blogspot.com/2022/08/python-3106-is-available.html
- 3.10.7 announcement: https://pythoninsider.blogspot.com/2022/09/python-releases-3107-3914-3814-and-3714.html
- 3.10.8 announcement: https://pythoninsider.blogspot.com/2022/10/python-versions-3108-3915-3815-3715-now.html
- 3.10.9 announcement: https://pythoninsider.blogspot.com/2022/12/python-3111-3109-3916-3816-3716-and.html
- 3.10.10 announcement: https://pythoninsider.blogspot.com/2023/02/python-3112-python-31010-and-3120-alpha.html
- 3.10.11 announcement: https://pythoninsider.blogspot.com/2023/04/its-time-for-another-set-of-python.html
Release schedule: https://www.python.org/dev/peps/pep-0619/
What’s new in Python 3.10: https://docs.python.org/3.10/whatsnew/3.10.html

Please?

Any implementation timeline?

some features look interesting, looking forward!

keen !

+1
as much as I love building python from source - maybe 3.7 is a little long in the tooth to be the standard...

@jbesw

+1 as much as I love building python from source - maybe 3.7 is a little long in the tooth to be the standard...

Agree - looks like @jbesw did the work for 3.9... maybe he can help with 3.10 support?

https://aws.amazon.com/blogs/compute/python-3-9-runtime-now-available-in-aws-lambda/

Thank you everyone for your feedback regarding AWS Lambda support for Python 3.10. We are aware of the ask, however, we don't currently have a timeline to share.

Python 3.9 just hit the security-only stage of its maintenance lifecycle:

https://pythoninsider.blogspot.com/2022/05/python-3913-is-now-available.html

@jtuliani

Two months since the last update. Any (hopefully good) news here? @jtuliani

@jtuliani

I had a desire to update my lambda builds to 3.10 the other day and began looking into this more. Since this issue has been open for a little while I thought I might take a stab at building this from the amazonlinux base image (as the python 3.9 builds are). None of the guides I found quite made this copy / paste possible.

So I came up with this, which I tested and seems to work just fine at least for my use cases which includes things like numpy. But depending on your project complexity your mileage may vary. It doesn't follow all the same conventions with putting task logic in /var/task or app logic in /var/runtime but that didn't cause any conflicts for me. It's kind of a hodgepodge of various AWS docs and articles I found. Wouldn't surprise me if it can be cleaned up a bit.

I don't think I have access to contribute this back anywhere otherwise I would create PR. @jtuliani please feel free to point me in the right direction for that and I'm happy to help iterate on a better vetted and reviewed 3.10 image.

In the meantime I hope this helps others.

ARG FUNCTION_DIR="/app/"
ARG AWS_LINUX_VERSION="2022"
ARG PYTHON_VERSION="3.10.4"


FROM amazonlinux:${AWS_LINUX_VERSION} as python-layer

ARG PYTHON_VERSION

# install python
RUN yum update -y
RUN yum groupinstall "Development Tools" -y
RUN yum install openssl1.1 openssl1.1-devel libffi-devel bzip2-devel wget -y
RUN wget https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tgz
RUN tar -xf Python-${PYTHON_VERSION}.tgz
RUN cd Python-${PYTHON_VERSION}/ && \
    ./configure --enable-optimizations && \
    make install
RUN ln -s /Python-${PYTHON_VERSION}/python /usr/bin/python
RUN python -m pip install --upgrade pip


FROM amazonlinux:${AWS_LINUX_VERSION} as base-layer

# copy over python
ARG PYTHON_VERSION
COPY --from=python-layer /Python-${PYTHON_VERSION} /Python-${PYTHON_VERSION}
COPY --from=python-layer /usr/local/bin /usr/local/bin
COPY --from=python-layer /usr/local/lib /usr/local/lib
RUN ln -s /Python-${PYTHON_VERSION}/python /usr/bin/python

ARG FUNCTION_DIR
RUN mkdir -p ${FUNCTION_DIR}
WORKDIR ${FUNCTION_DIR}


FROM base-layer as build-layer

######## YOUR OWN SETUP PROCESS HERE ########################
# copy over requirements and install those
COPY setup.py .
RUN pip install . --target "${FUNCTION_DIR}"

# copy over configuration and service code then install it
COPY config/ ${FUNCTION_DIR}/config/
COPY service/ ${FUNCTION_DIR}/
RUN pip install . --target "${FUNCTION_DIR}"
######## ########################### ########################

# install lambda runtime interface client for python
RUN pip install awslambdaric --target "${FUNCTION_DIR}"


FROM base-layer as runtime-layer

# copy in the built dependencies
ARG FUNCTION_DIR
COPY --from=build-layer ${FUNCTION_DIR} ${FUNCTION_DIR}
WORKDIR ${FUNCTION_DIR}

# (optional) add lambda runtime interface emulator
#ADD https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie /usr/bin/aws-lambda-rie
#RUN chmod 755 /usr/bin/aws-lambda-rie
#ENTRYPOINT [ "/usr/bin/aws-lambda-rie", "python", "-m", "awslambdaric" ]

ENTRYPOINT [ "python", "-m", "awslambdaric" ]

######## REFERENCE YOUR OWN HANDLER HERE ########################
CMD [ "main.app" ]```
######## ############################### ########################

@jwgstr

@jwgstr - can you post this Dockerfile as a gist or repo so I can discuss issues with it w/o cluttering the main thread? (in particular, yum update fails on a self-hosted Gitlab.)

@jwgstr

@jwgstr - can you post this Dockerfile as a gist or repo so I can discuss issues with it w/o cluttering the main thread? (in particular, yum update fails on a self-hosted Gitlab.)

excellent idea: https://gist.github.com/jwgstr/4dcc9f16c52529486de0f6fa78c55aef

This is now an issue for users of Gitlab who use the Docker:latest image to build lambda images using DinD. The Docker image is built off of Alpine 3.16 which now only supports python3.10 natively.

Our workaround is to use pyenv to install a python 3.9 version... but that adds significant time to our builds.

You could also just not use the latest tag and continue with the older image (e.g. the docker:20.10.16-dind-alpine3.15 tag). Using the latest tag is generally not a great idea, because it'll often lead to issues like these in the worst possible moment.

Not saying that it isn't ridiculous for AWS to still not support Python 3.10 in Lambdas, just offering my opinion on best practices when using container images.

Let me try that. (Thanks, I had gone through the list of available Docker tags and I didn't see that one.)

Ty,
Stephen

Error: expected runtime to be one of [... python3.6 python3.7 python3.8 python3.9 ...], got python3.10

Seems like the api for creating functions don't know about 3.10 yet. Hopefully it won't take too long to be added, I couldn't find the info in the blog post 🤞

@jpmatusik

Chiming in with @jpmatusik, kudos for the release!

Very glad to see this, dismayed at a few comments not keeping it "above the belt" with the criticism of the delay. There are humans on both ends here.

@jtuliani

@jtuliani Thank you AWS team for releasing this update. I would love to hear the details of what was the blocker, it would make a great postmortem to prevent this in the future.

Finally... We Germans like to say "Was lange währt, wird endlich gut". Thank you, AWS, for basically doing your job. It's not easy sometimes, I know.

But seriously, AWS, you should really, I mean, like, REALLY check on and fix your delivery/upgrade process. There's something terribly flawed and smelly if a from the viewpoint of your customers pretty simple upgrade to a new runtime version takes one and a half year.

@jtuliani

@jtuliani Thank you AWS team for releasing this update. I would love to hear the details of what was the blocker, it would make a great postmortem to prevent this in the future.

This would be great, even if we just got the fully digested conclusions of the postmortem. I doubt that it was the runtime itself causing the issue, which would make a postmortem even more insightful for those of us on the outside. I suspect that this took so long because it wasn't just an update to the Python Lambda runtime, but a massive update across a large part of the system(evidenced by the large number of updates that came with this, and the fact that the 3.11 base image is already in preview).

It would be great if we could learn with AWS about what the stumbling blocks were so that when we need to plan for a massive update at our own companies we can apply those lessons.

But seriously, AWS, you should really, I mean, like, REALLY check on and fix your delivery/upgrade process. There's something terribly flawed and smelly if a from the viewpoint of your customers pretty simple upgrade to a new runtime version takes one and a half year.

As a customer, it’s worrying that it’s taking AWS more than 12 months to update for a system that has a fixed 12 month upgrade cycle. If AWS continues the way they have for Python 3.9 and 3.10, Lambda is going to continually fall further and further behind on Python versions.

I'm not gonna be the one to do it, but for the people requesting some kind of post-mortem it might make sense to open a new top level issue.
This one is now closed and probably less likely to receive a substantial response.

@alxsavard

@alxsavard Thanks for flagging the doc issue, we're looking into that. However, this should be a documentation issue only, the CreateFunction API does support Python 3.10. Are you seeing an error when calling the API? If so, in which region?

@jtuliani

@jtuliani I did get an error yesterday (in us-east-1) but it seems to be fixed now! Thanks for asking!

@alxsavard

@alxsavard Thanks for confirming. It takes quite a while to roll out a runtime release safely across all regions (which is why folks on this thread found the release before the official announcement). I expect you tried it before the us-east-1 deployment was complete. We appreciate the quick follow-up, thank you.

Is there an ETA for Python 3.10 on Lambda@Edge? The announcement doesn't mention Lambda@Edge and the documentation doesn't list Python 3.10 (but might be outdated).

Or do I have to ask the CloudFront team?

@Felixoid

@Felixoid you need to bump TF version to =>0.13.2 and aws provider version =>4.64 then you will see it
UPD sorry missed a part with Lambda@Edge

Kindly excuse me, but it's the web console.

update

the message was deleted. There was a screenshot about the inability to use python 3.10 as a lambda@edge function

@Felixoid

@Felixoid you need to bump TF version to =>0.13.2 and aws provider version =>4.64 then you will see it

@AntonKyrychekCrit They are trying to create a Lambda@Edge function which has further restrictions. Please see the Lambda@Edge supported runtimes and note that Python 3.10 is not listed.

Navigation Menu

Repository navigation

Add support for Python 3.10 #31

138 remaining items

update

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Add support for Python 3.10 #31

Description

Activity

andreif commented on Feb 8, 2022

volphy commented on Feb 10, 2022

nikitajz commented on Feb 17, 2022

leslie-alldridge commented on Feb 25, 2022

fadesibert commented on Mar 2, 2022

alimcmaster1 commented on Mar 10, 2022

jtuliani commented on Mar 14, 2022

acdha commented on May 17, 2022

HWiese1980 commented on May 18, 2022

jwgstr commented on May 19, 2022

matthewdeanmartin commented on May 24, 2022

jwgstr commented on May 24, 2022

sjzabel commented on Jun 15, 2022

BjoernPetersen commented on Jun 15, 2022

sjzabel commented on Jun 15, 2022

138 remaining items

alxsavard commented on Apr 18, 2023

acbellini commented on Apr 18, 2023

quinnypig commented on Apr 18, 2023

sdaves commented on Apr 18, 2023

HWiese1980 commented on Apr 19, 2023

BTripp1986 commented on Apr 19, 2023

kinghuang commented on Apr 19, 2023

fitzoh commented on Apr 19, 2023

jtuliani commented on Apr 19, 2023

alxsavard commented on Apr 19, 2023

jtuliani commented on Apr 19, 2023

michael-k commented on Apr 20, 2023

AntonKyrychekCrit commented on May 11, 2023

Felixoid commented on May 11, 2023

update

mcdonnnj commented on May 11, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions