Description
Currently the 'minimal' version of Airflow Docker image ships with 20+ extra dependencies, which according to my understanding, are the most common used libraries. The idea of this PR to prepare another, pure Airflow image, which wouldn't have those dependencies and would only ship with core Airflow package.
https://github.com/apache/airflow/blob/main/Dockerfile#L37
Alternative solutions would be:
- ship separate Airflow images per each executor type (kubernetes, dask, celery, etc.). I like this one equally to having vanilla Airflow image
- fix the issue with dask. I don't like this idea as it's less scalable as any of packages mentioned below could yield similar issues
Use case/motivation
We are using official Airflow image for Airflow on our k8s cluster, which is constantly scanned for security vulnerabilities. The fact that dask is pinned to very low version is problematic for us cause there are a lot of red flags coming from this package. We do not need dask at all as we rely on KubernetesExecutor. Some of the security scanners work on docker layer level so removing/upgrading dask doesn't solve the problem.
Related issues
No response
Are you willing to submit a PR?
Code of Conduct
Description
Currently the 'minimal' version of Airflow Docker image ships with 20+ extra dependencies, which according to my understanding, are the most common used libraries. The idea of this PR to prepare another, pure Airflow image, which wouldn't have those dependencies and would only ship with core Airflow package.
https://github.com/apache/airflow/blob/main/Dockerfile#L37
Alternative solutions would be:
Use case/motivation
We are using official Airflow image for Airflow on our k8s cluster, which is constantly scanned for security vulnerabilities. The fact that dask is pinned to very low version is problematic for us cause there are a lot of red flags coming from this package. We do not need dask at all as we rely on KubernetesExecutor. Some of the security scanners work on docker layer level so removing/upgrading dask doesn't solve the problem.
Related issues
No response
Are you willing to submit a PR?
Code of Conduct