run_as_user shows none even when default_impersonation set in the config

[sudarshan2906]

Apache Airflow version:
2.1.2

Environment:

• Cloud provider or hardware configuration: AWS ECS
• OS (e.g. from /etc/os-release): debian
• Kernel (e.g. uname -a): Linux
• Install tools:
• Others:

What happened:
When setting default_impersonation in airflow.cfg as testuser and not passing run_as_user in the default_parm or operator parameters, it is showing run_as_user in the UI as None (It is using testuser to execute the dag though):

And also throwing error in logs and some of the dags are failing

[2021-07-26 09:53:41,929: ERROR/ForkPoolWorker-7] Failed to execute task PID of job runner does not match.
--
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/airflow/executors/celery_executor.py", line 117, in _execute_in_fork
args.func(args)
File "/usr/local/lib/python3.7/site-packages/airflow/cli/cli_parser.py", line 48, in command
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/utils/cli.py", line 91, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/cli/commands/task_command.py", line 238, in task_run
_run_task_by_selected_method(args, dag, ti)
File "/usr/local/lib/python3.7/site-packages/airflow/cli/commands/task_command.py", line 64, in _run_task_by_selected_method
_run_task_by_local_task_job(args, ti)
File "/usr/local/lib/python3.7/site-packages/airflow/cli/commands/task_command.py", line 121, in _run_task_by_local_task_job
run_job.run()
File "/usr/local/lib/python3.7/site-packages/airflow/jobs/base_job.py", line 245, in run
self._execute()
File "/usr/local/lib/python3.7/site-packages/airflow/jobs/local_task_job.py", line 131, in _execute
self.heartbeat()
File "/usr/local/lib/python3.7/site-packages/airflow/jobs/base_job.py", line 226, in heartbeat
self.heartbeat_callback(session=session)
File "/usr/local/lib/python3.7/site-packages/airflow/utils/session.py", line 67, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/airflow/jobs/local_task_job.py", line 195, in heartbeat_callback
raise AirflowException("PID of job runner does not match")
airflow.exceptions.AirflowException: PID of job runner does not match

What you expected to happen:
Dags not to fail and logs not to show this error messages.

How to reproduce it:
Set default_impersonation in airflow.cfg and don't pass run_as_user in task
Run a dag with multiple task (more than 1) which runs for more than 10sec

Anything else we need to know:
I suspect that the if-else statement here is causing the issue

Also I am trying to set run_as_user for all dags to be testuser using cluster policy, doing that also its giving the same error

def task_policy(task):
  task.run_as_user = 'testuser'

[ephraimbuddy]

Did this work in 2.1.1 @sudarshan2906? I mean the part that tasks are failing

[sudarshan2906]

Not sure. It started giving error when I enabled impersonationation in 2.1.2 as mentioned here -> http://airflow.apache.org/docs/apache-airflow/stable/security/workload.html

I haven't tried it in 2.1.1

[ephraimbuddy]

I have added a PR for the part that tasks are failing to run #17229

[sudarshan2906]

@ephraimbuddy This should also fix the task cluster policy which I am using?

def task_policy(task):
  task.run_as_user = 'testuser'

[ephraimbuddy]

@ephraimbuddy This should also fix the task cluster policy which I am using?

def task_policy(task):
  task.run_as_user = 'testuser'

I will check ...

[github-actions]

This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activity on this issue. Kindly asking to recheck the report against latest Airflow version and let us know if the issue is reproducible. The issue will be closed in next 30 days if no further activity occurs from the issue author.

[github-actions]

This issue has been closed because it has not received response from the issue author.