Skip to content
Merged
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
f5d1bcf
Fixes #28452: "TaskInstances do not succeed when using enable_logging…
stavdav143 Nov 16, 2023
86ad14f
Merge branch 'apache:main' into main
stavdav143 Nov 16, 2023
cde67d1
Merge branch 'main' into main
stavdav143 Nov 18, 2023
bb031e7
Merge branch 'apache:main' into main
stavdav143 Nov 19, 2023
7653624
Merge branch 'apache:main' into main
stavdav143 Nov 26, 2023
f00d92b
Merge branch 'apache:main' into main
stavdav143 Dec 1, 2023
9c3e3b1
tty=True/False to be chosen by the user as it was the case before thi…
stavdav143 Dec 3, 2023
041e4c5
Follow=true for logs will always result in tasks not ending. This is …
stavdav143 Dec 3, 2023
c8063ac
service_logs called multiple times as we continuously poll the Docker…
stavdav143 Dec 3, 2023
af24289
service_logs called multiple times. In this test the tasks increase (…
stavdav143 Dec 3, 2023
048ba1e
Final commit of this PR marking the test case that validates logging …
stavdav143 Dec 3, 2023
6882e56
Formatting ruff
stavdav143 Dec 3, 2023
45beccf
Merge branch 'apache:main' into main
stavdav143 Dec 3, 2023
235e565
Reverting as Github actions don't run this test as a swarm node:
stavdav143 Dec 3, 2023
e481651
Merge branch 'main' of https://github.com/stavdav143/airflow into main
stavdav143 Dec 3, 2023
b4e9142
Merge branch 'main' into main
stavdav143 Dec 4, 2023
f823ceb
Merge branch 'apache:main' into main
stavdav143 Dec 5, 2023
9434117
Logging "since" timestamp to avoid memory issues
stavdav143 Dec 5, 2023
08dd84b
Formatting - Fix for #28452
stavdav143 Dec 6, 2023
41966a7
Fixes #28452: "TaskInstances do not succeed when using enable_logging…
stavdav143 Nov 16, 2023
dcdbbce
tty=True/False to be chosen by the user as it was the case before thi…
stavdav143 Dec 3, 2023
1c4422d
Follow=true for logs will always result in tasks not ending. This is …
stavdav143 Dec 3, 2023
6b445a3
service_logs called multiple times as we continuously poll the Docker…
stavdav143 Dec 3, 2023
6f615d6
service_logs called multiple times. In this test the tasks increase (…
stavdav143 Dec 3, 2023
ac8f4a9
Final commit of this PR marking the test case that validates logging …
stavdav143 Dec 3, 2023
2dc2813
Formatting ruff
stavdav143 Dec 3, 2023
965f6aa
Reverting as Github actions don't run this test as a swarm node:
stavdav143 Dec 3, 2023
231e984
Logging "since" timestamp to avoid memory issues
stavdav143 Dec 5, 2023
b73bb50
Formatting - Fix for #28452
stavdav143 Dec 6, 2023
3cf6bc4
Merge branch 'main' of https://github.com/stavdav143/airflow into main
stavdav143 Dec 8, 2023
ae4ff49
Merge branch 'apache:main' into main
stavdav143 Dec 8, 2023
a8526f3
Merge branch 'main' of https://github.com/stavdav143/airflow into main
stavdav143 Dec 8, 2023
5adc110
Fix bugs (#28452): Correctly assign last_line_logged, last_timestamp
stavdav143 Dec 8, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions airflow/providers/docker/operators/docker_swarm.py
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,12 @@ def stream_new_logs(last_line_logged, since=0):
match = re.match(r"(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.\d{6,}Z) (.*)", line)
timestamp, message = match.groups()
self.log.info(message)

if len(logs) == 0:
return last_line_logged, since

last_line_logged = line

# Floor nanoseconds to microseconds
last_timestamp = re.sub(r"(\.\d{6})\d+Z", r"\1Z", timestamp)
last_timestamp = datetime.strptime(last_timestamp, "%Y-%m-%dT%H:%M:%S.%fZ")
Expand Down