Repository navigation

[SPARK-39489][CORE] Improve event logging JsonProtocol performance by using Jackson instead of Json4s - #36885

Closed

JoshRosen wants to merge 30 commits into

apache:masterfrom

JoshRosen:remove-json4s-from-jsonprotocol

Closed

[SPARK-39489][CORE] Improve event logging JsonProtocol performance by using Jackson instead of Json4s#36885
JoshRosen wants to merge 30 commits into
apache:masterfrom
JoshRosen:remove-json4s-from-jsonprotocol

Commits on Jun 15, 2022
- Remove from write path.
  JoshRosen
  committed
  7c3132b
- Add sparkEventToJsonString helper method.
  JoshRosen
  committed
  27b84a8
- Remove from read path.
  JoshRosen
  committed
  4b30f5e
- WIP towards fixing SQL suites.
  JoshRosen
  committed
  6175d81
- Keep using Json4s for ResourceInfo because it's (unfortunately) part of the DeveloperApi
  JoshRosen
  committed
  685ddf8
- Fix writing of "Scope" field.
  JoshRosen
  committed
  9fda82b
- Fix "Task Metrics" field
  JoshRosen
  committed
  22d3884
- Fix executorMetricsFromJson: old Spark versions don't write these metrics.
  JoshRosen
  committed
  77a3fc4
Commits on Jun 16, 2022
- roll back unnecessary whitespace change
  JoshRosen
  committed
  7d257a6
Commits on Jun 17, 2022
- Close ByteArrayOuputStream
  JoshRosen
  committed
  1cf8edd
- Fix Scala 2.13 compilation
  JoshRosen
  committed
  631dfc8
Commits on Jun 18, 2022
- Fix handling of null `properties` field when writing job and stage start events.
  JoshRosen
  committed
  c5e59b6
Commits on Jun 27, 2022
- Backwards-compatibility test for EnvironmentUpdate
  JoshRosen
  committed
  698bfbc
- Backwards-compatibility for TaskInfo.partitionId
  JoshRosen
  committed
  12dfddc
- Backwards compatibilty: RDDInfo DeterministicLevel
  JoshRosen
  committed
  b2bcf0b
- Backward compatibility: RDDInfo Barrier
  JoshRosen
  committed
  469eb1d
- Add missing assertions in existing "InputMetrics backward compatibility" test
  JoshRosen
  committed
  b628d97
- Add missing assertions in existing "OutputMetrics backward compatibility" test
  JoshRosen
  committed
  7805c13
- Strengten backwards compatibility tests for TaskMetrics
  JoshRosen
  committed
  6629d0d
- Backwards compatibility for ExecutorInfo
  JoshRosen
  committed
  fd50562
- Backwards compatibility: TaskKilled
  JoshRosen
  committed
  fae84ee
- Backwards compatibility for "Remote Bytes Read To Disk" metric
  JoshRosen
  committed
  ccdba66
Commits on Jun 28, 2022
- Variable rename
  JoshRosen
  committed
  e491ff0
- Merge remote-tracking branch 'apache/master' into remove-json4s-from-jsonprotocol
  JoshRosen
  committed
  70a1df4
Commits on Jun 29, 2022
- Scala 2.13 fixes
  JoshRosen
  committed
  277755a
- More 2.13 fixes
  JoshRosen
  committed
  f4cf51d
Commits on Jun 30, 2022
Commits on Jul 2, 2022
- Add TODO comment around Json4s in ResourceInformation.toJson
  JoshRosen
  committed
  f324e5e

Commits

Commits on Jun 15, 2022

Commits on Jun 16, 2022

Commits on Jun 17, 2022

Commits on Jun 18, 2022

Commits on Jun 27, 2022

Commits on Jun 28, 2022

Commits on Jun 29, 2022

Commits on Jun 30, 2022

Commits on Jul 2, 2022