Repository navigation

[SPARK-1132] Persisting Web UI through refactoring the SparkListener interface - #42

Closed

andrewor14 wants to merge 78 commits into

apache:masterfrom

andrewor14:master

Closed

[SPARK-1132] Persisting Web UI through refactoring the SparkListener interface#42
andrewor14 wants to merge 78 commits into
apache:masterfrom
andrewor14:master

Commits on Feb 4, 2014
Commits on Feb 5, 2014
- Update docs
  andrewor14
  committed
  13920c9
- Privatize methods
  andrewor14
  committed
  090544a
- Also privatize fields
  andrewor14
  committed
  3ddeb7e
Commits on Feb 11, 2014
- Merge github.com:apache/incubator-spark
  andrewor14
  committed
  e3ae35f
Commits on Feb 12, 2014
- Use JSON for ExecutorsUI
  andrewor14
  committed
  8e09306
- Merge github.com:apache/incubator-spark into persist-ui
  andrewor14
  committed
  10ed49d
- Add JSON Serializability for all SparkListenerEvent's
  andrewor14
  committed
  dcbd312
Commits on Feb 13, 2014
- ExecutorUI: render completely from JSON
  andrewor14
  committed
  bb222b9
Commits on Feb 14, 2014
- ExecutorUI: Serialize events rather than arbitary executor information
  andrewor14
  committed
  bf0b2e9
Commits on Feb 15, 2014
- Serialize events both to and from JSON (rather than just to)
  andrewor14
  committed
  de8a1cd
- Fix bugs for EnvironmentUI and ExecutorsUI
  andrewor14
  committed
  8a2ebe6
- Also deserialize new events
  andrewor14
  committed
  c4cd480
- BlockManagerUI: Add JSON functionality
  andrewor14
  committed
  d859efc
- JobProgressUI: Add JSON functionality
  andrewor14
  committed
  8add36b
Commits on Feb 16, 2014
- Add functionality of reconstructing a persisted UI from SparkContext
  andrewor14
  committed
  b3976b0
Commits on Feb 17, 2014
- Merge git://git.apache.org/incubator-spark into persist-ui
  andrewor14
  committed
  4dfcd22
- General refactor
  andrewor14
  committed
  f3fc13b
Commits on Feb 18, 2014
- Fix two major bugs
  andrewor14
  committed
  3fd584e
- Mostly naming, formatting, and code style changes
  andrewor14
  committed
  5ac906d
- Fix another major bug
  andrewor14
  committed
  904c729
- Add a gateway SparkListener to simplify event logging
  andrewor14
  committed
  4273013
Commits on Feb 19, 2014
- Fix BlockManagerUI bug by introducing new event
  andrewor14
  committed
  64d2ce1
- Explicitly register each UI listener rather than through some magic
  andrewor14
  committed
  6814da0
Commits on Feb 20, 2014
- Completely decouple SparkUI from SparkContext
  andrewor14
  committed
  d646df6
Commits on Feb 21, 2014
- Move all JSON de/serialization logic to JsonProtocol
  andrewor14
  committed
  e9e1c6d
Commits on Feb 22, 2014
- Formatting changes
  andrewor14
  committed
  70e7e7a
Commits on Feb 24, 2014
- More formatting changes, this time mainly for Json DSL
  andrewor14
  committed
  6631c02
Commits on Feb 26, 2014
- Embed storage status and RDD info in Task events
  andrewor14
  committed
  bbe3501
Commits on Feb 27, 2014
- Merge github.com:apache/spark
  andrewor14
  committed
  28019ca
- Migrate from lift-json to json4s-jackson
  andrewor14
  committed
  d1f4285
- Guard against TaskMetrics NPE + Fix tests
  andrewor14
  committed
  7b2f811
- Reflect RDD unpersist on UI
  andrewor14
  committed
  996d7a2
- Fix a couple of tests
  andrewor14
  committed
  472fd8a
Commits on Feb 28, 2014
- Clean up FileLogger
  andrewor14
  committed
  d47585f
- General clean up
  andrewor14
  committed
  faa113e
- Fix format fail
  andrewor14
  committed
  4d2fb0c
- Fix PySpark tests + remove sc.clearFiles/clearJars
  andrewor14
  committed
  0503e4b
- JobLogger: ID -> Id
  andrewor14
  committed
  5d2cec1
Commits on Mar 1, 2014
- Move SparkListenerBus out of DAGScheduler + Clean up
  andrewor14
  committed
  2981d61
- Address Patrick's comments
  andrewor14
  committed
  2fee310
- Fix 100 char format fail
  andrewor14
  committed
  cceff2b
Commits on Mar 3, 2014
- Add HDFS support for event logging
  andrewor14
  committed
  36b3e5d
Commits on Mar 4, 2014
- Fix HDFS flush behavior
  andrewor14
  committed
  03eda0b
- Fix bug: storage status was not reflected on UI in the local case
  andrewor14
  committed
  aef411c
- Use a more mnemonic path for logging
  andrewor14
  committed
  bb4c503
Commits on Mar 5, 2014
- Refactor out event logging and replaying logic from UI
  andrewor14
  committed
  18b256d
- Add new constructors for SparkUI
  andrewor14
  committed
  e375431
- Add a few configurable options to event logging
  andrewor14
  committed
  1ba3407
- Correct directory in log message "INFO: Logging events to <dir>"
  andrewor14
  committed
  291b2be
- Master UI - Rebuild SparkUI on application finish
  andrewor14
  committed
  4f69c4a
Commits on Mar 6, 2014
- Fix deprecated message for JavaSparkContext (minor)
  andrewor14
  committed
  176e68e
- Master UI - add support for reading compressed event logs
  andrewor14
  committed
  ca258a4
Commits on Mar 7, 2014
- Merge github.com:apache/spark
  andrewor14
  committed
  d6e3b4a
- Avoid logging all the blocks on each executor
  andrewor14
  committed
  d59da5f
Commits on Mar 10, 2014
- Treating SparkUI as a handler of MasterUI
  andrewor14
  committed
  b6eaea7
Commits on Mar 11, 2014
- Address Kay's and Patrick's comments
  andrewor14
  committed
  77ba283
- Imports, comments, and code formatting (minor)
  andrewor14
  committed
  dc93915
Commits on Mar 12, 2014
- Merge github.com:apache/spark (major)
  andrewor14
  committed
  d801d11
- Fix test fail
  andrewor14
  committed
  ac69ec8
- Imports, comments, and code formatting, once again (minor)
  andrewor14
  committed
  bf80e3d
Commits on Mar 13, 2014
- Address Patrick's comments
  andrewor14
  committed
  3456090
- Remove list of (TaskInfo, TaskMetrics) from StageInfo
  andrewor14
  committed
  c5c2c8f
Commits on Mar 14, 2014
- Remove now deprecated test
  andrewor14
  committed
  45fd84c
Commits on Mar 15, 2014
- Add unit tests + Fix bugs found through tests
  andrewor14
  committed
  650eb12
- Fix comment nits
  andrewor14
  committed
  6740e49
- Moved around functionality + renamed classes per Patrick
  andrewor14
  committed
  9e14f97
Commits on Mar 17, 2014
- Simplify static handler and BlockManager status update logic
  andrewor14
  committed
  f80bd31
Commits on Mar 18, 2014
- Clarify LiveListenerBus behavior + Add tests for new behavior
  andrewor14
  committed
  124429f
- Merge github.com:apache/spark
  andrewor14
  committed
  222adcd
Commits on Mar 19, 2014
- Scraps and pieces (no functionality change)
  andrewor14
  committed
  83af656
- Remove UI from map when removing application in Master
  andrewor14
  committed
  b8ba817
- Merge github.com:apache/spark
  andrewor14
  committed
  a1c5cd9
- Merge github.com:apache/spark
  andrewor14
  committed
  e5f14fa

Commits

Commits on Feb 4, 2014

Commits on Feb 5, 2014

Commits on Feb 11, 2014

Commits on Feb 12, 2014

Commits on Feb 13, 2014

Commits on Feb 14, 2014

Commits on Feb 15, 2014

Commits on Feb 16, 2014

Commits on Feb 17, 2014

Commits on Feb 18, 2014

Commits on Feb 19, 2014

Commits on Feb 20, 2014

Commits on Feb 21, 2014

Commits on Feb 22, 2014

Commits on Feb 24, 2014

Commits on Feb 26, 2014

Commits on Feb 27, 2014

Commits on Feb 28, 2014

Commits on Mar 1, 2014

Commits on Mar 3, 2014

Commits on Mar 4, 2014

Commits on Mar 5, 2014

Commits on Mar 6, 2014

Commits on Mar 7, 2014

Commits on Mar 10, 2014

Commits on Mar 11, 2014

Commits on Mar 12, 2014

Commits on Mar 13, 2014

Commits on Mar 14, 2014

Commits on Mar 15, 2014

Commits on Mar 17, 2014

Commits on Mar 18, 2014

Commits on Mar 19, 2014