[SPARK-4642] Add description about spark.yarn.queue to running-on-YARN document. by tsudukim · Pull Request #3500 · apache/spark

tsudukim · 2014-11-28T01:05:24Z

Added descriptions about these parameters.

spark.yarn.queue

Modified description about the defalut value of this parameter.

spark.yarn.submit.file.replication

Added descriptions about these parameters. - spark.yarn.report.interval - spark.yarn.queue - spark.yarn.user.classpath.first - spark.yarn.scheduler.reporterThread.maxFailures Modified description about the defalut value of this parameter. - spark.yarn.submit.file.replication

SparkQA · 2014-11-28T01:07:49Z

Test build #23927 has started for PR 3500 at commit 88cac9b.

This patch merges cleanly.

SparkQA · 2014-11-28T02:25:57Z

Test build #23927 has finished for PR 3500 at commit 88cac9b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-11-28T02:26:00Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/23927/
Test PASSed.

sryza · 2014-12-01T07:25:06Z

Hi @tsudukim , spark.yarn.queue would be useful to document, but I think the others were left intentionally undocumented. Because the Spark-on-YARN internals still need some hardening, we'd like to avoid exposing some properties so that we can change them later.

@tgravescs

tgravescs · 2014-12-01T14:17:06Z

@sryza is correct. Most of those were intentionally left undocumented. If you have reason they need to be changed then we can revisit them sooner to make sure they are what we want and get them documented. Note there is a different pr up to fix up spark.yarn.user.classpath.first (#3233)

SparkQA · 2014-12-02T01:22:32Z

Test build #24013 has started for PR 3500 at commit 21cf624.

This patch merges cleanly.

tsudukim · 2014-12-02T01:24:43Z

@sryza and @tgravescs Thank you for your review. I removed them. Only spark.yarn.queue is added.

SparkQA · 2014-12-02T02:41:01Z

Test build #24013 has finished for PR 3500 at commit 21cf624.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-12-02T02:41:04Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24013/
Test PASSed.

sryza · 2014-12-03T02:43:24Z

A little better gramatically: "The name of the YARN queue to which the application is submitted."

sryza · 2014-12-03T02:43:41Z

Had one more nit. Otherwise this looks good to me.

tsudukim · 2014-12-03T19:59:15Z

@sryza Thank you for your helpful advice! Modified it.

SparkQA · 2014-12-03T20:02:58Z

Test build #24098 has started for PR 3500 at commit ce99655.

This patch merges cleanly.

andrewor14 · 2014-12-03T20:03:24Z

Ok merging this into master and 1.2 thank you @tsudukim and reviewers

andrewor14 · 2014-12-03T20:04:30Z

Actually @tsudukim could you update the PR description and the title? Now this only documents spark.yarn.queue, and it would be good if these reflect the actual changes here. I will merge this once you make the modifications.

tsudukim · 2014-12-03T21:03:25Z

@andrewor14 Thank you!
I modified description and title of this tickets from this web ui, but I'm not sure if it's right where I modified. Is this OK?

andrewor14 · 2014-12-03T21:14:23Z

Alright I'm merging this into master 1.2 and 1.1 thanks

SparkQA · 2014-12-03T21:18:13Z

Test build #24098 has finished for PR 3500 at commit ce99655.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

AmplabJenkins · 2014-12-03T21:18:16Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24098/
Test PASSed.

…N document. Added descriptions about these parameters. - spark.yarn.queue Modified description about the defalut value of this parameter. - spark.yarn.submit.file.replication Author: Masayoshi TSUZUKI <tsudukim@oss.nttdata.co.jp> Closes #3500 from tsudukim/feature/SPARK-4642 and squashes the following commits: ce99655 [Masayoshi TSUZUKI] better gramatically. 21cf624 [Masayoshi TSUZUKI] Removed intentionally undocumented properties. 88cac9b [Masayoshi TSUZUKI] [SPARK-4642] Documents about running-on-YARN needs update (cherry picked from commit 692f493) Signed-off-by: Andrew Or <andrew@databricks.com>

…N document. Added descriptions about these parameters. - spark.yarn.queue Modified description about the defalut value of this parameter. - spark.yarn.submit.file.replication Author: Masayoshi TSUZUKI <tsudukim@oss.nttdata.co.jp> Closes #3500 from tsudukim/feature/SPARK-4642 and squashes the following commits: ce99655 [Masayoshi TSUZUKI] better gramatically. 21cf624 [Masayoshi TSUZUKI] Removed intentionally undocumented properties. 88cac9b [Masayoshi TSUZUKI] [SPARK-4642] Documents about running-on-YARN needs update

Removed intentionally undocumented properties.

21cf624

sryza reviewed Dec 3, 2014
View reviewed changes

better gramatically.

ce99655

tsudukim changed the title ~~[SPARK-4642] Documents about running-on-YARN needs update~~ [SPARK-4642] Add description about spark.yarn.queue to running-on-YARN document. Dec 3, 2014

asfgit closed this in 692f493 Dec 3, 2014

Conversation

tsudukim commented Nov 28, 2014

Uh oh!

SparkQA commented Nov 28, 2014

Uh oh!

SparkQA commented Nov 28, 2014

Uh oh!

AmplabJenkins commented Nov 28, 2014

Uh oh!

sryza commented Dec 1, 2014

Uh oh!

tgravescs commented Dec 1, 2014

Uh oh!

SparkQA commented Dec 2, 2014

Uh oh!

tsudukim commented Dec 2, 2014

Uh oh!

SparkQA commented Dec 2, 2014

Uh oh!

AmplabJenkins commented Dec 2, 2014

Uh oh!

sryza Dec 3, 2014

Choose a reason for hiding this comment

Uh oh!

sryza commented Dec 3, 2014

Uh oh!

tsudukim commented Dec 3, 2014

Uh oh!

SparkQA commented Dec 3, 2014

Uh oh!

andrewor14 commented Dec 3, 2014

Uh oh!

andrewor14 commented Dec 3, 2014

Uh oh!

tsudukim commented Dec 3, 2014

Uh oh!

andrewor14 commented Dec 3, 2014

Uh oh!

SparkQA commented Dec 3, 2014

Uh oh!

AmplabJenkins commented Dec 3, 2014

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants