Skip to content

Spark-7063 when lz4 compression is used, it causes core dump - #6226

Closed
JihongMA wants to merge 11 commits into
apache:masterfrom
JihongMA:SPARK-7063-1
Closed

JihongMA wants to merge 11 commits into
apache:masterfrom
JihongMA:SPARK-7063-1

Conversation

@JihongMA

Copy link
Copy Markdown
Contributor

this fix is to solve one issue found in lz4 1.2.0, which caused core dump in Spark Core with IBM JDK. that issue is fixed in lz4 1.3.0 version.

@AmplabJenkins

Copy link
Copy Markdown

Can one of the admins verify this patch?

@rxin

rxin commented May 18, 2015

Copy link
Copy Markdown
Contributor

Can you update the pull request title to make it more informative?

@srowen

srowen commented May 18, 2015

Copy link
Copy Markdown
Member

ok to test

@AmplabJenkins

Copy link
Copy Markdown

Merged build triggered.

@AmplabJenkins

Copy link
Copy Markdown

Merged build started.

@SparkQA

SparkQA commented May 18, 2015

Copy link
Copy Markdown

Test build #32976 has started for PR 6226 at commit 0cca781.

@SparkQA

SparkQA commented May 18, 2015

Copy link
Copy Markdown

Test build #32976 has finished for PR 6226 at commit 0cca781.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins

Copy link
Copy Markdown

Merged build finished. Test PASSed.

@AmplabJenkins

Copy link
Copy Markdown

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32976/
Test PASSed.

@srowen

srowen commented May 18, 2015

Copy link
Copy Markdown
Member

@JihongMA I can merge this, and can even fix the title on merge, but would you fix it up if you have a moment please?

@JihongMA

Copy link
Copy Markdown
Contributor Author

I have just updated the title. Thanks!

@srowen

srowen commented May 18, 2015

Copy link
Copy Markdown
Member

@JihongMA the title of this PR has not been updated, please give it a title that matches the JIRA

@JihongMA JihongMA changed the title Spark-7063 Spark-7063 when lz4 compression is used, it causes core dump May 18, 2015
@JihongMA

Copy link
Copy Markdown
Contributor Author

sorry, it is done now.

@asfgit asfgit closed this in 6525fc0 May 18, 2015
jeanlyn pushed a commit to jeanlyn/spark that referenced this pull request May 28, 2015
this fix is to solve one issue found in lz4 1.2.0, which caused core dump in Spark Core with IBM JDK.  that issue is fixed in lz4 1.3.0 version.

Author: Jihong MA <linlin200605@gmail.com>

Closes apache#6226 from JihongMA/SPARK-7063-1 and squashes the following commits:

0cca781 [Jihong MA] SPARK-7063
4559ed5 [Jihong MA] SPARK-7063
daa520f [Jihong MA] SPARK-7063 upgrade lz4 jars
71738ee [Jihong MA] Merge remote-tracking branch 'upstream/master'
dfaa971 [Jihong MA] SPARK-7265 minor fix of the content
ace454d [Jihong MA] SPARK-7265 take out PySpark on YARN limitation
9ea0832 [Jihong MA] Merge remote-tracking branch 'upstream/master'
d5bf3f5 [Jihong MA] Merge remote-tracking branch 'upstream/master'
7b842e6 [Jihong MA] Merge remote-tracking branch 'upstream/master'
9c84695 [Jihong MA] SPARK-7265 address review comment
a399aa6 [Jihong MA] SPARK-7265 Improving documentation for Spark SQL Hive support
jeanlyn pushed a commit to jeanlyn/spark that referenced this pull request Jun 12, 2015
this fix is to solve one issue found in lz4 1.2.0, which caused core dump in Spark Core with IBM JDK.  that issue is fixed in lz4 1.3.0 version.

Author: Jihong MA <linlin200605@gmail.com>

Closes apache#6226 from JihongMA/SPARK-7063-1 and squashes the following commits:

0cca781 [Jihong MA] SPARK-7063
4559ed5 [Jihong MA] SPARK-7063
daa520f [Jihong MA] SPARK-7063 upgrade lz4 jars
71738ee [Jihong MA] Merge remote-tracking branch 'upstream/master'
dfaa971 [Jihong MA] SPARK-7265 minor fix of the content
ace454d [Jihong MA] SPARK-7265 take out PySpark on YARN limitation
9ea0832 [Jihong MA] Merge remote-tracking branch 'upstream/master'
d5bf3f5 [Jihong MA] Merge remote-tracking branch 'upstream/master'
7b842e6 [Jihong MA] Merge remote-tracking branch 'upstream/master'
9c84695 [Jihong MA] SPARK-7265 address review comment
a399aa6 [Jihong MA] SPARK-7265 Improving documentation for Spark SQL Hive support
nemccarthy pushed a commit to nemccarthy/spark that referenced this pull request Jun 19, 2015
this fix is to solve one issue found in lz4 1.2.0, which caused core dump in Spark Core with IBM JDK.  that issue is fixed in lz4 1.3.0 version.

Author: Jihong MA <linlin200605@gmail.com>

Closes apache#6226 from JihongMA/SPARK-7063-1 and squashes the following commits:

0cca781 [Jihong MA] SPARK-7063
4559ed5 [Jihong MA] SPARK-7063
daa520f [Jihong MA] SPARK-7063 upgrade lz4 jars
71738ee [Jihong MA] Merge remote-tracking branch 'upstream/master'
dfaa971 [Jihong MA] SPARK-7265 minor fix of the content
ace454d [Jihong MA] SPARK-7265 take out PySpark on YARN limitation
9ea0832 [Jihong MA] Merge remote-tracking branch 'upstream/master'
d5bf3f5 [Jihong MA] Merge remote-tracking branch 'upstream/master'
7b842e6 [Jihong MA] Merge remote-tracking branch 'upstream/master'
9c84695 [Jihong MA] SPARK-7265 address review comment
a399aa6 [Jihong MA] SPARK-7265 Improving documentation for Spark SQL Hive support
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants