This repository was archived by the owner on Dec 15, 2021. It is now read-only.
Added support for Spark 2.4.0 and Scala 2.12. #69
Merged
Conversation
…since Databricks Avro library moved into Spark with small syntax changes.
Contributor
|
Just realized that |
Contributor
Author
|
It compiled fine, but I have yet to test the code change. Plan to do that today and will make adjustments if necessary. |
|
Neville, I think Databricks donated the Avro library to Spark. Adding it as a dependency like this seems to work for me: https://mvnrepository.com/artifact/org.apache.spark/spark-avro |
|
hi, thx for that, was the version published ? |
|
It wasn't yet, I think we will do it next week. |
lverzosa
added a commit
to lverzosa/spark-bigquery
that referenced
this pull request
Apr 26, 2019
…lightly since Databricks Avro library moved into Spark with small syntax changes. (spotify#69)" This reverts commit ee9136b.
lverzosa
added a commit
to lverzosa/spark-bigquery
that referenced
this pull request
May 1, 2019
* Updating versions * Revert "Update README.md" This reverts commit 55c75ec. * Revert "Added support for Spark 2.4.0 and Scala 2.12. Adjusted code slightly since Databricks Avro library moved into Spark with small syntax changes. (spotify#69)" This reverts commit ee9136b. * Revert "Update README.md (spotify#68)" This reverts commit a2d5a63. * Revert "Create daily partitioned table when necessary (spotify#67)" This reverts commit f5b4815. * Revert "Add maintenance mode notice" This reverts commit ab8fd9c. * Revert "Enable user to specify write options, hinting about issues with nested records (spotify#59)" This reverts commit 7f65455. * Revert "Configuration parameter for intermediate dataset id (spotify#63)" This reverts commit a0ecb3f. * Revert "Make a defensive copy to stop polution of sparkconf (spotify#60)" This reverts commit ffcd9a6. * Revert "Support setting query job priority (spotify#54)" This reverts commit 07e012a. * Fixing GoogleCredentials to use supplied file
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adjusted code slightly since Databricks Avro library moved into Spark with small syntax changes.