From aba737f9be520ec33d0841d718f8250fbfeb09e1 Mon Sep 17 00:00:00 2001 From: sandy Date: Fri, 8 Jul 2016 14:45:12 +0530 Subject: [PATCH 1/6] Add Asynchronous Actions documentation --- docs/programming-guide.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/docs/programming-guide.md b/docs/programming-guide.md index 2d6f7767d93ef..0244c085b6429 100644 --- a/docs/programming-guide.md +++ b/docs/programming-guide.md @@ -1099,6 +1099,34 @@ for details. +#### Asynchronous Actions +Spark provide asynchronous actions to execute two or more actions concurrently, these actions are execute asynchronously without blocking each other. +The following table lists some of the asynchronous actions supported by Spark. Refer to the RDD API doc ([Scala](api/scala/index.html#org.apache.spark.rdd.AsyncRDDActions),[Java](api/java/org/apache/spark/rdd/AsyncRDDActions.html)) + + + + + + + + + + + + + + + + + + + + + + + +
Asynchronous ActionMeaning
collectAsync() Returns a future for retrieving all the elements of the dataset as an array at the driver program. This is usually useful after a filter or other operation that returns a sufficiently small subset of the data.
countAsync() Returns a future for counting the number of elements in the RDD.
foreachAsync(func) Applies a function f to all elements of this RDD.
foreachPartitionAsync(func) Applies a function f to all elements of this RDD.
takeAsync(n) Returns a future for retrieving the first num elements of the RDD.
+ ### Shuffle operations Certain operations within Spark trigger an event known as the shuffle. The shuffle is Spark's From 4afcec803218376200559730e8f1c9873641c140 Mon Sep 17 00:00:00 2001 From: sandy Date: Mon, 11 Jul 2016 00:34:03 +0530 Subject: [PATCH 2/6] fix statement to more clear about blocking statement --- docs/programming-guide.md | 26 +------------------------- 1 file changed, 1 insertion(+), 25 deletions(-) diff --git a/docs/programming-guide.md b/docs/programming-guide.md index 0244c085b6429..5e335fc40160a 100644 --- a/docs/programming-guide.md +++ b/docs/programming-guide.md @@ -1099,33 +1099,9 @@ for details. -#### Asynchronous Actions -Spark provide asynchronous actions to execute two or more actions concurrently, these actions are execute asynchronously without blocking each other. +Spark provide asynchronous actions to execute two or more actions concurrently, these actions are execute asynchronously without blocking calling thread. The following table lists some of the asynchronous actions supported by Spark. Refer to the RDD API doc ([Scala](api/scala/index.html#org.apache.spark.rdd.AsyncRDDActions),[Java](api/java/org/apache/spark/rdd/AsyncRDDActions.html)) - - - - - - - - - - - - - - - - - - - - - - -
Asynchronous ActionMeaning
collectAsync() Returns a future for retrieving all the elements of the dataset as an array at the driver program. This is usually useful after a filter or other operation that returns a sufficiently small subset of the data.
countAsync() Returns a future for counting the number of elements in the RDD.
foreachAsync(func) Applies a function f to all elements of this RDD.
foreachPartitionAsync(func) Applies a function f to all elements of this RDD.
takeAsync(n) Returns a future for retrieving the first num elements of the RDD.
### Shuffle operations From 81d1f716e279dab5f5edfdc5eb3624f28d0387f1 Mon Sep 17 00:00:00 2001 From: sandy Date: Mon, 11 Jul 2016 11:26:11 +0530 Subject: [PATCH 3/6] remove some statement related to table --- docs/programming-guide.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/programming-guide.md b/docs/programming-guide.md index 5e335fc40160a..4650ef29e2a1c 100644 --- a/docs/programming-guide.md +++ b/docs/programming-guide.md @@ -1099,8 +1099,7 @@ for details. -Spark provide asynchronous actions to execute two or more actions concurrently, these actions are execute asynchronously without blocking calling thread. -The following table lists some of the asynchronous actions supported by Spark. Refer to the RDD API doc ([Scala](api/scala/index.html#org.apache.spark.rdd.AsyncRDDActions),[Java](api/java/org/apache/spark/rdd/AsyncRDDActions.html)) +Spark provide asynchronous actions to execute two or more actions concurrently, these actions are execute asynchronously without blocking calling thread. Refer to the RDD API doc for asynchronous actions ([Scala](api/scala/index.html#org.apache.spark.rdd.AsyncRDDActions),[Java](api/java/org/apache/spark/rdd/AsyncRDDActions.html)) ### Shuffle operations From db09b2c3bfb441cf20dfebd5a7def79252c49cec Mon Sep 17 00:00:00 2001 From: sandy Date: Mon, 11 Jul 2016 15:30:39 +0530 Subject: [PATCH 4/6] make changes in programming guide doc for asynchronous actions --- docs/programming-guide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/programming-guide.md b/docs/programming-guide.md index 4650ef29e2a1c..a96bf6b3502c2 100644 --- a/docs/programming-guide.md +++ b/docs/programming-guide.md @@ -1099,7 +1099,7 @@ for details. -Spark provide asynchronous actions to execute two or more actions concurrently, these actions are execute asynchronously without blocking calling thread. Refer to the RDD API doc for asynchronous actions ([Scala](api/scala/index.html#org.apache.spark.rdd.AsyncRDDActions),[Java](api/java/org/apache/spark/rdd/AsyncRDDActions.html)) +Spark RDD API also exposes asynchronous versions of some actions like foreach, collect, count etc. which return a handle immediately to the caller which can be used to wait for its completion. ### Shuffle operations From 2321d64f1d8e03c5f2fdf04ba64f3b183e5992c3 Mon Sep 17 00:00:00 2001 From: sandy Date: Mon, 11 Jul 2016 17:01:17 +0530 Subject: [PATCH 5/6] add explanation about async actions --- docs/programming-guide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/programming-guide.md b/docs/programming-guide.md index a96bf6b3502c2..3e046adefa8f8 100644 --- a/docs/programming-guide.md +++ b/docs/programming-guide.md @@ -1099,7 +1099,7 @@ for details. -Spark RDD API also exposes asynchronous versions of some actions like foreach, collect, count etc. which return a handle immediately to the caller which can be used to wait for its completion. +The Spark RDD API also exposes asynchronous versions of some actions, like foreachAsync for foreach, which immediately return a FutureAction to the caller instead of blocking on completion of the action. This can be used to manage or wait for the asynchronous execution of the action. ### Shuffle operations From 845a1233a0b458cdf056b8957afd99ea74be02da Mon Sep 17 00:00:00 2001 From: sandy Date: Mon, 11 Jul 2016 17:21:12 +0530 Subject: [PATCH 6/6] add back-tick-quote for code noun --- docs/programming-guide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/programming-guide.md b/docs/programming-guide.md index 3e046adefa8f8..590f41266c7c9 100644 --- a/docs/programming-guide.md +++ b/docs/programming-guide.md @@ -1099,7 +1099,7 @@ for details. -The Spark RDD API also exposes asynchronous versions of some actions, like foreachAsync for foreach, which immediately return a FutureAction to the caller instead of blocking on completion of the action. This can be used to manage or wait for the asynchronous execution of the action. +The Spark RDD API also exposes asynchronous versions of some actions, like `foreachAsync` for `foreach`, which immediately return a `FutureAction` to the caller instead of blocking on completion of the action. This can be used to manage or wait for the asynchronous execution of the action. ### Shuffle operations