Skip to content

TiStore: add implement Serializable#251

Merged
marsishandsome merged 1 commit intotikv:masterfrom
liangyouze:feature/TiStore
Aug 9, 2021
Merged

TiStore: add implement Serializable#251
marsishandsome merged 1 commit intotikv:masterfrom
liangyouze:feature/TiStore

Conversation

@liangyouze
Copy link
Copy Markdown
Contributor

What problem does this PR solve?

When I use it in Tispark, the following exception will be thrown

Caused by: java.io.NotSerializableException: org.tikv.common.region.TiStore
Serialization stack:
	- object not serializable (class: org.tikv.common.region.TiStore, value: org.tikv.common.region.TiStore@1f547af8)
	- writeObject data (class: java.util.ArrayList)
	- object (class java.util.ArrayList, [org.tikv.common.region.TiStore@1f547af8, org.tikv.common.region.TiStore@701c482e, org.tikv.common.region.TiStore@4738131e])
	- field (class: org.tikv.common.region.TiRegion, name: stores, type: interface java.util.List)
	- object (class org.tikv.common.region.TiRegion, {Region[2009] ConfVer[5] Version[8] Store[4] KeyRange[]:[bNmuyJQt]})
	- writeObject data (class: scala.collection.immutable.List$SerializationProxy)
	- object (class scala.collection.immutable.List$SerializationProxy, scala.collection.immutable.List$SerializationProxy@416d90f0)
	- writeReplace data (class: scala.collection.immutable.List$SerializationProxy)
	- object (class scala.collection.immutable.$colon$colon, List({Region[2009] ConfVer[5] Version[8] Store[4] KeyRange[]:[bNmuyJQt]}, {Region[2] ConfVer[5] Version[8] Store[1] KeyRange[bNmuyJQt]:[]}))
	- field (class: org.apache.spark.rawkvbulkload.TiReginSplitPartitionerV2, name: orderedRegions, type: class scala.collection.immutable.List)
	- object (class org.apache.spark.rawkvbulkload.TiReginSplitPartitionerV2, org.apache.spark.rawkvbulkload.TiReginSplitPartitionerV2@3c98981e)
	- field (class: org.apache.spark.rawkvbulkload.RawKVBulkLoad, name: partitioner, type: class org.apache.spark.rawkvbulkload.TiReginSplitPartitionerV2)
	- object (class org.apache.spark.rawkvbulkload.RawKVBulkLoad, org.apache.spark.rawkvbulkload.RawKVBulkLoad@6a38e3d1)
	- element of array (index: 0)
	- array (class [Ljava.lang.Object;, size 1)
	- field (class: java.lang.invoke.SerializedLambda, name: capturedArgs, type: class [Ljava.lang.Object;)
	- object (class java.lang.invoke.SerializedLambda, SerializedLambda[capturingClass=class org.apache.spark.rawkvbulkload.RawKVBulkLoad, functionalInterfaceMethod=scala/Function1.apply:(Ljava/lang/Object;)Ljava/lang/Object;, implementation=invokeStatic org/apache/spark/rawkvbulkload/RawKVBulkLoad.$anonfun$bulkLoad$5$adapted:(Lorg/apache/spark/rawkvbulkload/RawKVBulkLoad;Lscala/collection/Iterator;)Ljava/lang/Object;, instantiatedMethodType=(Lscala/collection/Iterator;)Ljava/lang/Object;, numCaptured=1])
	- writeReplace data (class: java.lang.invoke.SerializedLambda)
	- object (class org.apache.spark.rawkvbulkload.RawKVBulkLoad$$Lambda$2335/456897159, org.apache.spark.rawkvbulkload.RawKVBulkLoad$$Lambda$2335/456897159@7dd7ec56)
	at org.apache.spark.serializer.SerializationDebugger$.improveException(SerializationDebugger.scala:41)
	at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:47)
	at org.apache.spark.serializer.JavaSerializerInstance.serialize(JavaSerializer.scala:101)
	at org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:413)
	... 23 more

What is changed and how it works?

The Class TiStore should implement Serializable

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Code changes

  • Has exported function/method change
  • Has exported variable/fields change
  • Has interface methods change
  • Has persistent data change

Side effects

  • Possible performance regression
  • Increased code complexity
  • Breaking backward compatibility

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation
  • Need to be included in the release note

@marsishandsome
Copy link
Copy Markdown
Collaborator

please add sign-off

@marsishandsome
Copy link
Copy Markdown
Collaborator

/run-all-tests

Signed-off-by: liangyouze <liangyouze876@gmail.com>
Copy link
Copy Markdown
Collaborator

@marsishandsome marsishandsome left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@marsishandsome
Copy link
Copy Markdown
Collaborator

/merge

@ti-srebot
Copy link
Copy Markdown
Collaborator

/run-all-tests

@ti-srebot
Copy link
Copy Markdown
Collaborator

@liangyouze merge failed.

@marsishandsome
Copy link
Copy Markdown
Collaborator

/run-all-tests

@marsishandsome marsishandsome merged commit 68e097d into tikv:master Aug 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants