[close #749] Fix health checking issue

[shiyuhang0]

What problem does this PR solve?

Issue Number: close #749

pingcap/tispark#2707

Problem Description: StoreHealthyChecker fails to check the status of TiFlash

What is changed and how does it work?

Use isMppalive RPC to probe the status of TiFlash.

Check List for Tests

Manual test：

I have tested this locally with TiSpark. TiSpark will request TiFlash using client-java

1. TiSpark will with the client-java before this PR will throw errors even if TiFlash is alive

23/05/20 15:31:12 WARN StoreHealthyChecker: fail to check TiFlash health, regrade as unhealthy
org.tikv.shade.io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after 1.999882667s. [closed=[], open=[[buffered_nanos=2005712916, waiting_for_connection]]]
	at org.tikv.shade.io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:287)
	at org.tikv.shade.io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:268)
	at org.tikv.shade.io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:175)
	at org.tikv.common.region.StoreHealthyChecker.checkTiFlashHealth(StoreHealthyChecker.java:99)
	at org.tikv.common.region.StoreHealthyChecker.checkStoreHealth(StoreHealthyChecker.java:86)
	at org.tikv.common.region.StoreHealthyChecker.run(StoreHealthyChecker.java:154)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
23/05/20 15:31:13 WARN RangeSplitter: Cannot find valid store on TiFlash(engine=tiflash)

2. TiSpark success with the client-java build by this pr

Side effects

• Possible performance regression, WHY: TBD
• Increased code complexity, WHY: TBD
• Breaking backward compatibility, WHY: TBD
• NO side effects

Related changes

• Need to cherry-pick to the release branch
• Need to update the documentation
• Need to be included in the release note
• NO related changes

[codecov]

Codecov Report

Patch coverage: 26.08% and project coverage change: -0.23 ⚠️

Comparison is base (8d70ed2) 38.15% compared to head (c08ea89) 37.92%.

Additional details and impacted files

@@             Coverage Diff              @@
##             master     #748      +/-   ##
============================================
- Coverage     38.15%   37.92%   -0.23%     
+ Complexity     1614     1611       -3     
============================================
  Files           278      278              
  Lines         17493    17511      +18     
  Branches       1989     1991       +2     
============================================
- Hits           6674     6641      -33     
- Misses        10149    10210      +61     
+ Partials        670      660      -10     

Impacted Files	Coverage Δ
...va/org/tikv/common/region/StoreHealthyChecker.java	60.63% <13.33%> (-12.78%)	⬇️
...ain/java/org/tikv/common/region/RegionManager.java	79.76% <50.00%> (-0.35%)	⬇️
src/main/java/org/tikv/common/region/TiStore.java	60.00% <50.00%> (-1.54%)	⬇️

... and 8 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[pingyu]

Rest LGTM.

[pingyu]

LGTM~

[shiyuhang0]

/cherry-pick release-3.3

[ti-chi-bot]

In response to a cherrypick label: new pull request created to branch release-3.3: #753.