Throw application-level error instead of killing raylet on task lineage not found.

[ericl]

Describe the problem

After #3499, LRU evicted lineage causes a raylet check failure. It would probably be better to raise an application-level exception instead of crashing the backend.

[robertnishihara]

Here's a simple example to reproduce the issue.

import ray
import numpy as np
import time

ray.init(object_store_memory=10**8)

@ray.remote
def f():
    return np.zeros(8 * 10**7, dtype=np.uint8)

x_id = f.remote()
ray.wait([x_id])

time.sleep(1)

# Delete the key from the task table to simulate LRU eviction from Redis.
[task_table_key] = ray.global_state.redis_clients[0].keys(b"*RAYLET_TASK*" + x_id.binary()[1:])
assert ray.global_state.redis_clients[0].delete(task_table_key) == 1

# Evict the object from the object store.
ray.wait([f.remote()])

# Force reconstruction of the task.
ray.get(x_id)

This causes the raylet to crash with

F0321 00:28:28.386531 2466321280 node_manager.cc:1895] Check failed: lineage_cache_.ContainsTask(task_id) Task metadata not found in either GCS or lineage cache. It may have been evicted by the redis LRU configuration. Consider increasing the memory allocation via ray.init(redis_max_memory=<max_memory_bytes>).