node_grpc_compile fails without further error message

[haikalpribadi]

I've followed the instructions in README.md but node_grpc_compile simply fails without any detailed error message. The error simply says "<Filename>_grpc_pb.js" was not created, for 2 out of 4 of my files, and I can't get more error message even when using --verbose_failures.

I've isolated the issue on a branch on my repo for you to reproduce it:
https://github.com/haikalpribadi/grakn/tree/bazel-init-pubref-rules-proto

Once you clone the repo, run bazel build //protocol:client-node-proto to reproduce the problem.

[haikalpribadi]

I've added verbose = 3 to node_grpc_compile, and it just dumps out lots of lines of file paths, but no useful information on where it breaks, unfortunately. You can find the output here: https://pastebin.com/cX4FWUp5

[promiseofcake]

It could have something to do with your proto layout, try to follow the Bazel proto_library best practices and see if that helps: https://blog.bazel.build/2017/02/27/protocol-buffers.html

[haikalpribadi]

do you mean that I shouldn't use srcs = glob(["proto/*.proto"]) in proto_library and split the rules per proto file, @promiseofcake?

[promiseofcake]

Correct, or at least begin by placing the BUILD file in the directory with the protos. We have historically ran into issues with node proto paths by using a non-optimal layout.

[pcj]

Agreed definitely an anti-pattern to glob protos into one rule. Ideally one proto file per proto-library. I prefer each in own subdirectory with one proto file and one BUILD.bazel file.

[haikalpribadi]

Amazing - it worked! I removed the srcs = glob(["proto/*.proto"]) in proto_library, and made a proto_library rule for each proto file, all the <lang>_grpc_compile works now!

Thank you, @promiseofcake and @pcj ! :)

(I do have a different issue now, but I'll post that separately).

[haikalpribadi]

Hi @pcj and @promiseofcake , sorry for reopening this issue. I've tried following the best practice blog you linked.

I've declared the proto_librarys to be:

proto_library(
    name = "concept-proto",
    srcs = ["Concept.proto"],
)
proto_library(
    name = "answer-proto",
    srcs = ["Answer.proto"],
    deps = [":concept-proto"],
)
proto_library(
    name = "session-proto",
    srcs = ["Session.proto"],
    deps = [":concept-proto", ":answer-proto"],
)

And when I declare the java_grpc_library like this (as recommended by the blog):

java_grpc_compile(
    name = "client-java-proto",
    deps = ["//protocol/session:session-proto"],
)

Dependants of :client-java-proto complains for not being able to find the GRPC-generated files for Answer.proto and Concept.proto. So I added them to the java_grpc_library like the following:

java_grpc_compile(
    name = "client-java-proto",
    deps = [
        "//protocol/session:session-proto",
        "//protocol/session:answer-proto",
        "//protocol/session:concept-proto",
    ]
)

And then everything works for Java - good.

However, when I try to do the same for node_grpc_compile:

node_grpc_compile(
    name = "client-node-proto",
    deps = [
        "//protocol/session:session-proto",
        "//protocol/session:answer-proto",
        "//protocol/session:concept-proto",
    ]
)

bazel build //protocol:client-node-proto fails with error message:

ERROR: /protocol/BUILD:47:1: output 'protocol/client-node-proto/protocol/session/Answer_grpc_pb.js' was not created
ERROR: /protocol/BUILD:47:1: output 'protocol/client-node-proto/protocol/session/Concept_grpc_pb.js' was not created
ERROR: /protocol/BUILD:47:1: not all outputs were created or valid

I even try to just compile node_grpc_library for Concept.proto:

node_grpc_compile(
    name = "client-node-proto",
    deps = ["//protocol/session:concept-proto"]
)

And it also fails!

ERROR: /protocol/BUILD:47:1: output 'protocol/client-node-proto/protocol/session/Concept_grpc_pb.js' was not created
ERROR: /protocol/BUILD:47:1: not all outputs were created or valid

Doing verbose = 3 and --verbose_failures does not produce any useful information than just a list of files, which you can see the console output here: https://pastebin.com/DMpSpiMu

I've narrowed it down to the fact that Concept.proto does not have a service declared in the proto file (which is by design).

Can you help figure out what's going on and if I'm doing something wrong?

I've pushed to code onto haikalpribadi/grakn branch bazel-init-node-grpc-compile-fails. You can reproduce the problem by running bazel buid //protocol:client-node-proto.

Thank you so much!

[promiseofcake]

I think this is an underlying problem with the node plugin for gRPC, see this issue we opened: grpc/grpc#10646