Skip to content

fix: Put tensor literal values in the dispatch cache key - #146

Open
georgeguimaraes wants to merge 3 commits into
elixir-nx:mainfrom
georgeguimaraes:fix/literal-values-in-cache-key
Open

georgeguimaraes wants to merge 3 commits into
elixir-nx:mainfrom
georgeguimaraes:fix/literal-values-in-cache-key

Conversation

@georgeguimaraes

Copy link
Copy Markdown
Member

The dispatch cache keys a tensor literal by its shape and type only. Two functions that differ only in the values of a literal get the same key, so the second function runs the program of the first:

a = Nx.tensor([1.0, 2.0, 3.0], backend: Nx.BinaryBackend)
b = Nx.tensor([10.0, 20.0, 30.0], backend: Nx.BinaryBackend)
zero = Nx.tensor([0.0, 0.0, 0.0], backend: EMLX.Backend)

Nx.Defn.jit(fn x -> Nx.add(x, a) end, compiler: EMLX).(zero)
# [1.0, 2.0, 3.0]
Nx.Defn.jit(fn x -> Nx.add(x, b) end, compiler: EMLX).(zero)
# [1.0, 2.0, 3.0], expected [10.0, 20.0, 30.0]

I found this in an ONNX interpreter, where graphs with the same structure have different constant tensors (window functions, resize weights, gather indices). There is no error, only wrong values.

With this change, the key also has an md5 of the literal's data. Template tensors have no data and keep the old key.

The dispatch cache keyed a tensor literal by its shape and type only. Two functions that differed only in the values of a literal got the same key, so the second function ran the program of the first and returned wrong results.

Now the key also has an md5 of the literal's data. Template tensors have no data and keep the old key.
Comment thread emlx/lib/emlx.ex Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants