[TIR][USMP] Augmenting the algo interface with memory pressure#9649
Conversation
manupak
requested review from
Hzfengsy,
ZihengJiang,
areusch,
comaniac,
icemelon,
jroesch,
junrushao,
kparzysz-quic,
masahi,
merrymercy,
tqchen,
vinx13 and
yzhliu
as code owners
This commit adds memory pressue to be an arugment to the USMP algorithm interface as certain iterative algorithms could use this as a guide determine the termination criteria. Change-Id: I3fb5eea3fe5ba43e68c23625d411e557f6dd89a3
manupak
force-pushed
the
usmp_algo_interface_memory_pressure
branch
from
016de4e to
4a62348
Compare
| Map<BufferInfo, tir::Stmt> buffer_info_stmts; | ||
| /*! \brief This represent maximum amount of memory being used at | ||
| * any point of time in the inference. This value is largely the | ||
| * best allocation an algorithm could achieve. Due to |
There was a problem hiding this comment.
A naive question which I think I might already know the answer to... why is the 'best' allocation considered one that is closer to the maximum memory pressure rather than say the least amount of memory used?
There was a problem hiding this comment.
Thanks! @lhutton1 .
The maximum memory used is the summation of all conflicted buffers/tensors at any point in the lifetime of the inference. No matter how a memory allocation perform, it will not beat this value because those tensors "have" to be live with each other. Therefore this is the value a memory planning algorithm strives to achieve that corresponds to the theoretical "peak" memory usage
Then it begs the question, why would an algorithm will not achieve this ?
That is due to the complexity of the problem and it might not be able to figure out an offset ordering in tractable time that meets the above value. It might as well be there does not exist static a set of offsets that tightly bind the tensors to achieve that.
Does that help ?
|
Pinging @leandron. PTAL when you have some time. |
…e#9649) This commit adds memory pressue to be an arugment to the USMP algorithm interface as certain iterative algorithms could use this as a guide determine the termination criteria. Change-Id: I3fb5eea3fe5ba43e68c23625d411e557f6dd89a3
…e#9649) This commit adds memory pressue to be an arugment to the USMP algorithm interface as certain iterative algorithms could use this as a guide determine the termination criteria. Change-Id: I3fb5eea3fe5ba43e68c23625d411e557f6dd89a3
…e#9649) This commit adds memory pressue to be an arugment to the USMP algorithm interface as certain iterative algorithms could use this as a guide determine the termination criteria. Change-Id: I3fb5eea3fe5ba43e68c23625d411e557f6dd89a3
…e#9649) This commit adds memory pressue to be an arugment to the USMP algorithm interface as certain iterative algorithms could use this as a guide determine the termination criteria. Change-Id: I3fb5eea3fe5ba43e68c23625d411e557f6dd89a3
…e#9649) This commit adds memory pressue to be an arugment to the USMP algorithm interface as certain iterative algorithms could use this as a guide determine the termination criteria. Change-Id: I3fb5eea3fe5ba43e68c23625d411e557f6dd89a3
3 participants
This commit adds memory pressue to be an arugment to
the USMP algorithm interface as certain iterative algorithms
could use this as a guide determine the termination
criteria.