Why the Memory Subsystem is Critical in Inferencing Chips