If an Inference IP supplier or Inference Accelerator Chip supplier offers a benchmark, it is probably ResNet-50. As a result, it might seem logical to use ResNet-50 to compare inference offerings. If ...

Much has been written about the computational complexity of inference acceleration: very large matrix multiplies for fully-connected layers and huge numbers of 3×3 convolutions across megapixel images ...

It’s important to understand that an inference accelerator is a completely new kind of chip, with many unknowns for the broader market. In our industry, there’s a learning curve for everything, from ...

Today AI chip startup Groq announced that their new Tensor processor has achieved 21,700 inferences per second (IPS) for ResNet-50 v2 inference. Groq’s level of inference performance exceeds that of ...

There is much at stake in the world of datacenter inference and while the market has not yet decided its winners, there are finally some new metrics in the bucket to aid decision-making. Interpreting ...

A few months back, I wrote about the MLPerf consortium and the release of its Inference v0.5 benchmark. MLPerf had previously disclosed some performance results from its Training v0.6 benchmark, but ...

Processor hardware for machine learning is in their early stages but it already taking different paths. And that mainly has to do with dichotomy between training and inference. Not only do these two ...