公开(公告)号：US20240232596A1

公开(公告)日：2024-07-11

申请号：US18393320

申请日：2023-12-21

Applicant: Imagination Technologies Limited

Inventor： Xiran Huang

IPC: G06N3/063

CPC classification number: G06N3/063

Abstract: Methods of implementing a neural network in hardware, the neural network including a plurality of layers and the layers being grouped into a plurality of layer groups, each layer group comprising one or more layers of the neural network that are processed in a single pass through the hardware. The layer groups are grouped into a plurality of tile groups, each tile group comprising a set of layer groups that are evaluated when executing the neural network. The method comprises pre-fetching a portion of the input data for a first layer group in a tile group into a buffer slot in on-chip memory; and subsequently releasing the buffer slot after output data for the first layer group has been written to memory.

公开(公告)号：US11875248B2

公开(公告)日：2024-01-16

申请号：US17500840

申请日：2021-10-13

Applicant: Imagination Technologies Limited

Inventor： Xiran Huang ,  Fernando Escobar

IPC: G06N3/00 ,  G06N3/063 ,  G06N3/04 ,  G06N3/08 ,  G11C11/54 ,  G06N3/10

CPC classification number: G06N3/063 ,  G06N3/04 ,  G06N3/08 ,  G06N3/10 ,  G11C11/54

Abstract: A multicore hardware implementation of a deep neural network includes a plurality of layers arranged in plurality of layer groups. The input data to the network comprises a multidimensional tensor including one or more traversed dimensions that are traversed by strides in at least one layer of a first layer group, and one or more non-traversed dimensions. If a size of the input data in a first dimension is greater than a threshold, the hardware implementation splits the input data for the first layer group into at least a first tile and a second tile, along the first dimension. If the size of the input data in the first dimension is not greater than the threshold, the hardware implementation splits the evaluation of the first layer group into at least a first pass and a second pass, along a dimension other than the first dimension.

公开(公告)号：US11853866B2

公开(公告)日：2023-12-26

申请号：US17500206

申请日：2021-10-13

Applicant: Imagination Technologies Limited

Inventor： Xiran Huang ,  Fernando Escobar

IPC: G06N3/00 ,  G06N3/063 ,  G06N3/04 ,  G06N3/08 ,  G11C11/54 ,  G06N3/10

CPC classification number: G06N3/063 ,  G06N3/04 ,  G06N3/08 ,  G06N3/10 ,  G11C11/54

Abstract: A multicore hardware implementation of a deep neural network includes a plurality of layers arranged in plurality of layer groups. The input data to the network comprises a multidimensional tensor including one or more traversed dimensions, being dimensions that are traversed by strides in at least one layer of a first layer group. The hardware implementation is configured to split the input data for the first layer group into at least a first tile and a second tile, along at least one of the traversed dimensions, each tile comprising a plurality of data elements in each of the one or more traversed dimensions. A first core is configured to evaluate multiple layer groups, depth-first, based on the first tile. A second core is configured to evaluate multiple layer groups, depth-first, based on the second tile.

公开(公告)号：US20220147832A1

公开(公告)日：2022-05-12

申请号：US17500206

申请日：2021-10-13

Applicant: Imagination Technologies Limited

Inventor： Xiran Huang ,  Fernando Escobar

IPC: G06N3/10 ,  G06N3/04

Abstract: A multicore hardware implementation of a deep neural network includes a plurality of layers arranged in plurality of layer groups. The input data to the network comprises a multidimensional tensor including one or more traversed dimensions, being dimensions that are traversed by strides in at least one layer of a first layer group. The hardware implementation is configured to split the input data for the first layer group into at least a first tile and a second tile, along at least one of the traversed dimensions, each tile comprising a plurality of data elements in each of the one or more traversed dimensions. A first core is configured to evaluate multiple layer groups, depth-first, based on the first tile. A second core is configured to evaluate multiple layer groups, depth-first, based on the second tile.

公开(公告)号：US12061972B2

公开(公告)日：2024-08-13

申请号：US17106892

申请日：2020-11-30

Applicant: Imagination Technologies Limited

Inventor： Xiran Huang ,  Cagatay Dikici

IPC: G06N3/063 ,  G06N3/04

CPC classification number: G06N3/063 ,  G06N3/04

Abstract: A hardware implementation of a neural network and a method of processing data in such a hardware implementation are disclosed. Input data for a plurality of layers of the network is processed in blocks, to generate respective blocks of output data. The processing proceeds depth-wise through the plurality of layers, evaluating all layers of the plurality of layers for a given block, before proceeding to the next block.

公开(公告)号：US20240232597A1

公开(公告)日：2024-07-11

申请号：US18393536

申请日：2023-12-21

Applicant: Imagination Technologies Limited

Inventor： Xiran Huang

IPC: G06N3/063

CPC classification number: G06N3/063

Abstract: A neural network is mapped to hardware by defining a plurality of layer groups, each layer group comprising one or more layers of the neural network that are processed in a single pass through the hardware. The layer groups are grouped into tile groups, each tile group comprising a set of layers groups that are evaluated when executing the neural network. Grouping the layer groups into a tile group comprises selecting a layer group that precedes a first layer group in the tile group and determining a number of times that input data to the layer group is read from memory. In response to this number exceeding a threshold, it is determined whether to merger the layer group into the tile group by determining an amount of space in on-chip memory required for storing pre-fetched input data and assessing one or more criteria relating to output data of the layer group.

公开(公告)号：US20220121914A1

公开(公告)日：2022-04-21

申请号：US17500840

申请日：2021-10-13

Applicant: Imagination Technologies Limited

Inventor： Xiran Huang ,  Fernando Escobar

IPC: G06N3/063 ,  G06N3/04 ,  G06N3/08 ,  G11C11/54

Abstract: A multicore hardware implementation of a deep neural network includes a plurality of layers arranged in plurality of layer groups. The input data to the network comprises a multidimensional tensor including one or more traversed dimensions that are traversed by strides in at least one layer of a first layer group, and one or more non-traversed dimensions. If a size of the input data in a first dimension is greater than a threshold, the hardware implementation splits the input data for the first layer group into at least a first tile and a second tile, along the first dimension. If the size of the input data in the first dimension is not greater than the threshold, the hardware implementation splits the evaluation of the first layer group into at least a first pass and a second pass, along a dimension other than the first dimension.

公开(公告)号：US20240354560A1

公开(公告)日：2024-10-24

申请号：US18763981

申请日：2024-07-03

Applicant: Imagination Technologies Limited

Inventor： Xiran Huang ,  Cagatay Dikici

IPC: G06N3/063 ,  G06N3/04

CPC classification number: G06N3/063 ,  G06N3/04

Abstract: A hardware implementation of a neural network and a method of processing data in such a hardware implementation are disclosed. Input data for a plurality of layers of the network is processed in blocks, to generate respective blocks of output data. The processing proceeds depth-wise through the plurality of layers, evaluating all layers of the plurality of layers for a given block, before proceeding to the next block.

公开(公告)号：US20220129741A1

公开(公告)日：2022-04-28

申请号：US17500415

申请日：2021-10-13

Applicant: Imagination Technologies Limited

Inventor： Xiran Huang ,  Fernando Escobar

IPC: G06N3/063 ,  G06N3/10

Abstract: A multicore hardware implementation of a deep neural network includes a plurality of layers arranged in plurality of layer groups. The input data to the network comprises a multidimensional tensor including one or more traversed dimensions that are traversed by strides in at least one layer of a first layer group, and one or more non-traversed dimensions. The hardware implementation splits the evaluation of the first layer group into a first pass and a second pass, along one of the traversed dimensions or one of the non-traversed dimensions. A first core evaluates the first layer group for the first pass, to generate a first portion of output data. A second core evaluates the first layer group for the second pass, to generate a second portion of output data. The hardware implementation combines the first portion of output data and the second portion of output data to produce the output data of the first layer group.

公开(公告)号：US20210174181A1

公开(公告)日：2021-06-10

申请号：US17106892

申请日：2020-11-30

Applicant: Imagination Technologies Limited

Inventor： Xiran Huang ,  Cagatay Dikici

IPC: G06N3/063 ,  G06N3/04

Abstract: A hardware implementation of a neural network and a method of processing data in such a hardware implementation are disclosed. Input data for a plurality of layers of the network is processed in blocks, to generate respective blocks of output data. The processing proceeds depth-wise through the plurality of layers, evaluating all layers of the plurality of layers for a given block, before proceeding to the next block.