~/bend-docscommunity

kernel_spatial_packing.bend checks

raw source on the hub · import stelliferous@0.0.2.0/kernel_spatial_packing.bend as Kernel_spatial_packing

One packer for every output width. A tile holds eight consecutive output positions; its entries follow the reduction order: channel, kernel row, kernel column. Only a new tile divides: it finds the output row and column of its first position. A tile inside one or two output rows runs three nested loops whose counters give the input row and column of lane 0, and reads the lanes of each row at one stride. A tile that spans more rows (an output narrower than eight) locates every lane from its position, entry by entry.

4 imports
import Base
import ./traversal_loop.bend as Loop
import ./kernel_packing_samples.bend as PackingSamples
import ./kernel_tile.bend as Tile

Types

type PixelBlock source · line 21 · raw

Data

type WrapBlock source · line 39 · raw

Data

Lane 0 is output column offset. A lane past the last output column lies in the next output row, whose input row is next_row.

type RowState source · line 70 · raw

@-ownership:Quant -> @-InputStorage:Kind(ownership) -> @-OutputStorage:Kind(ownership) -> Kind(ownership)

The entries of a tile whose lane 0 is output column offset of output row row of an output output_width wide. block reads the eight lanes of an entry.

type CrossingState source · line 105 · raw

@-ownership:Quant -> @-InputStorage:Kind(ownership) -> @-OutputStorage:Kind(ownership) -> Kind(ownership)

The entries of a tile that crosses an output row, by reduction index.

type TileState source · line 124 · raw

@-ownership:Quant -> @-InputStorage:Kind(ownership) -> @-OutputStorage:Kind(ownership) -> Kind(ownership)

The tiles from output position position, eight positions each.

type Call source · line 170 · raw

Data

One packing call shared by raw convolution and certified Buffer entry points.

Definitions

def masked source · line 14 · raw

@inside:Bool -> @pair:Pair(Array<F32>, F32) -> Pair(Array<F32>, F32)

def pixel source · line 18 · raw

@input_storage:Array<F32> -> @+input_row:U32 -> @+input_column:U32 -> @+channel_base:U32 -> @+input_height:U32 -> @+input_width:U32 -> Pair(Array<F32>, F32)

def read_block_pixel source · line 24 · raw

@input:Array<F32> -> @context:PixelBlock -> @index:Nat -> Pair(Array<F32>, F32)

def block source · line 29 · raw

@input:Array<F32> -> @row:U32 -> @column:U32 -> @channel_base:U32 -> @height:U32 -> @width:U32 -> @stride:U32 -> @next_row:U32 -> @offset:U32 -> @output_width:U32 -> @kernel_column:U32 -> @padding:U32 -> Pair(Array<F32>, 0x9be3b13bf249759bc81e1958bcd1a4c0/kernel_tile.Tile8)

Eight lanes of one output row.

def wrap_pixel source · line 42 · raw

@same:Bool -> @input:Array<F32> -> @row:U32 -> @column:U32 -> @channel_base:U32 -> @height:U32 -> @width:U32 -> @+stride:U32 -> @next_row:U32 -> @offset:U32 -> @output_width:U32 -> @kernel_column:U32 -> @padding:U32 -> @+lane:U32 -> Pair(Array<F32>, F32)

def wrap_lane source · line 48 · raw

@input:Array<F32> -> @row:U32 -> @column:U32 -> @channel_base:U32 -> @height:U32 -> @width:U32 -> @stride:U32 -> @next_row:U32 -> @+offset:U32 -> @+output_width:U32 -> @kernel_column:U32 -> @padding:U32 -> @+lane:U32 -> Pair(Array<F32>, F32)

def read_wrap_pixel source · line 52 · raw

@input:Array<F32> -> @context:WrapBlock -> @index:Nat -> Pair(Array<F32>, F32)

def wrap_block source · line 56 · raw

@input:Array<F32> -> @row:U32 -> @column:U32 -> @channel_base:U32 -> @height:U32 -> @width:U32 -> @stride:U32 -> @next_row:U32 -> @offset:U32 -> @output_width:U32 -> @kernel_column:U32 -> @padding:U32 -> Pair(Array<F32>, 0x9be3b13bf249759bc81e1958bcd1a4c0/kernel_tile.Tile8)

def Block source · line 65 · raw

Type

def RowTiles source · line 74 · raw

Type

def store_block source · line 77 · raw

@output:Array<0x9be3b13bf249759bc81e1958bcd1a4c0/kernel_tile.Tile8> -> @+index:U32 -> @channel:U32 -> @kernel_row:U32 -> @kernel_column:U32 -> @row:U32 -> @offset:U32 -> @output_width:U32 -> @height:U32 -> @width:U32 -> @stride:U32 -> @padding:U32 -> @taps:Nat -> @pair:Pair(Array<F32>, 0x9be3b13bf249759bc81e1958bcd1a4c0/kernel_tile.Tile8) -> RowTiles

def next_kernel_row source · line 88 · raw

@state:RowTiles -> RowTiles

def next_channel source · line 96 · raw

@state:RowTiles -> RowTiles

def Crossings source · line 108 · raw

Type

def store_crossing source · line 111 · raw

@output:Array<0x9be3b13bf249759bc81e1958bcd1a4c0/kernel_tile.Tile8> -> @+index:U32 -> @reduction:U32 -> @position:U32 -> @height:U32 -> @width:U32 -> @kernel:U32 -> @stride:U32 -> @padding:U32 -> @output_width:U32 -> @pair:Pair(Array<F32>, 0x9be3b13bf249759bc81e1958bcd1a4c0/kernel_tile.Tile8) -> Crossings

def crossing_step source · line 116 · raw

@state:Crossings -> Crossings

def Tiled source · line 128 · raw

Type

def after_row_tile source · line 131 · raw

@position:U32 -> @kernel:U32 -> @channels:Nat -> @inner:Nat -> @state:RowTiles -> Tiled

def after_crossing source · line 135 · raw

@channels:Nat -> @taps:Nat -> @inner:Nat -> @state:Crossings -> Tiled

def tile source · line 140 · raw

@within:Bool -> @wraps:Bool -> @input:Array<F32> -> @output:Array<0x9be3b13bf249759bc81e1958bcd1a4c0/kernel_tile.Tile8> -> @index:U32 -> @+position:U32 -> @height:U32 -> @width:U32 -> @kernel:U32 -> @stride:U32 -> @padding:U32 -> @+output_width:U32 -> @+channels:Nat -> @taps:Nat -> @+inner:Nat -> Tiled

A tile inside one output row, one that ends in the next row, and any other.

def tile_step source · line 154 · raw

@state:Tiled -> Tiled

Eight positions from output column c stay in one output row when c + 8 is at most the output width, and in two when it is at most twice the width.

def finish source · line 159 · raw

@state:Tiled -> Pair(Array<F32>, Array<0x9be3b13bf249759bc81e1958bcd1a4c0/kernel_tile.Tile8>)

def pack source · line 164 · raw

@count:Nat -> @start:U32 -> @input:Array<F32> -> @output:Array<0x9be3b13bf249759bc81e1958bcd1a4c0/kernel_tile.Tile8> -> @height:U32 -> @width:U32 -> @+kernel:U32 -> @stride:U32 -> @padding:U32 -> @output_width:U32 -> @+inner:U32 -> Pair(Array<F32>, Array<0x9be3b13bf249759bc81e1958bcd1a4c0/kernel_tile.Tile8>)

count entries, inner per tile, from output position start.

def run source · line 173 · raw

@call:Call -> @input:Array<F32> -> @output:Array<0x9be3b13bf249759bc81e1958bcd1a4c0/kernel_tile.Tile8> -> Pair(Array<F32>, Array<0x9be3b13bf249759bc81e1958bcd1a4c0/kernel_tile.Tile8>)

Templates

template column_step source · line 82 · raw

@-block:Block -> @state:RowTiles -> RowTiles

template kernel_row_step source · line 92 · raw

@-block:Block -> @state:RowTiles -> RowTiles

template channel_step source · line 100 · raw

@-block:Block -> @state:RowTiles -> RowTiles