Data Format¶
As we all know, input/output tensors in CNN model have data format like
NHWC
(tensorflow) or NCHW
(caffe), but there is no data format for non-CNN model.
However, in MACE, CNN model run on CPU
with float
type using NCHW
data format,
while the others using NHWC
data format.
To support all models, so there are some concepts in MACE
you should know.
Source Data Format¶
Source Data Format(src_df
for short) stands for the original data format where
the model come from. For example, if you use caffe, the src_df
is NCHW
.
We need this data format because some operators(Reshape etc.) are
related to the data format.
Operators Partition¶
Generally, operators could be divided into 2 categories
based on whether the operator needs inputs with fixed data format(NHWC
or NCHW
),
one is the operators whose inputs have fixed data format(like convolution
),
the other is the operators whose inputs should be the same with source framework.
Since the data format the operators need in MACE may be inconsistent with the original framework,
we need to add Transpose
operator to transpose the input tensors if necessary.
However, for some operators like concat
,
we could transpose their arguments to eliminate Transpose
op for acceleration.
Based on these conditions, We partition the ops into 3 categories.
- Ops with fixed inputs' data format(
FixedDataFormatOps
):Convolution
,Depthwise Convolution
, etc. - Ops could eliminate
Transpose
by transposing their arguments(TransposableDataFormatOps
):Concat
,Element-wise
, etc. - Ops keeping consistent with source platform(
SourceDataFormatOps
):Reshape
,ExpandDims
, etc.
By default, the operators not in either FixedDataFormatOps
or TransposableDataFormatOps
are listed in SourceDataFormatOps
.
For detailed information, you could refer to code.
Data Format in Operator¶
Based on the operator partition strategy, every operator in MACE
has
data format argument which stands for the wanted inputs' data format,
the values could be one of the [NHWC
, NCHW
, AUTO
].
NHWC
orNCHW
representsrc_df
.AUTO
represents the operator's inputs must have fixed data format, and the real data format will be determined at runtime. the data format of operators inFixedDataFormatOps
must beAUTO
, while the data format of operators inTransposableDataFormatOps
is determined based on their inputs' ops data format.
MACE will transpose the input tensors based on the data format information automatically at runtime.
Data Format of Model's Inputs/Outputs¶
- If the model's inputs/outputs have data format, MACE supports the data format
NHWC
andNCHW
. - If the model's inputs/outputs do not have data format, just set
NONE
for model's inputs and outputs atmodel deployment file
andMaceTensor
.