As we all know, input/output tensors in CNN model have data format like
NCHW(caffe), but there is no data format for non-CNN model.
However, in MACE, CNN model run on
float type using
NCHW data format,
while the others using
NHWC data format.
To support all models, so there are some concepts in
MACE you should know.
Source Data Format¶
Source Data Format(
src_df for short) stands for the original data format where
the model come from. For example, if you use caffe, the
We need this data format because some operators(Reshape etc.) are
related to the data format.
Generally, operators could be divided into 2 categories
based on whether the operator needs inputs with fixed data format(
one is the operators whose inputs have fixed data format(like
the other is the operators whose inputs should be the same with source framework.
Since the data format the operators need in MACE may be inconsistent with the original framework,
we need to add
Transpose operator to transpose the input tensors if necessary.
However, for some operators like
we could transpose their arguments to eliminate
Transpose op for acceleration.
Based on these conditions, We partition the ops into 3 categories.
- Ops with fixed inputs' data format(
Depthwise Convolution, etc.
- Ops could eliminate
Transposeby transposing their arguments(
- Ops keeping consistent with source platform(
By default, the operators not in either
are listed in
For detailed information, you could refer to code.
Data Format in Operator¶
Based on the operator partition strategy, every operator in
data format argument which stands for the wanted inputs' data format,
the values could be one of the [
AUTOrepresents the operator's inputs must have fixed data format, and the real data format will be determined at runtime. the data format of operators in
AUTO, while the data format of operators in
TransposableDataFormatOpsis determined based on their inputs' ops data format.
MACE will transpose the input tensors based on the data format information automatically at runtime.
Data Format of Model's Inputs/Outputs¶
- If the model's inputs/outputs have data format, MACE supports the data format
- If the model's inputs/outputs do not have data format, just set
NONEfor model's inputs and outputs at
model deployment fileand