Add fake_quantize_op. #11359

achao2013 · 2018-06-11T08:31:03Z

add quant code for test

qingqing01 · 2018-06-20T03:36:38Z

paddle/fluid/operators/fake_quantize_op.cc

+    // PADDLE_ENFORCE_EQ(ctx->Inputs("InScales")[0],
+    // ctx->Outputs("OutScales")[0],
+    //                  "Mean and MeanOut should share the same memory");
+    //}


Please remove the commented lines.

the comment is for test of python , the commented lines is used for train

qingqing01 · 2018-06-20T03:37:03Z

paddle/fluid/operators/fake_quantize_op.cc

+                   "Input(X) of FakeQuantizeOp should not be null.");
+    PADDLE_ENFORCE(ctx->HasOutput("Out"),
+                   "Output(Out) of FakeQuantizeOp should not be null.");
+    PADDLE_ENFORCE(ctx->HasOutput("OutMovingScale"), "");


Please add the error message.

qingqing01 · 2018-06-20T03:37:15Z

paddle/fluid/operators/fake_quantize_op.cc

+    ctx->SetOutputDim("OutMovingScale", ctx->GetInputDim("InMovingScale"));
+    //}
+    // if (ctx->HasInput("InScales")) {
+    PADDLE_ENFORCE(ctx->HasOutput("OutScales"), "");


Please add the error message.

qingqing01 · 2018-06-20T03:40:03Z

paddle/fluid/operators/fake_quantize_op.cc

+ public:
+  void Make() override {
+    AddInput("X", "(Tensor) Input tensor of scale operator.");
+    AddInput("InScales", "(Tensor) scale buffer").AsDispensable();


Please add more comments for why this argument is optional. When need it and when don't need it. The same is the following.

qingqing01 · 2018-06-20T03:42:04Z

paddle/fluid/operators/fake_quantize_op.cu

+namespace operators {
+
+template <typename T>
+__global__ void find_abs_max_kernel(const int n, const T* in, T* out) {


find_abs_max_kernel -> FindAbsMaxKernel

Please follow Google C++ code style: https://google.github.io/styleguide/cppguide.html#Function_Names

Please modify other code with the same problem.

qingqing01 · 2018-06-20T03:56:23Z

paddle/fluid/operators/fake_quantize_op.cu

+float find_abs_max_gpu(const platform::CUDADeviceContext& ctx,
+                       const float* array, int length) {
+  float host_max;
+  int NUM_THREADS = 1024;


NUM_THREADS -> kNumTheads Please follow Goolge code style.

qingqing01 · 2018-06-20T03:58:57Z

paddle/fluid/operators/fake_quantize_op.cu

+      cudaMemcpy(&host_max, device_max, sizeof(float), cudaMemcpyDeviceToHost),
+      cudaSuccess, "cudaMemcpy failed");
+  return host_max;
+}


Maybe can use thrust::reduce + thrust::max_element to find the maximum value for more simply.

this will be slow

qingqing01 · 2018-06-20T04:01:36Z

paddle/fluid/operators/fake_quantize_op.h

+    int window_size = context.Attr<int>("window_size");
+    int bit_length = context.Attr<int>("bit_length");
+    int bin_cnt = std::pow(2, bit_length - 1) - 1;
+    LOG(ERROR) << "bin_cnt:" << bin_cnt;


Remove this line.

qingqing01 · 2018-06-20T04:04:28Z

paddle/fluid/operators/fake_quantize_op.h

+        auto* scale_list = context.Output<framework::Tensor>("OutScales");
+        auto* saving_scale =
+            context.Output<framework::Tensor>("OutMovingScale");
+        scale = find_abs_max(const_cast<framework::Tensor*>(in), in->numel());


Here can use Eigen's method：

https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/adamax_op.h#L57

this cwiseMax is an elemwise max operation, i need a reduce max op .

Please refer https://github.com/RobotLocomotion/eigen-mirror/tree/master/unsupported/Eigen/CXX11/src/Tensor#-reduceconst-dimensions-new_dims-const-reducer-reducer

or

https://github.com/RobotLocomotion/eigen-mirror/tree/master/unsupported/Eigen/CXX11/src/Tensor#-maximum

qingqing01

Need unit testing.

qingqing01 · 2018-06-20T04:49:46Z

paddle/fluid/operators/fake_quantize_op.h

+      }
+    }
+
+    apply_saturate(const_cast<framework::Tensor*>(in), tensor, -scale, scale);


Also can refer https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/operators/clip_op.h#L70 for more simply.

qingqing01 · 2018-06-20T04:51:56Z

paddle/fluid/operators/fake_quantize_op.cc

+    AddComment(R"DOC(
+FakeQuantize operator
+
+$$Out = scale*X$$


Need comments for how to calculate scale.

qingqing01 · 2018-06-20T04:56:58Z

paddle/fluid/operators/fake_quantize_op.cc

+
+$$Out = scale*X$$
+)DOC");
+    AddAttr<std::string>("quantize_type",


quantize_type -> scale_type for more accurate ?

if the quantization method is non-uniform, scale is not need, so i think this should not be scale_type

我理解quantize_type一般指： Abs-Max，或者Min-Max等不同的量化方式。
而这里，这个attr是想来标示，计算scale的方式吧？

如果是非均匀量化，那浮点输入和定点输出可能是个函数，或者离散数值映射，就没有scale操作了

… quant

qingqing01 · 2018-06-29T03:01:24Z

@achao2013

[07:05:06]W: [Step 1/1] /paddle/paddle/fluid/operators/distributed/rpc_client.h:61:26: error: 'FLAGS_grpc_deadline' was not declared in this scope
[07:05:06]W: [Step 1/1] int64_t time_out = FLAGS_grpc_deadline) = 0;

CI没有通过，需要更新至最新develop代码。

… quant

qingqing01 · 2018-07-02T08:43:40Z

[07:00:33]	294/359 Test #290: test_fake_quantize_op ................................***Failed    5.52 sec
[07:00:33]	test_fake_quantize_op failed
[07:00:33]	E
[07:00:33]	======================================================================
[07:00:33]	ERROR: test_check_output (test_fake_quantize_op.TestFakeQuantizeOp)
[07:00:33]	----------------------------------------------------------------------
[07:00:33]	Traceback (most recent call last):
[07:00:33]	  File "/paddle/build/python/paddle/fluid/tests/unittests/test_fake_quantize_op.py", line 47, in test_check_output
[07:00:33]	    self.check_output()
[07:00:33]	  File "/paddle/build/python/paddle/fluid/tests/unittests/op_test.py", line 325, in check_output
[07:00:33]	    self.check_output_with_place(place, atol)
[07:00:33]	  File "/paddle/build/python/paddle/fluid/tests/unittests/op_test.py", line 263, in check_output_with_place
[07:00:33]	    outs, fetch_list = self._calc_output(place)
[07:00:33]	  File "/paddle/build/python/paddle/fluid/tests/unittests/op_test.py", line 226, in _calc_output
[07:00:33]	    inputs = self._get_inputs(block)
[07:00:33]	  File "/paddle/build/python/paddle/fluid/tests/unittests/op_test.py", line 211, in _get_inputs
[07:00:33]	    return self._get_io_vars(block, self.inputs)
[07:00:33]	  File "/paddle/build/python/paddle/fluid/tests/unittests/op_test.py", line 207, in _get_io_vars
[07:00:33]	    inputs[name] = block.var(name)
[07:00:33]	  File "/paddle/build/python/paddle/fluid/framework.py", line 900, in var
[07:00:33]	    raise ValueError("var %s not in this block" % name)
[07:00:33]	ValueError: var InCurrentScale not in this block
[07:00:33]	
[07:00:33]	----------------------------------------------------------------------
[07:00:33]	Ran 1 test in 0.002s
[07:00:33]	
[07:00:33]	FAILED (errors=1)

The unit testing did not pass.

… quant

qingqing01

Approved. @dangqingqing will refine and add more unit testing.

* Add a fake_quantize_op, which quantize an input tensor to a tensor with lower bits.

quantize

e5954c0

achao2013 changed the title ~~Quant~~ quantization code Jun 11, 2018

achao2013 changed the title ~~quantization code~~ add quantization code Jun 11, 2018

qingqing01 self-requested a review June 11, 2018 09:20

qingqing01 changed the title ~~add quantization code~~ Add fake_quantize_op. Jun 11, 2018

qingqing01 requested a review from wanghaoshuang June 14, 2018 08:06

qingqing01 reviewed Jun 20, 2018

View reviewed changes

achao2013 added 2 commits June 22, 2018 16:27

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

e26c2df

… quant

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

8bee0f4

… quant

achao2013 force-pushed the quant branch from c2a7847 to 8bee0f4 Compare June 22, 2018 09:49

achao2013 added 2 commits June 25, 2018 13:05

test

d3abd23

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

15cc228

… quant

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

65819c9

… quant

qingqing01 and others added 9 commits July 10, 2018 03:09

Small fix

26ada02

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

d6d03ea

… quant

unitest fake_quantize

051593a

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

ad3dea4

… quant

add new line

f8a2b7f

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

609701b

… quant

Merge branch 'quant' of https://github.com/achao2013/Paddle into quant

ca7b759

Revert convert_protobin.sh.

9926d46

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

ce3c4f7

… quant

qingqing01 approved these changes Jul 11, 2018

View reviewed changes

qingqing01 merged commit 8e4b225 into PaddlePaddle:develop Jul 11, 2018

kuke pushed a commit to kuke/Paddle that referenced this pull request Aug 25, 2018

Add fake_quantize_op. (PaddlePaddle#11359)

5607033

* Add a fake_quantize_op, which quantize an input tensor to a tensor with lower bits.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add fake_quantize_op. #11359

Add fake_quantize_op. #11359

achao2013 commented Jun 11, 2018

qingqing01 Jun 20, 2018

achao2013 Jun 20, 2018

qingqing01 Jun 20, 2018

achao2013 Jun 22, 2018

qingqing01 Jun 20, 2018

achao2013 Jun 22, 2018

qingqing01 Jun 20, 2018

achao2013 Jun 22, 2018

qingqing01 Jun 20, 2018

achao2013 Jun 22, 2018

qingqing01 Jun 20, 2018

achao2013 Jun 22, 2018

qingqing01 Jun 20, 2018

achao2013 Jun 20, 2018

qingqing01 Jun 20, 2018

achao2013 Jun 22, 2018

qingqing01 Jun 20, 2018 •

edited

Loading

achao2013 Jun 21, 2018

qingqing01 Jun 21, 2018

achao2013 Jun 22, 2018

qingqing01 left a comment

qingqing01 Jun 20, 2018

achao2013 Jun 22, 2018

qingqing01 Jun 20, 2018

achao2013 Jun 22, 2018

qingqing01 Jun 20, 2018

achao2013 Jun 20, 2018

qingqing01 Jun 21, 2018

achao2013 Jun 22, 2018

qingqing01 commented Jun 29, 2018

qingqing01 commented Jul 2, 2018

qingqing01 left a comment

Add fake_quantize_op. #11359

Add fake_quantize_op. #11359

Conversation

achao2013 commented Jun 11, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qingqing01 Jun 20, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qingqing01 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qingqing01 commented Jun 29, 2018

qingqing01 commented Jul 2, 2018

qingqing01 left a comment

Choose a reason for hiding this comment

qingqing01 Jun 20, 2018 •

edited

Loading