python train/train.py's error info is : Input to reshape is a tensor with 1 values, but the requested shape has 0

[boshanyiqiao]

Restored 267(640) vars from ./data/pretrained_models/resnet_v1_50.ckpt
iter 0: image-id:0434145, time:5.684(sec), regular_loss: 0.242197, total-loss 2.3483(0.0444, 0.6189, 0.002725, 0.8900, 0.7922), instances: 6, batch:(40|176, 1|16, 1|1)
labels
['cell phone']
classes
['boat' 'bench' 'backpack' 'apple' 'donut' 'clock' 'scissors' 'toothbrush']
iter 10: image-id:0245582, time:1.631(sec), regular_loss: 0.242204, total-loss 1.8110(0.0906, 0.6112, 0.004583, 0.3162, 0.7884), instances: 5, batch:(18|80, 1|16, 1|1)
labels
['person']
classes
['background']
iter 20: image-id:0507776, time:0.678(sec), regular_loss: 0.242249, total-loss 2.1038(0.0973, 0.5978, 0.009474, 0.5543, 0.8449), instances: 11, batch:(55|222, 2|33, 2|2)
labels
['person' 'car']
classes
['background']
iter 30: image-id:0376740, time:0.689(sec), regular_loss: 0.242356, total-loss 0.7036(0.0772, 0.6263, 0.000000, 0.0000, 0.0000), instances: 14, batch:(126|504, 0|39, 0|0)
labels
[]
classes
['background']
2018-05-11 04:57:57.626665: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Input to reshape is a tensor with 1 values, but the requested shape has 0
[[Node: Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](DecodeRaw_1/_551, Reshape/shape)]]
Traceback (most recent call last):
File "train/train.py", line 339, in
train()
File "train/train.py", line 335, in train
coord.join(threads)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/coordinator.py", line 389, in join
six.reraise(*self._exc_info_to_raise)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/queue_runner_impl.py", line 238, in _run
enqueue_callable()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1231, in _single_operation_run
target_list_as_strings, status, None)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/errors_impl.py", line 473, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Input to reshape is a tensor with 1 values, but the requested shape has 0
[[Node: Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](DecodeRaw_1/_551, Reshape/shape)]]
[[Node: ReverseV2/_603 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_250_ReverseV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

how should I fix it?

[LiuPearl1]

@boshanyiqiao Have you solved this problem? I also meet this problem and I don't know how to deal with it.

[codesteller86]

@boshanyiqiao, @LiuPearl1 : were you guys able to solve the issue?

[codesteller86]

Finally, I found the solution from another thread under this repo ##113.
The issue is created during the creation of TFRecords, there are few GT_boxes which are NULL; so a null check solves the problem.
Fix:

• Delete all previous TFRecords files
• go to ./libs/datasets/download_and_convert_coco.py

Change lines 309 to 316 & 421 to 428

if gt_boxes.shape[0] > 0:
    example = _to_tfexample_coco_raw(
    img_id,
    img_raw,
    mask_raw,
    height, width, gt_boxes.shape[0],
    gt_boxes.tostring(), masks.tostring())
    tfrecord_writer.write(example.SerializeToString())