You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hello! I've found a performance issue in utils/data_loader.py: batch() should be called before map(), which could make your program more efficient. Here is the tensorflow document to support it.
Detailed description is listed below:
data_loader.py: train_dataset.batch(batchsize)(line 103) should be called before train_dataset.map(parse_data, num_parallel_calls=2)(line 101).
data_loader.py: val_dataset.batch(batchsize)(line 140) should be called before val_dataset.map(parse_data_without_augmentation)(line 138).
data_loader.py: train_dataset.batch(batchsize)(line 194) should be called before train_dataset.map(parse_single_record, num_parallel_calls=4)(line 192).
Besides, you need to check the function called in map()(e.g., parse_single_record called in train_dataset.map(parse_single_record, num_parallel_calls=4)) whether to be affected or not to make the changed code work properly. For example, if parse_single_record needs data with shape (x, y, z) as its input before fix, it would require data with shape (batch_size, x, y, z).
Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.
The text was updated successfully, but these errors were encountered:
Hello! I've found a performance issue in utils/data_loader.py:
batch()
should be called beforemap()
, which could make your program more efficient. Here is the tensorflow document to support it.Detailed description is listed below:
train_dataset.batch(batchsize)
(line 103) should be called beforetrain_dataset.map(parse_data, num_parallel_calls=2)
(line 101).val_dataset.batch(batchsize)
(line 140) should be called beforeval_dataset.map(parse_data_without_augmentation)
(line 138).train_dataset.batch(batchsize)
(line 194) should be called beforetrain_dataset.map(parse_single_record, num_parallel_calls=4)
(line 192).Besides, you need to check the function called in
map()
(e.g.,parse_single_record
called intrain_dataset.map(parse_single_record, num_parallel_calls=4)
) whether to be affected or not to make the changed code work properly. For example, ifparse_single_record
needs data with shape (x, y, z) as its input before fix, it would require data with shape (batch_size, x, y, z).Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.
The text was updated successfully, but these errors were encountered: