Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add image-recordio-gen cmd and RecordIO reader #377

Open
wants to merge 3 commits into
base: develop
Choose a base branch
from

Conversation

QiJune
Copy link
Collaborator

@QiJune QiJune commented Nov 6, 2020

The image-recordio-gen command converts an image folder with label txt to recordio file format.

Let's take mnist dataset as an example. We could download the dataset from https://github.com/myleott/mnist_png.git.

The dataset contains two directories: training and testing. We need to make a label file. The label file maps a class string to an int index. Following is the label file for mnist dataset.

0
1
2
3
4
5
6
7
8
9

Then, we could run the image-recordio-gen command:

$GOPATH/bin/image-recordio-gen -label=$MNIST/label.txt -dataset=$MNIST/training -output=$MNIST/train_record -recordsPerShard=1500

We could find the recordio shard files in train_record directory:

data-00000
data-00001
...
...

@QiJune QiJune changed the title add image-record-gen cmd add image-recordio-gen cmd Nov 6, 2020
@QiJune QiJune changed the title add image-recordio-gen cmd add image-recordio-gen cmd and RecordIO reader Nov 9, 2020
@codecov
Copy link

codecov bot commented Nov 9, 2020

Codecov Report

Merging #377 (58f47af) into develop (9afed2f) will decrease coverage by 0.51%.
The diff coverage is 64.86%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #377      +/-   ##
===========================================
- Coverage    87.77%   87.25%   -0.52%     
===========================================
  Files           33       35       +2     
  Lines         1505     1538      +33     
===========================================
+ Hits          1321     1342      +21     
- Misses         121      129       +8     
- Partials        63       67       +4     
Impacted Files Coverage Δ
tool/recordio/imagerecord.go 33.33% <33.33%> (ø)
vision/imageloader/imageloader.go 91.15% <66.66%> (-0.06%) ⬇️
tool/recordio/reader.go 76.00% <76.00%> (ø)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant