s9xie 833a3849c9 | 7 years ago | |
---|---|---|
.. | ||
README.md | 7 years ago | |
cifar10-gen.lua | 7 years ago | |
cifar10.lua | 7 years ago | |
cifar100-gen.lua | 7 years ago | |
cifar100.lua | 7 years ago | |
imagenet-gen.lua | 7 years ago | |
imagenet.lua | 7 years ago | |
init.lua | 7 years ago | |
transforms.lua | 7 years ago |
Each dataset consist of two files: dataset-gen.lua
and dataset.lua
. The dataset-gen.lua
is responsible for one-time setup, while
the dataset.lua
handles the actual data loading.
If you want to be able to use the new dataset from main.lua, you should also modify opts.lua
to handle the new dataset name.
dataset-gen.lua
The dataset-gen.lua
performs any necessary one-time setup. For example, the cifar10-gen.lua
file downloads the CIFAR-10 dataset, and the imagenet-gen.lua
file indexes all the training and validation data.
The module should have a single function exec(opt, cacheFile)
.
opt
: the command line optionscacheFile
: path to outputlocal M = {}
function M.exec(opt, cacheFile)
local imageInfo = {}
-- preprocess dataset, store results in imageInfo, save to cacheFile
torch.save(cacheFile, imageInfo)
end
return M
dataset.lua
The dataset.lua
should return a class that implements three functions:
get(i)
: returns a table containing two entries, input
and target
input
: the training or validation image as a Torch tensortarget
: the image category as a number 1-Nsize()
: returns the number of entries in the datasetpreprocess()
: returns a function that transforms the input
for data augmentation or input normalizationlocal M = {}
local FakeDataset = torch.class('resnet.FakeDataset', M)
function FakeDataset:__init(imageInfo, opt, split)
-- imageInfo: result from dataset-gen.lua
-- opt: command-line arguments
-- split: "train" or "val"
end
function FakeDataset:get(i)
return {
input = torch.Tensor(3, 800, 600):uniform(),
target = 42,
}
end
function FakeDataset:size()
-- size of dataset
return 2000
end
function FakeDataset:preprocess()
-- Scale smaller side to 256 and take 224x224 center-crop
return t.Compose{
t.Scale(256),
t.CenterCrop(224),
}
end
return M.FakeDataset
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》