Adding More Augmentation.

[kartikdutt18]

Hey everyone,
Recently Augmentation class was added. It currently supports only resize function. This issue aims to add other augmentations as well.
Background :

1. Each augmentation must be accessible by passing string to the class.
2. Each Transform should have same function definition as ResizeTransform.
3. Idea is to create a map of function pointers and call respective function based upon the string type.

Some augmentations that would be really nice to see would be horizontal flip, vertical flip and gaussian noise. Thanks a lot.

[ghost]

are we going for something like keras ImageGenerator

tf.keras.preprocessing.image.ImageDataGenerator(
featurewise_center=False, samplewise_center=False,
featurewise_std_normalization=False, samplewise_std_normalization=False,
zca_whitening=False, zca_epsilon=1e-06, rotation_range=0, width_shift_range=0.0,
height_shift_range=0.0, brightness_range=None, shear_range=0.0, zoom_range=0.0,
channel_shift_range=0.0, fill_mode='nearest', cval=0.0, horizontal_flip=False,
vertical_flip=False, rescale=None, preprocessing_function=None,
data_format=None, validation_split=0.0, dtype=None
)

?
If yes, I wish to contribute but I'm totally new to mlpack and don't really understand it's functioning as of now, will you please guide me on what should I read upon to be able to contribute.
Thanks

[kartikdutt18]

Hey @RakeshBhugra, Right. We already have LoadFromDirectory functionality and augmentation class that currently only supports resize function. The issue aims to add more functions like horizontal-flip etc. Usage about the functions implemented can be found here. So you would have to open a PR adding some of the augmentations that you mentioned. Welcome to the community.
Regards.

[mlpack-bot]

This issue has been automatically marked as stale because it has not had any recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions! 👍

[kartikdutt18]

Keep open.

[mlpack-bot]

This issue has been automatically marked as stale because it has not had any recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions! 👍

[kartikdutt18]

Keep open.

[RituRajSingh878]

I am new here so it will take some to get familiar with mlpack. I am interested in machine learning so I am here to contribute. Also, I would like to work on this issue if open.
Thanks

[kartikdutt18]

Hey @RituRajSingh878, Welcome to the community. This issue is still open. Feel free to work on this.
Thanks a lot.

[RituRajSingh878]

models/augmentation/augmentation_impl.hpp

Lines 25 to 48 in 27c4814

	// Initialize the augmentation map.
	std::unordered_map<std::string, void(*)(DatasetType&,
	size_t, size_t, size_t, std::string&)> augmentationMap;
	for (size_t i = 0; i < augmentations.size(); i++)
	{
	if (augmentationMap.count(augmentations[i]))
	{
	augmentationMap[augmentations[i]](dataset, datapointWidth,
	datapointHeight, datapointDepth, augmentations[i]);
	}
	else if (this->HasResizeParam(augmentations[i]))
	{
	this->ResizeTransform(dataset, datapointWidth, datapointHeight,
	datapointDepth, augmentations[i]);
	}
	else
	{
	mlpack::Log::Warn << "Unknown augmentation : \'" <<
	augmentations[i] << "\' not found!" << std::endl;
	}
	}
	}

@kartikdutt18 Can you please explain this part of the code? I am not able to understand why are we storing it in an unordered map and what's the use of it?
Also, here we are applying resize augmentation so if I will implement other augmentations, then should I apply other augmentations after resizing or before the resizing on the data?

[kartikdutt18]

Hey @RituRajSingh878, I use a map so that each string could correspond to a function. This would prevent us from writing multiple if conditions.
Simply put, augmentations[stringAugmentationToDoX] ----> FunctionThatDoesX.
I think we can resize before (I think that is the case now) and we can leave it as it is.

[RituRajSingh878]

okay, I got the idea behind it. Now, my question is, we are creating augmentationMap in line 27. Inside the loop, we have
if (augmentationMap.count(augmentations[i])) . Now augmentationMap.count(augmentations[i]) value will be either 1 or 0 if element is present or not present. But augmentationMap was never initialized and have zero no. of elements so augmentationMap.count(augmentations[i]) value will be always zero and code will never run into if (augmentationMap.count(augmentations[i])) this condition. That is why, I had asked the above question.

[kartikdutt18]

Yes the reason is because we don't have any augmentations yet other than resize. It was done to set the structure. That's the aim of this issue.

[RituRajSingh878]

okay, got it. I will open a pr tomorrow.

[mlpack-bot]

This issue has been automatically marked as stale because it has not had any recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions! 👍

[mlpack-bot]

This issue has been automatically marked as stale because it has not had any recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions! 👍

[sirish-gambhira]

Hey @kartikdutt18, I am Sirish Gambhira. I am new to the community and this issue sounds interesting. I understand that horizontal and vertical flips in this issue are being taken care in the PR #38. Maybe, can we add rotation (like arbitrary angle) which can be useful for image datasets and gaussian noise as mentioned. Let me know if this sounds good.

[sirish-gambhira]

@RituRajSingh878 are you still working on this? If not I shall takeup the issue.

[kartikdutt18]

Hey @Sirish07, Sure, feel free to take up other augmentations. Adding rotation based augmentations sounds good to me.
Regards,
Kartik.

[sirish-gambhira]

Hey @kartikdutt18 , thanks for getting back. I added gaussian blurring and modified the current augmentation_impl.hpp and augmentation.hpp files. I would like to know what the next step is. Should I open a PR or should I test my implementation (if so, kindly explain how (I couldn't find any docs)). Thanks in advance.

[RishabhGarg108]

Hii @kartikdutt18 , I was went through the GSOC idea of "Ready to use models in mlpack" and I really liked it. I have been a contributor to mlpack main repo but I am new to this models repo and would like to get familiar with it. So, can I take up this issue? I would like to get started with horizontal/vertical flip. Can I take this ?
Warm regards.

[kartikdutt18]

Hey @RishabhGarg108, Please feel free to pursue it.
Regards,
Kartik.

[RishabhGarg108]

Thanks for the quick response. I can see that there are some incomplete PRs. So, I will try to take some motivation from them :)

[frostbyte012]

Hello @kartikdutt18 I'm a newbie here want to contribute to this issue, can u show me the pre- requisites and roadmaps for this. Thanks :) !