fernandoperlar/preprocessing_image

1<br />2<p align="center">3  <a href="https://github.com/FernandoPerezLara/image-preprocessing-layer">4    <img src="https://huggingface.co/fernandoperlar/preprocessing_image/resolve/main/duck.png" alt="Logo" width="100" height="146">5  </a>6 7  <h3 align="center">Image Preprocessing Model</h3>8 9  <p align="center">10    Image preprocessing in a convolutional model11    <br />12    <a href="https://github.com/FernandoPerezLara/image-preprocessing-layer"><strong>Read more about the model »</strong></a>13    <br />14    <br />15    <a href="https://github.com/FernandoPerezLara/image-preprocessing-layer">View Code</a>16    ·17    <a href="https://github.com/FernandoPerezLara/image-preprocessing-layer/issues">Report Bug</a>18    ·19    <a href="https://github.com/FernandoPerezLara/image-preprocessing-layer/discussions">Start a discussion</a>20  </p>21</p>22<br />23 24The main objective of this project is to apply preprocessing to an image dataset while the model is being trained.25 26The solution has been taken because we do not want to apply preprocessing to the data before training (i.e. create a copy of the data but already preprocessed) because we want to apply data augmentation while the model trains.27 28The use of `Lambda` layers has been discarded because they do not allow the use of external libraries that do not work with tensors, since we want to use the functions provided by *OpenCV* and *NumPy*.29 30## Preprocessing31In this example found in this repository we wanted to divide the images from HSV color masks, where it is divided into:32* **Warm zones**: red and white colors are obtained.33* **Warm zones**: The green color is obtained.34* **Cold zones**: The color blue is obtained.35 36Within the code you can find the declaration of these filters as:37```python38filters = {39	"original": lambda x: x,40	"red": lambda x: data.getImageTensor(x, (330, 0, 0), (360, 255, 255)) + data.getImageTensor(x, (0, 0, 0), (50, 255, 255)),41	"green": lambda x: data.getImageTensor(x, (60, 0, 0), (130, 255, 255)),42	"blue": lambda x: data.getImageTensor(x, (180, 0, 0), (270, 255, 255)),43}44```45 46On the other hand, the preprocessing functions are located inside `scripts/Data.py` file as follows:47```python48def detectColor(self, image, lower, upper):49	if tf.is_tensor(image):50		temp_image = image.numpy().copy() # Used for training51	else:52		temp_image = image.copy() # Used for displaying the image53 54	hsv_image = temp_image.copy()55	hsv_image = cv.cvtColor(hsv_image, cv.COLOR_RGB2HSV)56	mask = cv.inRange(hsv_image, lower, upper)57 58	result = temp_image.copy()59	result[np.where(mask == 0)] = 060	61	return result62 63def getImageTensor(self, images, lower, upper):64	results = []65 66	for img in images:67		results.append(np.expand_dims(self.detectColor(img, lower, upper), axis=0))68 69	return np.concatenate(results, axis=0)70```71 72## Model73The model used to solve our problem was a *CNN* with a preprocessing layer:74 75![Model](./model.png "Model")76 77This model can be found in the `scripts/Model.py` file in the following function:78```python79def create_model():80	class FilterLayer(layers.Layer):81		def __init__(self, filter, **kwargs):82			self.filter = filter83 84			super(FilterLayer, self).__init__(name="filter_layer", **kwargs)85 86		def call(self, image):87			shape = image.shape88			[image, ] = tf.py_function(self.filter, [image], [tf.float32])89			image = backend.stop_gradient(image)90			image.set_shape(shape)91			92			return image93 94		def get_config(self):95			return super().get_config()96 97	model = models.Sequential()98 99	model.add(layers.Input(shape=(215, 538, 3)))100	model.add(FilterLayer(filter=self.filter))101 102	model.add(layers.Conv2D(32, (3, 3), activation="relu"))103	model.add(layers.MaxPooling2D(pool_size=(2, 2)))104 105	model.add(layers.Conv2D(32, (3, 3), activation="relu"))106	model.add(layers.GlobalAveragePooling2D())107 108	model.add(layers.Dropout(rate=0.4))109	model.add(layers.Dense(32, activation="relu"))110	model.add(layers.Dropout(rate=0.4))111	model.add(layers.Dense(2, activation="softmax"))112 113	return model114```115 116## Contributors117This work has been possible thanks to:118- [Fernando Pérez Lara](https://www.linkedin.com/in/fernandoperezlara/) ([**@FernandoPerezLara**](https://github.com/FernandoPerezLara)) for having developed the model to make this idea come true.119 120## License121Copyright (c) 2021 Fernando Pérez Lara.122 123Licensed and distributed under the [MIT](LICENSE.txt) license.124