This project shows the streaming of open-cv by applying the style-transfer to the background (or person) except the person (or background) in real time using a web-cam connected to a laptop or computer.
- Basic streaming function
- Camera zoom function, capture function, video recording function
- Style conversion function using Open-CV dedicated UI button
- Provides a function to apply style (background-to-person) using icons
Style Transfer can be applied by using a pre-trained model or by training yourself. There are many sources and sites that provide the ability to convert images, but there are few sources that are easily applied in real time, and in order to realize a simple idea that applies only to backgrounds other than people, we need to customize it to start the project.
So we came up with this project.
real-time-style-transfer-opencv was developed using the following library version:
- [Python3] - 3.7.4
- [Tensorflow] - 2.0.0
- [opencv-python] - 4.1.2.30
and [window 10] Environment
real-time-style-transfer-opencv require python3 v3+ and tensorflow v2+ to run.
Install the dependencies.
$ pip install opencv-python
$ pip intall tensorflow==2.0.0Clone Repository...
$ mkdir project
$ cd project
$ git clone https://github.com/harimkang/real-time-style-transfer-opencv.git
$ cd real-time-style-transfer-opencvreal-time-style-transfer-opencv requires a model that segmentes people and a style transfer model.
| Model | README |
|---|---|
| Style Transfer | magnta/arbitrary-image-stylization-v1-256 |
| People Segmentation | [U-Net] |
Just Start:
$ python Camera.pyAnd Use Buttons and Icon:
- Buttons: Click to switch the video to the painter's style. The button is blue in the On state and the photo in the Off state. Currently, there are Gogh, Kandinsky, Monet, Picasso, Na Hye-suk and Super Mario painting styles.
- Icon: The icon is available when you are in Style Transfer (when a certain button is On), and if Style Transfer is applied only to the background, it is switched to apply to people other than the background. The reverse is also possible.
- Adding Style:
-
Add the image of the style you want to add to the folder.
-
You can edit style_img in StyleTransfer class in style_transfer.py
-
Adding Button - In Button setting function of ButtonManager class in Button.py, you can add Button object like other buttons.
Below is an example of adding btn10.
btn10 = Button("Button's Name")
btn_list = [btn, btn2, btn3, btn4, btn5, btn6, btn7, btn10]
self.add_button_list(btn_list)
- FPS improvement
The project was conducted at the Korea Lab of Artificial Intelligence and formed a team called Mevia.
MIT
From MEVIA