Real-time browser-ready computer vision apps with Streamlit

Room:

Liffey Hall 2

Start (Dublin time):

14:00 on 14 July 2022

Start (your time):

09:00 on 14 July 2022

Duration:

30 minutes

Abstract

By using Streamlit and streamlit-webrtc, we can create web-based real-time computer vision apps only with ~10 or 20 additional lines of Python code.

To turn computer vision models into real-time demos, we have conventionally used OpenCV modules such as cv2.VideoCapture and cv2.imshow(). However, such apps are difficult or impossible to share with friends, run on smartphones, or integrate with modern interactive widgets and other data views and inputs.

Web-based apps don't have such problems.

Streamlit provides an easy way to build web apps quickly, and streamlit-webrtc allows to use real-time video streams. You can create real-time video apps with modern interactive views and inputs, and host these apps on the cloud to use from any devices with browsers.

In this talk, I will demonstrate the development process using these libraries and show a variety of examples so that we see how easy and useful they are and can make use of them in daily development and research.streamlit-webrtc extends Streamlit to be capable of dealing with real-time video and audio streams. With a combination of these libraries, developers can rapidly create real-time computer vision and audio processing apps for which OpenCV has typically been used.

TalkPyData: Deep Learning, NLP, CV

Description

I am the author of streamlit-webrtc and a member of the Streamlit Creators program (selected community members). The repository of streamlit-webrtc is here: https://github.com/whitphx/streamlit-webrtc

My lightning talk about streamlit-webrtc at PyCon JP 2021 is available: https://youtu.be/_LuLs8H1gJc

Articles about this library:

• Developing Web-Based Real-Time Video/Audio Processing Apps Quickly with Streamlit
• Real-Time Video Streams With Streamlit-WebRTC

As linked from the repo, demo apps I have developed are available online:

• Demo showcase including real-time object detection: https://share.streamlit.io/whitphx/streamlit-webrtc-example/main/app.py
  • Source code: https://github.com/whitphx/streamlit-webrtc-example/blob/main/app.py
• Real-time Speech-to-Text: https://share.streamlit.io/whitphx/streamlit-stt-app/main/app_deepspeech.py
  • Source code: https://github.com/whitphx/streamlit-stt-app
• Real-time style transfer: https://share.streamlit.io/whitphx/style-transfer-web-app/main/app.py
  • Source code: https://share.streamlit.io/whitphx/style-transfer-web-app/main/app.py
• Real-time Tokyo 2020 Pictogram: https://share.streamlit.io/whitphx/tokyo2020-pictogram-using-mediapipe/streamlit-app
  • Source code: https://github.com/whitphx/Tokyo2020-Pictogram-using-MediaPipe
• Video chat: online demo is not available because it does not have an auth mechanism and is only for private use.
  • Source code: https://github.com/whitphx/streamlit-video-chat-example

The speaker