A simple way for Motion JPEG in Flask
If you are running some remote device, like the workstation, Raspberry Pi, Nvidia Jetson, edge devices…, then MJPEG is easy to implementation with Python Flask
Introduction
This article will show:
- set up flask server to get MJPEG on the browser
- get camera image (OpenCV) and send it to the browser, like streaming camera to the browser without WebRTC.
- Convert plot canvas/image to np array
- Plot ( Matplotlib ) image/data to the browser
Motion JPEG (MJPEG) is easy implementation with Python Flask and can quickly get results on the web browser, unlike WebRTC need a signaling server, TURN server, and more code on the device/browser.
It’s a good idea to use motion jpeg if you are running some edge device and want to simple profile some remote data.
(But make sure you have enough bandwidth/stability for internet, like local network, or can think about using WebRTC)
Prepare a Python Flask Server
Let’s import
import time
import numpy as np
import matplotlib.pyplot as plt
import cv2
from flask import Flask, ResponseNow, let’s set up the Flask server.
( for more details about Flask: https://flask.palletsprojects.com/en/2.0.x/ )
app = Flask(__name__)@app.route("/")
def hello_world():
return "<p>Hello, World!</p>"app.run(host='0.0.0.0', threaded=True)
Please note, we set the threaded as True because running motion jpeg request will occupy a thread.
If you don’t, you may not get other requests to run. ( like some page/resource request will be stuck ). And this is just for demo/debug usage. In production, please follow the WSGI.
Serve the Image with MJPEG
here is the code:
def gather_img():
while True:
time.sleep(0.2)
img = np.random.randint(0, 255, size=(128, 128, 3), dtype=np.uint8)
_, frame = cv2.imencode('.jpg', img)
yield (b'--frame\r\nContent-Type: image/jpeg\r\n\r\n' + frame.tobytes() + b'\r\n')@app.route("/mjpeg")
def mjpeg():
return Response(gather_img(), mimetype='multipart/x-mixed-replace; boundary=frame')
If you open the browser to the page “/mjpeg” you should get the result like:
In the above code image, which is a random noise for demo serve image to the browser.
Camera streaming to the browser
So if you want to work with a camera, you can do this
app = Flask(__name__)@app.route("/")
def hello_world():
return """
<body style="background: black;">
<div style="width: 240px; margin: 0px auto;">
<img src="/mjpeg" />
</div>
</body>
"""# setup camera and resolution
cam = cv2.VideoCapture(0)
cam.set(cv2.CAP_PROP_FRAME_WIDTH, 320)
cam.set(cv2.CAP_PROP_FRAME_HEIGHT, 240)def gather_img():
while True:
time.sleep(0.1)
_, img = cam.read()
_, frame = cv2.imencode('.jpg', img)
yield (b'--frame\r\nContent-Type: image/jpeg\r\n\r\n' + frame.tobytes() + b'\r\n')@app.route("/mjpeg")
def mjpeg():
return Response(gather_img(), mimetype='multipart/x-mixed-replace; boundary=frame')app.run(host='0.0.0.0', threaded=True)
Here are we doing:
- setup camera by cv2 ( OpenCV) to 320x240 resolution.
- read frame from camera ( image )
- Send via MJPEG.
- time.sleep is for saving bandwidth and FPS
The result example:
Visualization data to the browser
Here is the code of plot ( by Matplotlib ) data to browser
app = Flask(__name__)@app.route("/")
def hello_world():
return """
<body style="background: black;">
<div style="width: 240px; margin: 0px auto;">
<img src="/mjpeg" />
</div>
</body>
"""def plot_data():
x1 = np.random.randn(5)
x2 = np.random.randn(10)
fig = plt.figure(figsize=(10, 10))
plt.plot(x1)
plt.plot(x2)
plt.close(fig)
fig.canvas.draw()
return np.array(fig.canvas.buffer_rgba())def gather_img():
while True:
time.sleep(0.1)
img = plot_data();
img = cv2.cvtColor(img, cv2.COLOR_RGBA2BGR)
_, frame = cv2.imencode('.jpg', img)
yield (b'--frame\r\nContent-Type: image/jpeg\r\n\r\n' + frame.tobytes() + b'\r\n')@app.route("/mjpeg")
def mjpeg():
return Response(gather_img(), mimetype='multipart/x-mixed-replace; boundary=frame')app.run(host='0.0.0.0', threaded=True)
- You can plot any data like x1, x2 here.
- We convert plot canvas/image to np array
- Because the canvas is RGBA, so need to convert to BGR ( OpenCV color format ) to get the correct color.
The result example:
Conclusion
Motion JPEG ( MJPEG ) is easy implementation on IoT/edge devices that have python flask env.
As the above example, you can easily get chart ( plot data ) or live camera streaming, moreover such as the bounding box of object detection result.
But the trade-off is bandwidth needs to be a concern. If you are in a local network or debug should be fine for it.
