HOWTO: How to dewarp RTSP stream from a Meraki Fisheye Camera

[PaulF]

Morning all. 

This has come up several times, and, prompted by a colleague, I went to investigate how to dewarp a 180 degree RTSP stream from a Meraki MV Camera

You're going to need two pieces of software. FFMPEG and FFPLAY

Firstly, there is no warranty, implied or otherwise here. FFMPEG is used at your own risk, and this is the suggestion of an engineer who needed a Proof of Concept. Nothing else

I'm going to do this on macOS, rather than windows!

So, firstly, download FFMPEG and FFPLAY from https://www.ffmpeg.org/

Secondly, go to Meraki Dashboard, Cameras, Cameras, select your camera (MV33 in my case), then Settings. Enable RTSP and get the URL. In should be in the format of rtsp://192.168.128.1:9000/live for example

Okay, let's first test that we can play the stream

Open up Terminal, navigate to where you've downloaded FFMPEG and FFPLAY

Type:

./ffplay -rtsp_transport tcp -t rtsp://192.168.128.163:9000/live

After a lot of text, it should spawn a new window and show you your stream:

 

If you've got this far, you've done the hard work. You can close the video window, which will terminate FFPLAY

Now, we need to do two things. The first is to dewarp. We are going to use FFMPEG for this

./ffmpeg -rtsp_transport tcp -i rtsp://192.168.128.163:9000/live -vf "v360=fisheye:output=c1x6:ih_fov=180:iv_fov=180:pitch=-90"

There's a lot to unpack here. We use FOC=180 because this is a fisheye, NOT a 360 camera. Pitch rotates the image too, if need be, depending on your view point

Now, we need to send the video stream, dwarfed, to FFPLAY

./ffmpeg -rtsp_transport tcp -i rtsp://192.168.128.163:9000/live -vf "v360=fisheye:output=c1x6:ih_fov=180:iv_fov=180:pitch=-90" -f matroska - | ./ffplay  - 

-f is format, and we chose the matroska format. We then pipe this to ffplay

Run this command

(I've only grabbed part of this image because it's 1056x6336 resolution

Now, the hard part. We can use a parameter in FFPLAY to both crop and rotate the image if need be

I'm going to chose the middle part of the image above and rotate:

./ffmpeg -rtsp_transport tcp -i rtsp://192.168.128.163:9000/live -vf "v360=fisheye:output=c1x6:ih_fov=180:iv_fov=180:pitch=-90" -f matroska - | ./ffplay  - -vf crop=800:800:1600:800,transpose=1

-vf crop=800:800:1600:800 takes the image at 1600:800 and creates a 800x800 image from there

 using the transpose=1 rotates it 90 degrees clockwise with no flip we get

 Now, this is not the central part of the original image, it's actually the left hand side, but it's dewarped!

The difficult part ahead of you now is understanding which part of the image it is you want to see using

vf crop=800:800:1600:800

You'll only need to change the last two parameters to experiment

Hope that's been useful!

[BlakeRichardson]

Nice work and thanks for sharing!

[j_bergler]

Hey @PaulF - thanks for doing the heavy lifting here! This really helped me get started.

I was trying to wrap my head around cropping the right part of the frame when I stumbled into something that might make the solution to this problem even easier!

I ended up with the following filter string for ffmpeg:

-vf v360=fisheye:output=flat:ih_fov=180:iv_fov=180:order=rpy:roll=165:pitch=60:d_fov=90

let me try and explain what's going on there (at least what I think is happening)

By saying `output=flat` we're asking ffmpeg to go straight to a normal flat/rectilinear projection

The `ih_fov=180:iv_fov=180` tells it the fisheye field of view, but interestingly enough ffmpeg seems to assume the fisheye is facing "forward" (for lack of a better word). This part was important to realise, since these cameras are almost certainly ceiling mounted facing down.

`order:rpy` says apply the camera movements in the order, roll, pitch then yaw.

This lets makes it easier to think about the camera movements. Imagine your standing under the camera and looking at your feet.

We start with roll, which equates to turning on the spot, positive roll is turning to your right, negative to the left.

Then pitch, which lifting your gaze from your feet (pitch=0) to the horizon (pitch=90)

Finally `d_fov` is like zooming in or out.

All of this is kind hurting my brain, but it did let me crop exactly what I wanted from the dewarped image, and I hope it helps someone else!