At the beginning of the pandemic, there seemed to be a spate of virtual tours, popping up all over the internet. At the time, it seemed like a good idea, though many seemed not to be very popular for very long. Nevertheless I thought I'd jump on the bandwagon and try my hand at making something similar. I figured I'd post all my 360 Antarctica videos in a sort of tour-format and see if anyone was interested.
When I go on big trips, I like to have a bit of a project to do on the way. On the Trans-Siberian, I did the big railway slitscan - when I visited Morocco, I did a lot of astronomy. On this trip, I figured capturing the place in a more immersive way might be fun. 360 video was getting popular - well, at least a little - on YouTube and other places so I thought I'd give it a go.
I'd looked previously at building my own rig, but there were several ready made options. I plumped for the Garmin Virb 360 in the end. It had the highest resolution at the time (5.7K -almost equivalent to 4K under certain views, so they say), has replaceable lens covers, replaceable battery and is waterproof - something I figure I'd need on a ship. I'd toyed with the idea of some sort of 3D but most of these options looked less sturdy and had pretty low resolutions.
The VIRB has GPS which is pretty neat. That way I could pin-point the video on a map. However, I needed a way to record a trace of the route, from start to finish. I have a really old Android phone with a program that saves the GPS track to a nice CSV file. Problem was, most of the time on the ship I didn't really get much of a track. When I was below decks, I didn't get much of a signal. I figured that might be the case but the coordinates I did get seemed to provide a good framework for the trip. At the same time, I recorded the headings and temperatures manually by looking at the displays on the ship's helm.
I wanted to present everything on a map, embedding some sort of google map or openstreetmap with the trip represented by a bunch of clickable pins. I plumped for mapbox which seems to work quite easily with another library called leaflet. Leaflet sits above a map back-end and provides some useful functions for dropping pins, drawing lines and what not. Mapbox charges you if you end up with all the requests in the world, but their free tier is quite generous.
I really wanted to get this working on the web with VR headsets. At first I thought I'd use my own WebGL library pxljs, but it would have required a bit more work to get the video texturing done. Rather than do that, I figured I'd use the A-Frame library, which has all the stuff I needed baked in. It was pretty trivial to get up and running and seems to have all the required code to look after different devices, such as using a phone instead of a headset. Pretty neat!
I wanted to blur out the faces of most of the people on the trip. The point of the project was not to record people, but to show off the beauty and isolation of Antarctica. Problem is, at the time I made this, face blurring technology wasn't actually that great. The best I could find was a program called blurryfaces over on github. I didn't have the time to do every frame manually, so I went with the automatic blurring after testing some of the parameters. It worked sometimes but more often than not it failed. I tried a few other programs but most were worse. In the end, I gave up. I don't think the faces are too recognisible anyway, but I think it's only polite to at least try. I asked folks on the ship if they wanted to be anonymous and only one person replied in the affirmative so I'm not to worried.
The VIRB is controlled by both an application on your phone and the controls on the device itself. Many of the options on the VIRB are not available via the on-camera controls as the unit is quite compact. I don't particularly like wifi or bluetooth connections to cameras like this, but all I needed to do was login and alter a few settings once. After that, I could use the large switch on the right hand side of the device and I was good to go.
Having a really large switch to start and stop filming is very, very useful when you are in a cold place! Something I never thought to appreciate until I had to use the camera with my gloves.
Garmin have a Windows program that stitches the two halves of the video together. It does a reasonable job, so I decided to use Windows in this instance. Normally, I'd try and stick on Linux full stop, but in this case, it was just quicker to set a stitch going, come back half an hour later and set another going. Boring, but once it was done it's done.
ffmpeg is such great little program! I love it to bits! I used it extensively to process the videos from the VIRB. The native resolution of the VIRB is great, but the file-sizes are really large. I didn't want a visitor to wait to long before the video appeared, so I decided to reduce down the resolution. By default, VIRB video has the following statistics:
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'ANT_V1970484_Stitched.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf57.71.100
Duration: 00:01:32.60, start: 0.000000, bitrate: 43582 kb/s
Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 4.0, fltp, 593 kb/s (default)
Metadata:
handler_name : SoundHandler
Stream #0:1(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 5760x2880, 42982 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
Metadata:
handler_name : VideoHandler
Side data:
spherical: equirectangular (0.000000/0.000000/0.000000)
A pretty high resolution there, and a good frame-rate. I decided to try a few different sizes, trading off file-size against perceived quality. The smallest size went like this:
ffmpeg -i ANT_V1970484_Stitched.mp4 -vf scale=1980x990,setdar=2:1 -r 25 -c:v libx264 -b:v 4M -pix_fmt yuv420p -an V1970484_smallest_google.mp4
This seemed a bit too rubbish. I'd heard that the way to go for the web was to use the webm format. I tried giving it a go as follows:
ffmpeg -i ANT_V1970484_Stitched.mp4 -c:v libvpx-vp9 -b:v 8M -vf scale=3840x2160,setdar=16:9 -r 25 -pass 1 -an -f webm /dev/null && ffmpeg -i ANT_V1970484_Stitched.mp4 -c:v libvpx-vp9 -vf scale=3840x2160,setdar=16:9 -r 25 -b:v 8M -pass 2 -an V1970484.webm
It didn't work very well at all, so I ditched the webm and went with mp4 and the h264 codec (via x264) codec as follows:
ffmpeg -i ANT_V1970484_Stitched.mp4 -vf scale=2880x1440,setdar=2:1 -r 25 -c:v libx264 -b:v 8M -pix_fmt yuv420p -an V1970484.mp4
There is one final thing to do though. 360 videos always have a spot at the bottom of the view where the camera would be. I decided to place my watermark there as it looks neat and isn't too obtrusive. The ffmpeg command looks like this:
ffmpeg -i V1970484.mp4 -i bcpu_04\ _360_watermark.png -filter_complex \"overlay=(main_w-overlay_w)/2:(main_h-overlay_h)/2" V1970484_watermark.mp4
I think most nerds have a project that sits around for a while. You keep thinking about it though, and then the time comes and you knuckle down and get it done. The early days of Lock-Down in the UK seemed like a good time to get this project finished. Maybe a few people would watch it for a bit of escapism? Truth is, most of these virtual tours didn't seem to take off. Some were lovely but there's no substitute for the real thing - something I suspect many have painfully realised this year.
There are plenty of improvements I'd love to make. The website is very bare-bones. This was deliberate; I didn't want to distract from the video and I'm a big believer in the minimalist web and removing all the stuff we don't need. However, I think a good list of the locations with some text, maybe presented in a story format, would be better. For instance, there are so many internal shots of the various Antarctica huts but finding these specifically is quite hard. Pressing the back button takes you back to the map with the pan & zoom reset which is quite annoying.
I've debated putting the videos up on YouTube as it'll probably reach more people that way. Problem is, I'd have to re-render the video at a higher resolution and make sure all the faces are properly blurred. Not sure if there are any bear-traps ready to spring on me if I upload footage containing people whose consent was not explicitly gained. Probably not as it's all public space, but I wouldn't want to bet on that.
Perhaps the major issue is the sound. I didn't have an external microphone and the VIRB doesn't support one. I thought about some sort of soundtrack, but what would you choose? If I had all the money in the world, I'd hire either British Sea Power, Public Service Broadcasting or 65daysofstatic, asking them to mix in some recorded sounds from the trip. Maybe next time, because I'd love to go back!
