Friday, August 18, 2017

Shooting the Eclipse 2017 with a GoPro

I'm planning to find my way into the path of totality, and be there with a bunch of GoPro gear. I've been asked many times how to shoot this event with a GoPro, so here are my thoughts. Disclaimer: this is my completely unpracticed opinion on shooting a total solar eclipse with a GoPro.

Some basics. The whole transit from start to finish, somewhat depending on your location, is about 3 hours. That is pushing the average GoPro a little beyond its battery life in a time-lapse mode, even more beyond in regular video mode. As three hours likely produces pretty boring video, time-lapse is the way to go. If you intend to time-lapse the entire transit, you can use any USB power brick to extend the GoPro's run time; I've done a week long time-lapse via USB power. If you intend to shoot on battery power alone, on a full charge I typically get about 2 to 2.5 hours of time-lapse on a HERO5 Black with a 5-10 second interval.  Plan to start your time-lapse about 1 hour before totality.

Timing. Time-lapse interval for two hour capture, my best estimate is 5 seconds. Which will be a 48 second video when played at 30p.  A shorter video to share would be better, yet if you are lucky to have two minutes of totality, this interval only gets you 24 frames (0.8s) of time in the totality.  If you intend to work on the video with a speed ramp for the less exciting bits, then a 1 or 2 second interval might be better, but watch out for your battery life.

Framing.  As you know the GoPro lens is very wide, forget about getting any close-up views -- well without mounting the GoPro against an eyepiece of a telescope, which I will be doing in one setup.

Practicing: A video frame extracted from GoPro through the eyepiece of a sub-$200 4" telescope with a solar filter.  
With a wide time-lapse, consider how the light will change across the landscape, so compose your framing to capture that.

Filters. A GoPro will not be shooting through those safety filters, those are for your eyes and telescopes etc. Using a solar filter on a GoPro will give you a very small orange dot that moves across the frame, if you are lucky -- don't do this.  If you have ND filters, you can use them or not, modern GoPros are used to shooting images that also contain the sun, the sun's image is too small on the sensor to do damage.

Exposure control. In most cases a GoPro is an auto exposing camera, this is a good thing for those in the path of totality as the camera will adjust for all lighting conditions, give you a good video throughout. The downside for those not in the path of totality the auto-exposure will reduce dramatic level of changing light level. On a Hero 4 Silver and on HERO 4/5 Black, you can lock the ISO to 100 and set a fixed shutter speed, but only in the video modes with Protune enabled, so you will be left to process a lot of video into a timelapse in post.  You will also need ND16 or ND32 filters to make locking the exposure work for a correctly exposed image at the beginning of you capture.

Time-lapse video vs time-lapse photo vs Night-Lapse photo. Time-lapse video (TLV) is the easiest by far, producing a small MP4 that is ready to share, as soon as the cell service recovers from the network load of millions of eclipse chasers filling small country towns. The downside of TLV is there are no Protune controls, it is all automatic. The other two time-lapse modes will produce JPGs (and GPRs if RAW is enabled) and you can have Protune level controls to set the look (GoPro vs Flat), white balance, ISO, sharpness etc.  If you are in the path of totality, choose Night-Lapse, it will still work during daylight, but will take much longer exposures as needed for the dark few minutes.

My Recommendations: For those willing to do color correction and post assemble a time-lapse: Night-Lapse, Auto shutter, 4 second interval, Protune Flat, Native White balance (or 5500K for simpler color correction), ISO Min 100, ISO Max 800.  I will enable RAW.  This will produce 1800 images over 2 hours, one set of JPGs and one set of GPRs, using about 18 GB for storage.  If you want a fast easy time-lapse, use Time-lapse video with a 5 second interval.

Wednesday, May 03, 2017

GPMF - GoPro's Metadata Format

Have you ever noticed how little metadata you get from a video file?  Take any JPEG or RAW photo from an iPhone, a GoPro or a DSLR, and you will find extensive metadata stored in Exif, a photographic metadata standard that stores hundreds of properties such as exposure, focal length and camera model etc.  For your average MP4, MOV or AVI, not so much, particularly without the various sidecar files which are so easily lost. Some of the earliest GoPro HD cameras didn't even say "GoPro" anywhere in their video bit-streams.  I feel a manufacturer ID is the most fundamental of metadata: answering, what made this file?  Even some of the professional cameras I worked with at CineForm weren't much better.  Metadata should also answer, how was this file made? In what environment? At what location? Was the camera moving?  Etc. So why is photo metadata ubiquitous, and video metadata spotty at best.  The lack of useful video metadata relates to standards, or the lack of them. We have no Exif equivalent for video files, particularly for consumer video within MP4, so GoPro created one.

For GoPro cameras we couldn't just place an Exif within an MP4, as the exposure is changing per frame, so that would be a lot of Exif data, and Exif had no clear model for time varying signals. This photo standard also assumes that the metadata applies to a particular frame, that doesn't work so well for gyro and accelerometer data. GoPro needed to store both high frequency sensor data and Exif-like frame-based data to describe the image exposure.

A multiple-year effort begins.

The software ecosystem would love JSON or some XML variant, but embedded camera developers will have none of that, complaining of excessive memory usage, system load, blah blah blah.  The camera developers would love nothing more than storing raw I²C packets (actually proposed once), very low overhead, but completely opaque to software developers.  There needed to be a compromise.

How I accidentally became a Firmware engineer.

To investigate a possible low overhead metadata format, I started with the internal format used by CineForm's Active Metadata, it needed some work and extensions, but it was efficient, easy to read and write, and should appeal to both embedded firmware and software camps.  So I wrote a sample application to demo the reading and writing of metadata stored in any camera-native format.  I though it was only a model, I was thinking if the format was accepted the camera team would write something better. Firmware liked it, used it as is, then filed bug reports against it which I had to fix. That was about two years ago, and I've been part-timing as a firmware engineer ever since, focusing on metadata.

Fast-forward to today.

Here is a taste of GoPro metadata and telemetry:

Freq. (Hz)
3-axis accelerometer
All HERO5 cameras
3-axis gyroscope
All HERO5 cameras
lattitude, longitute, altitude, 2D ground speed, and 3D speed
HERO5 Black with GPS enabled
UTC time and data from GPS
Within the GPS stream
Within the GPS stream: 0 - no lock, 2 or 3 - 2D or 3D Lock
GPS precision (PDOP)
Within the GPS stream, under 300 is good
Image sensor gain
24, 25 or 30
HERO5 v2.0 or greater firmware
Exposure time
24, 25 or 30
HERO5 v2.0 or greater firmware

You can use some of this data today within GoPro's Quik Desktop tool, generating cool overlays, or do even more using the GoPro acquired Dashware tools.   A random video a created 8 months ago using GPMF extracted within Dashware, gauges rendered and composited in HitFilm:

Dashware+Telemtry from David Newman on Vimeo.

However, for this to be truly interoperable, its support needs to go beyond GoPro.  In a big step in that direction, GoPro has open-sourced the GPMF reader, as a way for third parties to collect, process and display video metadata from MP4s.

Developers please check it out  There is extensive GPMF documentation and simple code samples.