Apple’s Live Photo: Two Separated Files

Apple’s Live Photo: Two Separated Files

Aug 13, 2023 04:40 PM
Last edited time
Aug 15, 2023 03:18 AM
I used to think that Apple's use of HEIF (High Efficiency Image File Format) was for supporting its Live Photo feature. However, both I and the Internet were mistaken.
When you connect your iPhone to a PC or a Mac and check its album folder, you will find that Apple stores Live Photos in two separate files: an HEIC file and a MOV file.
HEIC stands for High Efficiency Image Container, a variant of HEIF that stores static photos. The MOV file format is used to store videos in the HEVC (High Efficiency Video File Format).
That's it. The Live Photo is just a combination of a static photo with a short video.

So, can I make a live photo?

Now that we have a grasp of how Live Photos works, can we merge any still photo with a random short video to create a "Live Photo" in the iOS album?
My first attempt was to change the filename of both a photo and a video to be identical. Then, dragging them into the macOS Photo App. However, the app failed to recognize them as live photos. Of course, it's not that easy.
After reading the documents, I found a rule in the "metadata" field of both the image and the video: a magical number called “identifier” in these two files should be the same.
The information is outdated. Previously, Apple stored a live photo using a JPG file and a H.264 video file. They have now upgraded to using a HEIC file and a H.265 video file.
But anyway, let’s check the old rules first, and see if they still apply to new formats.
A live photo has two resources. They are tied together with an asset identifier (a UUID as a string).
  1. A JPEG; this must have a metadata entry for kCGImagePropertyMakerAppleDictionary with [17 : assetIdentifier] (17 is the Apple Maker Note Asset Identifier key).
  1. A Quicktime MOV encoded with H.264 at the appropriate framerate (12-15fps) and size (1080p). This MOV must have:
      • Top-level Quicktime Metadata entry for ["com.apple.quicktime.content.identifier" : assetIdentifier]. If using AVAsset you can get this from asset.metadataForFormat(AVMetadataFormatQuickTimeMetadata)
      • Timed Metadata track with ["com.apple.quicktime.still-image-time" : 0xFF]; The actual still image time matches up to the presentation timestamp for this metadata item. The payload seems to just be a single 0xFF byte (aka -1) and can be ignored. If using an AVAssetReader you can use CMSampleBufferGetOutputPresentationTimeStamp to get this time.
The assetIdentifier is what ties the two items together and the timed metadata track is what tells the system where the still image sits in the movie timeline.
Excellent! Let's quickly create a prototype. I think adding a metadata field is actually more difficult than modifying an existing one. Therefore, our goal is to swap the video part of two live photos.
First I took two live photos using my iPhone (13 Pro, iOS 17.0), and export them as separated HEIC and MOV files.
notion image
Let's write a Swift script to fetch metadata from HEIC photos. Swift is much more efficient in handling HEIC photos compared to Python, as I discovered after spending two hours struggling with Python libraries.
Let's write another Swift script to fetch metadata from MOV files. It's unfortunate that ChatGPT only has knowledge up until 2019, as writing Swift code can be challenging due to Apple's annual annoying API changes. Luckily, I still have GitHub Copilot Chat, which is great for development purposes.
And I got the same asset identifier, the rules still apply.
notion image
To swap the video portion in these two live photos, we can modify either the identifier in HEIC or MOV. For simplicity's sake, let's attempt to modify the HEIC file. Add another function.
This function can modify the identifier in HEIC photos to the constant string value in modifiedAssetIdentifier.
notion image
OK, now change the filename of replace.mov to example_modified.HEIC.
Drag both of example_modified.HEIC and example_modified.mov into macOS Photo App, and — boom! We have a swapped new live photo.