Timelapse and Image Recognition

A holy grail of camera trap analysis is automated image recognition, where a computer automatically inspects your images and identifies what is occurring within them. We will refer to two types of image recognition:

A detector detects whether something is in an image or not, assigns a coarse identifying label to it (e.g., empty, animal, person, vehicle) as well as a confidence value that very roughly indicates the likelihood that the detection is correct.
A classifier performs a fine-grained classification of what the detector detected. A classifier will typically produce a list of possible classifications for each detection, where each possibility is assigned a confidence value. For example, for wildlife image classification these categories will typically be different species of wildlife (e.g., elk, wolf, bear, dog).

Is image recognition useful for you?
Some knowledge will help you greatly when deciding on and when using image recognition. The time taken to peruse the materials below is well worth the effort.

Read: Automated Image Recognition for Wildlife Camera Traps: Making it Work for You. Greenberg, S. (2020) Automated Image Recognition for Wildlife Camera Traps: Making it Work for You. Research report, University of Calgary: Prism Digital Repository, August 21, 15 pages.
- This brief but must-read primer is oriented towards scientists (specifically wildlife ecologists) who are thinking about incorporating image recognition. It describes problems and prospects of using it for camera trap tagging, as well as how you can incorporate it effectively in the tagging workflow.
Watch: Image Recognition for Camera Traps: Making it Work for You [~20 minutes]. In Conference: Scaling Up Camera Trap Surveys to Inform Regional Wildlife Conservation, Columbia Mountains Institute of Applied Ecology, May 18, 2021.
- Like the primer, this video explains and illustrates image recognition: what it is, its fallibilities, and how it can still be incorporated for more efficient image tagging by adjusting your workflow..
Try out image recognition in Timelapse on some practice images
- Read the The Timelapse Image Recognition Guide, which explains how to use image recognition within timelapse
- Watch Video Companion - Timelapse Image Recognition, which mirrors some of the workflow descriptions in the Timelapse Image Recognition guide.
- Download the Image Recognition Practice Set), and try it out yourself on these sample image files

Timelapse and image recognition

Image recognition software is done by a 3rd party system
Timelapse does not do image recognition by itself. Rather, you use 3rd party software to do the actual image recognition processing.That software generates a file describing detections (and perhaps classifications) for each image. Timelapse imports that file. It uses that recognition data to display bounding boxes around selected images, and to allow you to select images by the image classification (e.g., all images with people in it).

Timelapse currently plays well with two image recognition systems, both with somewhat different capabilities.

MegaDetector. Microsoft’s AI for Earth program developed MegaDetector, a free, highly reliable and really good detector for wildlife-based camera trap images. MegaDetector is widely used around the world, particularly by ecologists interested in detecting and classifying images into empty, people, wildlife and vehicles. While MegaDetector is stored in the Microsoft repository, it is now maintained by Dan Morris , who has since moved on from Microsoft. An experimental wildlife classifier is also available to a limited audience.

If you want MegaDetector to produce an image recognition file for your images, you need to do the following.
1. Read the brief Getting started with MegaDetector
2. Try out MegaDetector on your own images by one of these approaches.
  - Approach 1: Download MegaDetector and do it yourself. Follow these instructions to download MegaDetector to your own coomputer, where you can run it yourself on your own images. Running MegaDetector on your computer gives you the ability to try out MegaDetector quickly on images of your choosing while still keeping everything in-house.
  - Approach 2: Submit your images. email Dan Morris cameratraps@lila.science, who manages MegaDetector, for information on how to submit your images to him. After you do that, he will send you the image recognition file. Dan is a great guy, and does this for free on a volunteer basis. Give him lots of credit. Submitting your files to Dan is an easy way to get a recognition file, but requires some back and forth emails. If your organization has privacy restrictions on sending images to an outside person, you may need to get their permission to do so.
Zendo from DeepAI. A commerical (fee for service) alternative is Zendo from DeepAI. Unlike MegaDetector, Zendo does classification. Importantly, you can, through training, customize the particular visual objects you are interested in detecting and classifying. Zendo allows you to label and curate your training data entirely within their cloud-based system, or on your own hardware. After training, you can run the recognizer on your own images, and have Zendo output a recognition file into the format recognized by Timelapse. While Zendo works on a fee for service, you can try it out for free through their 60-day free trial period, or their limited community version (described on the bottom of the Zendo page). Mail: zendo@deepai.org

Disclaimer: We are not associated with DeepAI, and have only limited experience with Zendo. We recommend discussing your needs and pricing policy with the DeepAI folks, perhaps trying out their free versions on a subset of your images to see how well it works for your needs.

Then use Timelapse to try out image recognition on your own camera trap files
Both image recognition systems produce the recognition file needed by Timelapse. When you get it, follow the example workflow detailed in Timelapse Image Recognition Guide. That guide explains how to read that file into Timelapse and use the image recognition results. Generally speaking:

Timelapse will display recognized entities as labelled bounding boxes.
Timelapse lets you use that recognition information in selecting subsets of images e.g., all empty images, all images with people in it, all animal images, (and if classifications are included, a specific species).
You can streamline your workflow by inspecting that category to see how well the recognizer is doing, quickly reviewing and correcting any errors, and tagging them all in a single operation.