Timelapse and Image Recognition

A holy grail of camera trap analysis is automated image recognition, where a computer automatically inspects your images and identifies what is occurring within them. We will refer to two types of image recognition:

• A detector detects whether something is in an image or not, assigns a coarse identifying label to it (e.g., empty, animal, person, vehicle) as well as a confidence value that very roughly indicates the likelihood that the detection is correct.
• A classifier performs a fine-grained classification of what the detector detected. A classifier will typically produce a list of possible classifications for each detection, where each possibility is assigned a confidence value. For example, for wildlife image classification these categories will typically be different species of wildlife (e.g., elk, wolf, bear, dog).

Is image recognition useful for you?
Some knowledge will help you greatly when deciding on and when using image recognition. The time taken to peruse the materials below is well worth the effort.

• Read: Automated Image Recognition for Wildlife Camera Traps: Making it Work for You. Greenberg, S. (2020) Automated Image Recognition for Wildlife Camera Traps: Making it Work for You. Research report, University of Calgary: Prism Digital Repository, August 21, 15 pages.
  • This brief but must-read primer is oriented towards scientists (specifically wildlife ecologists) who are thinking about incorporating image recognition. It describes problems and prospects of using it for camera trap tagging, as well as how you can incorporate it effectively in the tagging workflow.
• Watch: Image Recognition for Camera Traps: Making it Work for You [~20 minutes]. In Conference: Scaling Up Camera Trap Surveys to Inform Regional Wildlife Conservation, Columbia Mountains Institute of Applied Ecology, May 18, 2021.
  • Like the primer, this video explains and illustrates image recognition: what it is, its fallibilities, and how it can still be incorporated for more efficient image tagging by adjusting your workflow..
• Try out image recognition in Timelapse on some practice images
  • Read the The Timelapse Image Recognition Guide, which explains how to use image recognition within timelapse
  • Watch Video Companion - Timelapse Image Recognition, which mirrors some of the workflow descriptions in the Timelapse Image Recognition guide.
  • Download the Image Recognition Practice Set), and try it out yourself on these sample image files

Timelapse and image recognition

Image recognition software is done by a 3rd party system
Timelapse does not do image recognition by itself. Rather, you use 3rd party software to do the actual image recognition processing.That software generates a file describing detections (and perhaps classifications) for each image. Timelapse imports that file. It uses that recognition data to display bounding boxes around selected images, and to allow you to select images by the image classification (e.g., all images with people in it).

Timelapse currently plays extremely well with MegaDetector, a free, highly reliable and really good detector for wildlife-based camera trap images. It is maintained by Dan Morris, with the help of others, and is open source. MegaDetector is widely used around the world, particularly by ecologists interested in detecting and classifying images into empty, people, wildlife and vehicles. An experimental wildlife classifier is also available to a limited audience.

If you want MegaDetector to produce an image recognition file for your images, you need to do the following.

1. Read the brief Getting started with MegaDetector
2. Try out MegaDetector on your own images by one of these approaches.
   • Approach 1: Download MegaDetector (or EcoAssist which simplifies things) and do it yourself. Follow these instructions to download MegaDetector to your own coomputer, where you can run it yourself on your own images. Running MegaDetector on your computer gives you the ability to try out MegaDetector quickly on images of your choosing while still keeping everything in-house.
   • Approach 2: Submit your images. email Dan Morris cameratraps@lila.science, who manages MegaDetector, for information on how to submit your images to him. After you do that, he will send you the image recognition file. Dan is a great guy, and does this for free on a volunteer basis. Give him lots of credit. Submitting your files to Dan is an easy way to get a recognition file, but requires some back and forth emails. If your organization has privacy restrictions on sending images to an outside person, you may need to get their permission to do so.

Then use Timelapse to try out image recognition on your own camera trap files
MegaDetector will produce the recognition file needed by Timelapse. When you get it, follow the example workflow detailed in Timelapse Image Recognition Guide. That guide explains how to read that file into Timelapse and use the image recognition results. Generally speaking:

• Timelapse will display recognized entities as labelled bounding boxes.
• Timelapse lets you use that recognition information in selecting subsets of images e.g., all empty images, all images with people in it, all animal images, (and if classifications are included, a specific species).
• You can streamline your workflow by inspecting that category to see how well the recognizer is doing, quickly reviewing and correcting any errors, and tagging them all in a single operation.