New camera reveals what life looks like through animals’ eyes with near-perfect accuracy

BRIGHTON, United Kingdom — Have you ever wondered how animals perceive the colorful world around them? Scientists have long been fascinated by this question, and now, thanks to new research, we’re closer to understanding the colorful universe through the eyes of animals. A groundbreaking video camera is allowing people to replicate how various animals see the world.

The camera system, boasting over 92 percent accuracy, allows filmmakers to accurately depict colors as different animals see them. This technological advancement is not only beneficial for filmmakers but also provides scientists with a valuable tool to understand animal communication and navigation more effectively.

Animals experience the world differently from humans, thanks to their unique photoreceptors – the cells in the eye that respond to light. These photoreceptors can detect a range of colors from ultraviolet to infrared, far beyond human capability. Animals can even see polarized light, which is invisible to us. This diversity means each animal has its distinct way of perceiving color. However, there’s a catch. Our eyes and even the most advanced cameras can’t capture this vast spectrum of light. That’s where the new research comes in.

Scientists developed a cutting-edge tool that lets us see colors as various animals do, especially when they move. Movement can change the way colors look, and until now, this dynamic aspect of animal vision has been a mystery

Iridescent peacock feather through the eyes of 4 different animals.
The camera system can measure angle-dependent structural colors such as iridescence. This is illustrated here through an image of a highly iridescent peacock (Pavo cristatus) feather. The colors in this video still image represent (A) peafowl Pavo cristatus false color, where blue, green, and red quantum catches are depicted as blue, green, and red, respectively, and the UV is overlaid as magenta. Although broadly similar to a standard color video, the UV-iridescence can be seen on the blue-green barbs of the ocellus (“eyespot”). Further UV iridescence can be seen along the perimeter of the ocellus (between the outer 2 green stripes). Interestingly, the iridescence is more notable to the peafowl than to (B) humans (standard colors), (C) honeybees, or (D) dogs.

The Challenge of Capturing Animal Colors

Traditionally, researchers relied on spectrophotometry to study animal vision, a method that is often time-consuming, requires specific lighting conditions, and is unable to capture moving images. In response to these limitations, a team led by the University of Sussex developed this innovative camera and software system, capable of recording animal-view videos (see clip at end of article). 

The team then turned to multispectral photography, which captures images in different wavelength ranges, including ultraviolet and infrared. This method provides richer spatial details but still falls short in capturing the movement and temporal changes in color.

To address these limitations, researchers are now combining multispectral imaging with 3D digital modeling. This innovative approach lets them animate and study these 3D models under various simulated conditions, providing insights into how animal postures and viewpoints affect color perception.

A Revolutionary Camera System

The unique camera functions by recording videos across four color channels: blue, green, red, and ultraviolet (UV). These recordings are then processed to create videos that accurately represent how animals perceive these colors, based on our current understanding of their eye photoreceptors. The captured data is then transformed into a format that reflects how specific animals perceive these colors.

To do this, the research team designed an easy-to-use pipeline combining existing multispectral photography methods with their new hardware and software. The system uses a beam splitter to separate ultraviolet from visible light, directing these to two different cameras. The recorded footage is then processed to match the color perception of specific animals, like honeybees or UV-sensitive birds.

This method is not only groundbreaking in its scientific precision but also practical. It uses commercially available cameras and 3D-printed housing, making it accessible for wider research use. The team has ensured that all components of their system are open source, inviting further improvements and adaptations.

“We’ve long been fascinated by how animals see the world. Modern techniques in sensory ecology allow us to infer how static scenes might appear to an animal; however, animals often make crucial decisions on moving targets (e.g., detecting food items, evaluating a potential mate’s display, etc.),” says Assistant Professor Daniel Hanley from George Mason University in Virginia, in a media release. “Here, we introduce hardware and software tools for ecologists and filmmakers that can capture and display animal-perceived colors in motion.”

This approach marks a significant leap forward, as it captures the full complexity of visual signals in their natural context, with all the nuances of movement and varying light conditions.

“Many animals, including bees, birds and even mammals like reindeer and mice can perceive ultraviolet light. Indeed, the lack of UV-sensitivity in humans is more of an exception than the rule. At the other end of the visible light spectrum, human eyes have receptors that are sensitive to red while many animals – including bees, mice and dogs – are just as blind to red as we are to ultraviolet light,” Hanley writes in a post published on The Conversation. “Even when it comes to blues and greens, colors perceived across the animal kingdom, the precise wavelength of the light an animal would experience as “pure blue” or “pure green” is specific to the species. As a result, no two species see the world in the same colors.”

Here, we show the application of UV-blocking sunscreen in honeybee false colors.
HONEYBEE PERSPECTIVE: Here, we show the application of UV-blocking sunscreen in honeybee false colors. As in other depictions, we show the honeybee’s UV, blue, and green photoreceptor responses as blue, green, and red, respectively. Note that the light toned skin (DH) appears similar in honeybee false colors as in human vision, because skin reflectance increases progressively at longer wavelengths. The sunscreen appears white to our eye because it reflects broadly over the human visible range, but it appears yellow in honeybee false colors because it absorbs UV light. (Credit: PLoS Biology)
The image is shown in honeybee false colors (displaying the honeybee’s UV, blue, and green photoreceptor responses as blue, green, and red, respectively).
HONEYBEE PERSPECTIVE: The camera system is capable to capture naturally occurring behaviors in their original context. This is illustrated with an image that depict bees foraging and fighting in their natural environment. The image is shown in honeybee false colors (displaying the honeybee’s UV, blue, and green photoreceptor responses as blue, green, and red, respectively). (Credit: PLoS Biology)
Here, we illustrate 2 northern mockingbirds interacting in a tree, in avian false colors.
BIRD’S EYE VIEW: Here, we illustrate 2 northern mockingbirds interacting in a tree, in avian false colors. Specifically, we show blue, green, and red quantum catches as blue, green, and red, respectively, and UV quantum catches are overlaid as magenta. While the 80 mm lens is not designed for imaging distant subjects, the system captures avian-view imagery well and shows the “avian white” (reflective from the UV through the visible portions of the spectrum) patches of their feathers. It also illustrates that the sky as predominantly UV-colored (i.e., appearing magenta), due to shorter wavelengths being subjected to increased Rayleigh scattering. Thus, while the sky may appear blue to our eyes, it would appear UV-blue to many other organisms. (Credit: PLoS Biology)
 Here, we show a video of a black swallowtail Papilio polyxenes caterpillar displaying its osmeteria. We illustrate this video in honeybee false colors .
HONEYBEE PERSPECTIVE: Conceal and reveal displays can pose a problem for spectroscopy and standard multispectral photography. Here, we show a still shot of a black swallowtail Papilio polyxenes caterpillar displaying its osmeteria. We illustrate this image in honeybee false colors such that UV, blue, and green quantum catches are shown as blue, green, and red, respectively. The (human) yellow osmeteria as well as the yellow spots along the caterpillar’s back both reflect strongly in the UV and appear magenta when the colors are shifted into honeybee false colors (as the strong responses on the honeybee’s UV-sensitive and green-sensitive photoreceptors are depicted as blue and red, respectively). Many predators of caterpillars perceive UV, and accordingly, this coloration might be an effective aposematic signal. (Credit: PLoS Biology)
We illustrate an image depicting 2 eastern leaf-footed bug (Leptoglossus phyllopus) eggs.
HONEYBEE PERSPECTIVE: Image depicting 2 eastern leaf-footed bug (Leptoglossus phyllopus) eggs. These eggs are approximately 2 mm in diameter and were on the base of a small leaf, which is shown blowing in the wind in honeybee false color (photoreceptors sensitive to UV, blue, and green light are shown as blue, green, and red, respectively). This demonstrates the capability of the system to image small items close up. (Credit: PLoS Biology)
Museum specimen of a Phoebis philea butterfly in avian RNL false colors.
BIRD’S EYE VIEW: Museum specimen of a Phoebis philea butterfly in avian RNL false colors. Another potentially useful application of the system is the fast digitization of museum specimens. This butterfly possesses both pigmentary and structural UV coloration. Bright magenta colors highlight the predominantly UV-reflective areas, while the areas appearing purple reflect similar amounts of UV and long wavelength light. The specimen is mounted on a stand, showcasing how the iridescent colors change depending on viewing angle. (Credit: PLoS Biology)
Rainbows through animal eyes
Certain scenes would be challenging to measure using traditional methods and would appear entirely distinct to different animal viewers. As an illustration of this point, we show the same rainbow in (A) mouse, (B) honeybee, and (C) avian false colors, alongside a (D) video with standard colors (i.e., human colors)
Image of small (unidentified) jumping spider in honeybee false colors
HONEYBEE VIEW: Image of small (unidentified) jumping spider in honeybee false colors (displaying the honeybee’s UV, blue, and green photoreceptor responses as blue, green, and red, respectively) (Credit: PLoS Biology)

Implications and Future Directions

This advancement opens up numerous possibilities for researchers. Now, they can study how animals perceive dynamic visual signals in their natural habitats, a crucial aspect of understanding animal behavior and communication.

For instance, researchers can now capture rapid changes in color signals in real-world settings, like a leaf fluttering in the wind or a bird moving through undergrowth. This was previously impossible with static imaging techniques.

This new tool promises to revolutionize our understanding of sensory ecology. By revealing the unseen world of animal vision, as shared in the video below, scientists can explore new frontiers in the study of animal behavior, communication, and evolution. The research paper on the study is published in PLoS Biology.

YouTube video

Try it yourself

“We have made all codes necessary for implementing the video conversions and the plans of the camera system freely available online, along with our best attempt to explain how to build the camera from scratch,” writes Hanley on The Conversation. “Our goal is for other researchers to build their own cameras and to use them to answer their own questions about how other species see the world. There are so many possibilities.”

South West News Service writer Isobel Williams contributed to this report.


Comments

  1. Why not do a side by side comparison (or one of those slider bits) showing the difference?

    1. It cracks me up to see all these entirely made up statistics about how accurate this is. You dont know that. Nobody does. Nobody will. This is a guess.

    2. I put a picture of Obama and Biden in our cats litter box, and he could see it well enough to cover it up! Animals are amazing huh?

  2. Fine. How do we know WHAT color a cat, a rat, a bat, etc. perceives? You do your camera set-up but does that [annoying] mockingbird really see “magenta”, “mauve”, “ochre”, or “Pullman Green” for that matter? I see a McIntosh apple with a certain “redness” but is that “redness” actually the “greeness” my downstairs neighbor sees when looking at a lime? Usually we read that dogs only see black-and-white; awhile back a piece in the Daily Mail said cats saw yellows but not other colors, etc. and feline overhead vision was stilted. So creatures grow up and figure out what they’re seeing and act accordingly. Goes beyond colors: put up a picture of Joe Biden: who sees the Second Coming of J-s-s Chr-st–vs. who sees the Second Coming of Jimmy Carter?
    I had forgotten about Study Finds. Go out and find Some Other Clickbait vaguely interesting. Haven’t watched an amusing cat video lately.

    1. lots of great cat videos on x (twitter) check out “why you should have a cat”

    2. Excellent comment. I have not even a minor academic interest in what any bird’s vision perceives. Eliminate felines and canines and you have eliminated 90% of the human audience anyway.

  3. Most people are only interested in two particular animals and this article did not even show how they might see. Dogs and cats. Why show anything else unless you just want to disappoint the reader.

    1. I asked my cat to comment on this article and he declared it species-ist since it did not address the issue of how our feline overlords viewed the world. I think he has a point.

    2. 100% this comment. Don’t care what a hummingbird ‘sees’. Care what my dog sees. Didn’t show me, even after leading with an image of a cat and dog.

  4. Seems like every few years someone claims to know what and how Animals see . A Slightly educated guess at best .

    1. Why write an article on dog & cat vision, then show photos of what a bird, or insect ‘might’ see? Boo. Boo. Bad

    1. All you would see is rainbows and lollipops. And probably a dollar signs from ripping the American people off. And making us look like a mockery to the rest of the world

      1. I don’t disagree with your comment, but, we have been looking pretty disgraceful for a number of years now. Biden doesn’t help things and neither did Trump. We need to elect a real leader. Now back to the bird vision!

  5. If some animals see light spectrums that a human can’t see, how is it that you can show us how they see? I don’t get it. What does UV look like since we can’t see it? It’s impossible to show us. You can only “simulate” it.

  6. If this actually works, would be cool to develop an application for something like Apple Vision Pro that shows you how an animal sees and you can walk around your house looking at everything to see what your pets see.

  7. I disagree with the choice to make UV appear magenta, which is a mixture of blue and red. Rather, the electromagnetic spectrum should have been compressed towards green. UV would show as violet, violet as blue, blue as cyan. Similarly, IR would appear as red, red as orange, orange as yellow.

  8. It’s interesting that the scientists are guessing at how the “world” would appear based on suspected species’ eye chromaticity, but it doesn’t account for proportion of rods, tonal response, eye construct, or even brain mapping. Swing and a miss, but interesting, nonetheless.

  9. They see different parts of the color spectrum as their eyes are sensitive to those ranges. We have no idea how their brains render that into their perception.

  10. The picture at the top of the article is of a dog and cat. The actual article describes the vision of a couple of bugs and maybe some birds. First of all, I’d like to know how the authors determined that this is what those species would see, although I’m willing to give them the benefit of the doubt. But frankly, I am much more interested in how other mammals view the world.

  11. How do we know how these animals’ brains further adjust the received photon stimuli? Could be they perceive a completely different color palette than we assume they do.

  12. Makes no sense. If these pictures are examples of what birds and bugs see, then they have poor vision and wouldn’t be able able to find their food.

  13. Even assuming (it’s a stretch) that they nailed the color scheme here, it doesn’t really show us anything. I mean, when a dog or bird look at a piece of art, do they see it as art… Just in a different color? I think not. Perception is everything and without that added in this doesn’t help us understand a thing about animal vision. I mean, I bet a dog’s nose ‘sees’ as much or maybe even more than it’s eyes. Cool idea, but not within our grasp of reality

  14. My cats sees ghost. It’s so disturbing because I can’t see what is scaring the hell out of them looks like. I’ll be impressed when “they” can show us life from a dog or a cats eyes as well. Until then, who cares !


Comments are closed.