Bizarre AI camera turns images into sonnets and haiku

It looks like a homemade Polaroid camera, perhaps made with one of the best 3D printers. But this AI-powered device doesn't take photos at all. It 'captures' poems.

The Poetry Camera turns the visual input it receives through its lens into an AI-generated poem that describes the scene. It's kind of like an AI image generator but in reverse: it takes a real image and generates what could be compared to a prompt used in a text-to-image tool, but with more poetic language.

The Poetry Camera is an open-source passion project created by Kelin Carolyn Zhang and Ryan Mather. They describe it as a "new way to make memories — away from screens, notifs, and apps". The device houses a Raspberry Pi single-board computer to capture and process images, and then uses OpenAI’s GPT-4 AI model to generate poetry based on the extracted data about objects, situations, colour, patterns and other elements. Tech-savvy users can modify the source code to choose different types of poetic forms, like sonnets, haiku or free verse.

The creators of the camera have been putting it to the test in collaboration with photographers and they say they're planning a limited drop sometime soon. The camera throws out metaphors a plenty, and varies between the literal and the whimsical. A park bench becomes 'wooden boards' and a guy on a motorbike is on a 'chrome-plated pilgrimage'.

Mather told TechCrunch the device was conceived as a reaction to Instagram culture, generating text instead of a photo. “Everyone prefers the book version over the movie, so it’s like that for capturing moments," he said (is this the moment at which I should point out that the best books generally weren't written by AI?).

Despite all of the concerns about AI, and the fact that the 'poems' aren't exactly original works of literature, I actually really love this project. It's AI art at its most philosophical, delving into the question of what an image is, and what poetry is? If we rely on machines (the camera) to capture a visual record of a memory, can we rely on a different type of machine to generate a literary record? Is a poetry camera to literature what the camera itself is to painting?

Is an image really worth a thousand words (or a few dozen in this case)? And, which says more the image or the poem? What would result if we then feed the resulting poem into an AI image generator to create a new image? Would it be recognisable? The questions are endless, which is generally a sign of good art. That's something we can't say about the first AI romance movie.

For more AI art projects, see the AI fake retro movie trailers for films that never were.