Alec Radford

Computers Learn How to Paint Whatever You Tell Them To

Advancements in artificial intelligence give machines an imagination and produce some strange imagery.

Most self-respecting artists wouldn't agree to paint a portrait of a toilet in the middle of a field. Fortunately, advancements in artificial intelligence have given computers the ability to imagine just about any scenario, no matter how bizarre, and illustrate it.

A toilet seat sits open in the grass field.
A toilet seat sits open in the grass field.
Source: University of Toronto

Take a look at this image. It is a collage of toilets in fields, generated from computers' imaginations. Let that soak in for a minute.

Researchers at the University of Toronto have devised a system that allows a user to input keywords—say, "toilet in a field," or whatever other odd scenario one might think up—and get original, computer-drawn images.

Similar projects are under way in AI labs across Silicon Valley. The latest breakthrough is an extension of past research by tech companies, such as Microsoft and Baidu, on describing the contents of an image. This is different from the hallucinatory Deep Dream generator that came out of a Google research project this year. While that effort enabled computers to draw little details onto images, displaying a spooky bias toward dog faces and eyeballs, it was unable to compose large, coherent pictures.

The toilet picture, which comes from a paper published last month, is blurry and small. That's because training the underlying technology is a highly complicated process. Separate research, however, from Indico Data Solutions, a machine-learning startup, gives a glimpse of higher-quality, computer-imagined paintings. In collaboration with an AI researcher at Facebook, Indico developed a system to generate pictures of human faces. This is what they got:

Synthetic faces generated via a neural network
Synthetic faces generated via a neural network
Alec Radford

Kind of creepy, no? To create the AI tool, the team took what's known as a neural network, which is software designed to spot patterns and important features in a piece of content, and essentially reversed it. Instead of asking the software to look for features within an image, it was asked to generate features onto an image, creating faces or pictures of bedrooms. The tool isn't as complicated as the one from Toronto, which gives its neural network the ability to understand the relationship between images and language. But one important aspect of Indico's research is how remarkably little the computer has to be taught about the world before it can start generating its own, realistic-looking images, said Alec Radford, a co-founder of Indico who worked on the project.

The development has created a stir in the research community. Ian Goodfellow, a Google senior research scientist, said he looks forward to discussing it with other researchers at conferences next year and was pleasantly surprised to see it come out of a startup.

The research could have serious business implications that could one day put set designers and sketch artists out of a job. Tech companies have searched for ways to automate various aspects of the design process, including Autodesk's attempts to build software that can essentially grow furniture or work by a startup called the Grid, which claims to use AI for generating websites from scratch. As tech companies such as Facebook and Google rapidly decrease the time it takes to incorporate AI into products available to everyone, such research could appear in the real world sooner than you think.

A very large commercial plane flying in rainy skies.
A very large commercial plane flying in rainy skies.
Source: University of Toronto

Ruslan Salakhutdinov, an assistant professor at the University of Toronto who worked on the toilet project, said this research had a side benefit of helping them learn more about how neural networks work. "We can better understand why the model generates the image it generates by examining which words it pays attention to (and which words it ignores)," he wrote in an e-mail. That could help advance other kinds of AI research beyond those involving pictures of toilets in fields.