If fans of Netflixâ€™s Queer Eye have learned anything from Tan France, the flamboyant fashion consultant of the series, itâ€™s that a simple modification can take a look from fine to fabulous. Tricks like the French tuck or cuffing the sleeves of a T-shirt can create the illusion of a slimmer waist or a more robust bicep, all without changing the basic components of the look. Itâ€™s about working with what youâ€™ve got, then making it better.
Imagine, then, having your own personal Tan France to adjust your outfit every day. This kind of â€œminimal editingâ€� makes up new research from a group of computer scientists affiliated with Facebook AI Research, who have created a machine learning system called Fashion++ to make outfits more stylish with small changes. A suggestion might involve tucking in a shirt, adding a necklace, or cuffing a sleeve rather than changing into an entirely different outfit. The research will be presented later this month at the International Conference on Computer Vision.
Keep ReadingThe latest on artificial intelligence, from machine learning to computer vision and more
At this point in the story of AI, researchers have a good grasp on classic problems like object recognition or labeling the components of an image. In the fashion space, this has led to programs that can separate the individual components of an outfit (shirt, pants, shoes) and match the items in a photo to ones that are available to buy online. Pinterest, a leader in computer vision research with fashion applications, offers a tool that can zero in on one item in a pictureâ€”a black tulle skirt, for exampleâ€”and find similar items across its database of pins. Amazon has an analogous tool called StyleSnap, which uses machine learning to match an item in a photo to a similar garment for sale on Amazon.
Modeling creativity in fashion is a little more complex. â€œThink about a person trying to explain to another person their creative process, versus how to recognize a cat,â€� Kristen Grauman, a computer scientist at UT Austin who works with Facebook AI Research. â€œThese are very different ways of thinking.â€�
For Grauman, who contributed to the new research, this kind of work extends the effort to model creative problems with artificial intelligence. â€œSome of the challenges are around how you model things that are so small and subtle,â€� she says. â€œHow do you train a system and teach it these differences between â€˜goodâ€™ and â€˜slightly betterâ€™ outfits? How do you capture style in a computational way?â€�
While Fashion++ is pure research for now, you can easily picture it becoming a consumer-ready feature in one of Facebookâ€™s connected gadgets, like the Portal. Amazon already sells the Echo Look, a camera-enabled gadget that uses AI to choose the better of two outfits. â€œYou could imagine this future AI assistant that would have intelligence about what styles exist, what personal style is, what someone owns, and make intelligent suggestions,â€� says Grauman. If tech companiesâ€™ interest in fashion is any indication, that future wonâ€™t be far away.
Wear This, Not That
To build the data set for Fashion++, the researchers used thousands of publicly available images from the social fashion sharing site Chictopia, which features photos of real people wearing current trends. The definition of a â€œstylishâ€� outfit is constantly evolving, so the group opted for a set of photos that reflect whatâ€™s currently in style. The researchers then manipulated some of these photos to create a â€œworseâ€� version by swapping one part of the outfit with a garment from a different photo. These mismatches helped to train the model on how to improve the overall fashionability in an individual outfit.
The research also focuses on representing the various components of an outfitâ€”cataloging not just individual items (tops versus bottoms versus shoes), but also textures and shapes. â€œFor texture, things like materials or colors or things that have to do with the digital appearance,â€� says Grauman. Denim might create a more casual look; an all-black outfit might come across as more sophisticated. Different shapes, like a turtleneck versus a V-neck top, create different looks depending on how they are combined. â€œThe model learns which is more influential, which needs to be edited to be closer to the fashionable space,â€� says Grauman.