It was bad enough that companies like Facebook and Google relentlessly mine your information and sell it to advertisers. But now third-party startups are getting into the mix. Thanks to unprecedented access to photos uploaded to apps like Instagram, these third-parties are scanning and storing people's photos on behalf of major advertisers.

The Wall Street Journal delves into this practice of into public picture analysis, including a company called Ditto Labs, which specializes in scraping photographs from Twitter, Instagram, and Tumblr for brand information and customer feedback. All the information they glean from selfies and candid shots allows companies to do extensive market research.

Advertisers such as Kraft Foods Group Inc. pay Ditto Labs to find their products' logos in photos on Tumblr and Instagram. The Cambridge, Mass., company's software can detect patterns in consumer behavior, such as which kinds of beverages people like to drink with macaroni and cheese, and whether or not they are smiling in those images. Ditto Labs places users into categories, such as "sports fans" and "foodies" based on the context of their images.

Kraft might use those insights to cross-promote certain products in stores or ads, or to better target customers online. David Rose, who founded Ditto Labs in 2012, said one day his image-recognition software will enable consumers to "shop" their friends' selfies, he said. Kraft didn't respond to a request for comment.

Advertisers can already directly target photo-sharers. According to Ditto's founder, they already have a service which enables advertisers to "target specific users based on their photos posted on Twitter." But Ditto has found that corporations are resistant to the new technology, fearing customers will think it's "creepy."

Right now, Ditto has a realtime "firehose" of all photographs published on social media, allowing curious people to filter for topics like "coffee," "candy," and "beer." You can see how creepy it is for yourself.

Update 3:45pm: Vans and The North Face both emailed statements saying they do not use Ditto, despite the startup prominently featuring both brand on their homepage:

I can assure you that Vans does not utilize Ditto, yet the image in the article undoubtedly implies that we do. Any time a consumer's photo is used within Vans' social media platforms, they have tagged Vans and we have asked for permission to post it after an internal review of current photos uploaded to any platform.

To contact the author of this post, please email