“‘Every AI for everyone’ is kind of our tagline,” says Gupta. “We have organized all the AI models we can find today.” Yupp’s website encourages developers to reach out if they want their language or image model added to the options. It doesn’t currently have any deals with AI model builders, and provides these responses by making API calls.
Every time someone uses Yupp they are participating in a head-to-head comparison of two chatbot models, and sometimes getting a reward for providing their feedback and picking a winning answer. Basically, it’s a user survey disguised as a fun game. (The website has lots of emoji.)
He sees the data trade off in this situation for users as more explicit than past consumer apps, like Twitter—which he’s quick to tell me that he was the 27th employee at and now has one of that company’s cofounders, Biz Stone, as one of his backers. “This is a little bit of a departure from previous consumer companies,” he says. “You provide feedback data, that data is going to be used in an anonymized way and sent to the model builders.”
Which brings us to where the real money is at: Selling human feedback to AI companies that desperately want more data to fine tune their models.
“Crowdsourced human evaluations is what we’re doing here,” Gupta says. He estimates the amount of cash users can make will add up to enough for a few cups of coffee a month. Though, this kind of data labeling, often called reinforcement learning with human feedback in the AI industry, is extremely valuable for companies as they release iterative models and fine tune the outputs. It’s worth far more than the bougiest cup of coffee in all of San Francisco.
The main competitor to Yupp is a website called LMArena, which is quite popular with AI insiders for getting feedback on new models and bragging rights if a new release rises to the top of the pack. Whenever a powerful model is added to LMArena, it often stokes rumors about which major company is trying to test out its new release in stealth.
“This is a two-sided product with network effects of consumers helping the model builders,” Gupta says. “And model builders, hopefully, are improving the models and submitting them back to the consumers.” He shows me a beta version of Yupp’s leaderboard, which goes live today and includes an overall ranking of the models alongside more granular data. The rankings can be filtered by how well a model performs with specific demographic information users share during the sign-up process, like their age, or on a particular prompt category, like health-care related questions.
Near the end of our conversation, Gupta brings up artificial general intelligence—the theory of superintelligent, human-like algorithms—as a technology that is imminent. “These models are being built for human users at the end of the day, at least for the near future,” he says. It’s a fairly common belief, and marketing point, among people working at AI companies, despite many researchers still questioning whether the underlying technology behind large language models will ever be able to produce AGI.
Gupta wants Yupp users, who may be anxious about the future of humanity, to envision themselves as actively shaping these algorithms and improving their quality. “It’s better than free, because you are doing this great thing for AI’s future,” he says. “Now, some people would want to know that, and others just want the best answers.”
And even more users might just want extra cash and be willing to spend a few hours giving feedback during their chatbot conversations. I mean, $50 is $50.
Read the full article here