Captions, an AI-powered video enhancing startup, has launched a brand new app referred to as Lipdub for translating clips into 28 languages.
Lipdub is offered within the App Retailer free of charge and helps a number of languages, together with French, Hindi, Spanish, Italian, Portuguese, Japanese and extra. The app even lets customers translate movies into Texas slang, Gen Z, pirate and child discuss. The demo video reveals that the app may change lip motion in response to the chosen goal language. Nevertheless, at instances there’s a sure lag between the audio and the lip motion.
Customers can translate a video of a single particular person speaking throughout as much as one minute after which share it on different social media platforms.
On its web site, Captions says that greater than 3 million creators have used its eponymous video enhancing app. The startup claims that it has greater than 100,000 day by day customers. The Captions app presents a number of AI-powered options round video enhancing, equivalent to eradicating “ums” and “ahs,” lowering background noise and enhancing speech. The app additionally has an “AI Lipdub” function that may change lip motion in post-production enhancing for those who change the transcript.
Captions was based in 2021 by Gaurav Misra, who was head of design engineering at Snap. In June, the corporate secured $25 million in a Series B round led by Kleiner Perkins with participation from Sequoia Capital, Andreessen Horowitz (a16z) and SV Angel. To this point, Captions has raised $40 million in funding.
Utilizing translation and AI-dubbing to succeed in a wider viewers is a rising pattern. In June, YouTube introduced that it’s testing an AI-powered instrument to let customers routinely dub their movies in different languages. The corporate stated it’s even engaged on higher lip-syncing. Final month, the corporate stated that it’s integrating AI-powered dubbing instantly into YouTube Studio for simpler entry for folks seeking to convert movies into different languages.
Earlier this month, AI-powered voice-generating platform ElevenLabs released its dubbing tool with help for 29 languages. Rest of World beforehand reported that dubbing service supplier firms are producing thousands and thousands of {dollars} by translating content material for well-liked YouTubers like MrBeast.
AI-powered dubbing startups have generated plenty of investor curiosity, with startups like U.Ok.-based Papercup and Isreal-based Deepdub elevating thousands and thousands of {dollars}.