Describe: YouTube summarizer (based on text/audio and visuals)

Ask specific questions about your target languages. Beginner questions welcome!
Kraut
Black Belt - 2nd Dan
Posts: 2630
Joined: Mon Aug 07, 2017 10:37 pm
Languages: German (N)
French (C)
English (C)
Spanish (A2)
Lithuanian
x 3245

Describe: YouTube summarizer (based on text/audio and visuals)

Postby Kraut » Fri Mar 22, 2024 11:08 am

Describe: YouTube summarizer

https://www.youtube.com/watch?v=LgJ3UGrdt1k
Spanish Man Does Laundry - Beginner Spanish

the examples are in English, I haven't found any settings that will create the original Spanish
https://www.sievedata.com/functions/sie ... 2013414934

MyAIadvantage
Describe is a new AI tool that is tackling this issue using the power of Visual Language Models (VLMs.) When you upload a video to Describe, it analyzes both the visual and audio components of the video and gives you a summary that takes both into account.

Here's how to use Describe:

No need to log in or sign up, simply click the link to begin testing.
Upload a short video clip into the "Inputs" box on the right.
Adjust settings for Conciseness, Visual Detail, and Spoken Context (or leave them as default)
Click "Submit" at the bottom

Try comparing Describe's summary to a ChatGPT summary of the same transcript, and you'll see the difference is night-and-day.


In this video, a man humorously demonstrates how to do laundry in Spain, integrating daily chores with a light-hearted take on cultural practices. Initially, he sorts laundry, puzzled by the many settings on his washing machine before choosing a quick 30-minute cycle. Amidst the process, he comically realizes he only uses dark clothes due to being a metal fan, simplifies the use of detergent by discarding fabric softener as "not for real men," and mistakenly picks up a dish soap. After washing, he opts not to use a dryer, emphasizing Spain's sunny climate as a natural alternative, and introduces a foldable drying rack for an eco-friendly drying method. The man then meticulously hangs his clothes, including various metal-themed shirts and sports attire, using clothespins. In a quirky twist, he magically speeds up the pinning process with a joke about invoking magic. The clothes drying concludes with placing the rack on a terrace to dry under the sun, humorously topped with a Spanish flag to make the clothes "more Spanish." Finally, he subtly addresses climate change and solicits likes for a potential dryer purchase. The video shifts to promoting Dreaming Spanish's website, showcasing series on global cuisines, exotic animals, military adventures, and funny tutorials for daily Spanish learning. It mentions the website’s unique feature of tracking learners' progress in Spanish, inviting viewers to explore more through a provided QR code, hinting at a mix of cultural insights and language learning techniques, all while fostering engagement with a humorous narrative.


detailed

In this video, a man engagingly demonstrates how to do laundry in a humorous and educational manner, incorporating elements of his daily life and cultural background in Spain. Initially, the man humorously discusses having only dark clothes as a metalhead, bypassing the common practice of separating whites and darks. He then proceeds to add laundry products to the washing machine, jesting about not using fabric softener to maintain his "real man" persona. After starting the washing machine, he highlights the lack of a dryer in his household, explaining that the sunny Spanish climate negates the need for one. Instead, he introduces viewers to an eco-friendly drying rack, showing how to hang various clothes, including his metalhead T-shirts and jeans. The video takes a comedic turn when he finds an unfamiliar garment among his laundry.

Switching settings, the man performs a mock workout routine and appears in a kitchen, symbolically showcasing the importance of physical activity and possibly dietary advice, using QR codes for interactive engagement. He continues to detail his household chores, like folding clothes and organizing them with color-coded pins on the drying rack, using magic to speed up the tedious task. He humorously decides to "Spanish-ize" his laundry by draping a Spanish flag over the clothes on the rack, placed on the terrace to dry in the sun.

In a concluding scene, the man, now seated comfortably, addresses the audience directly, encouraging likes for the video to possibly afford a dryer, humorously nodding to the impacts of climate change. He transitions to promoting the Dreaming Spanish website, which offers a wide range of videos and series for learning Spanish, catering to various interests from unusual global cuisines to exciting tales from the army and even absurdly comedic situations for troubleshooting tech problems. The website promises a comprehensive and enjoyable learning experience with features to track progress and exposure to diverse accents and cultural insights from Spanish-speaking countries. The video ends with a call to action, inviting viewers to access premium content through a link in the description, against a backdrop filled with QR codes, thematic visuals, and a promise of diverse learning material.


visuals only
In this video, a man humorously guides viewers through the process of doing laundry in Spain, blending everyday chores with light-hearted commentary. Initially, he collects his dirty laundry, confused about how it got so dirty, and moves to the washing machine, which he describes as having many modes, although he only uses the rapid 30-minute program. Contrary to conventional laundry advice, he humorously decides not to separate his clothes due to his all-black metalhead wardrobe.

As he proceeds, he mistakenly picks up a dish soap bottle instead of laundry detergent, making a joke about not needing fabric softener because "a real man" doesn't use it. After starting the washing machine, he utilizes a 30-minute workout as a comedic timer for the laundry, emphasizing a relaxed approach to household chores.

The video then transitions outdoors, where he introduces viewers to an eco-friendly and traditional Spanish drying method — using a clothesline instead of a dryer, attributed to Spain's sunny climate. However, he humorously notes the challenge of climate change and ends with a plea for likes to potentially buy a dryer.

Throughout the video, QR codes appear, suggesting additional interactive content. The man then dives into promoting the Dreaming Spanish learning platform, highlighting various unusual and engaging series designed to teach Spanish, ranging from quirky food series showcasing bizarre dishes from around the world to adventurous tales from the army and humorous scenarios involving calling technicians. Each series is presented with unique visuals, including QR codes for extra content, and concludes with encouragement to explore more via the platform, showcasing its features like tracking progress and the plethora of series available for diverse Spanish learning experiences.

Visually, the video is rich with everyday scenes, from a kitchen and living room to a bathroom full of laundry essentials, and outdoor spaces, tying in educational content with humorous, relatable domestic activities. The man's engagement with the camera, alongside diverse props and settings, adds vibrancy and a personal touch, making Spanish learning entertaining and visually appealing.

---------------------
and this is from "videosummarizer.ai"

En este video, un hombre español muestra cómo se hace la colada en España. Comienza explicando que tiene algo de ropa sucia y decide lavarla en su lavadora. Muestra los diferentes modos y botones de la lavadora, pero elige el programa rápido de 30 minutos. Luego, separa la ropa blanca de la oscura y coloca toda la ropa en la lavadora junto con detergente y suavizante.

Después de iniciar la lavadora, el protagonista muestra cómo tiende la ropa en un tendedero en la terraza, aprovechando el sol en lugar de usar una secadora. Añade una bandera de España para darle un toque más español. Explica que en España no necesitan secadora debido al clima soleado.

En resumen, el video muestra cómo los españoles hacen la colada, desde el lavado hasta el secado al sol en un tendedero.
0 x

User avatar
emk
Black Belt - 1st Dan
Posts: 1708
Joined: Sat Jul 18, 2015 12:07 pm
Location: Vermont, USA
Languages: English (N), French (B2+)
Badly neglected "just for fun" languages: Middle Egyptian, Spanish.
Language Log: viewtopic.php?f=15&t=723
x 6744
Contact:

Re: Describe: YouTube summarizer (based on text/audio and visuals)

Postby emk » Fri Mar 22, 2024 2:05 pm

Yes, I would absolutely expect a product like this to work fairly well (but not perfectly), assuming they used the tools currently available. GPT-4V and Claude 3 Opus are both pretty accurate at describing still images, and Whisper-1 can transcribe audio tracks fairly accurately. GPT-3.5-Turbo or Claude 3 Haiku can translate if needed.

So you could snapshot an image every 10 seconds, transcribe the audio track, optionally translate it, and then ask any current-generation LLM to summarize the results. If you wanted to do a pretty good job for a 5 minute video, you might wind up spending up to US$1-2 on API costs. Or much less, if you accept worse results.

Although, as a language learner, I think I'd actually prefer the raw transcript, the translation and maybe a couple of image descriptions (for vocab). But I'm definitely interested in new approaches. Do you see any particularly interesting ways to use the summary format for language learning?
2 x

Kraut
Black Belt - 2nd Dan
Posts: 2630
Joined: Mon Aug 07, 2017 10:37 pm
Languages: German (N)
French (C)
English (C)
Spanish (A2)
Lithuanian
x 3245

Re: Describe: YouTube summarizer (based on text/audio and visuals)

Postby Kraut » Fri Mar 22, 2024 7:08 pm

emk wrote:.

Although, as a language learner, I think I'd actually prefer the raw transcript, the translation and maybe a couple of image descriptions (for vocab). But I'm definitely interested in new approaches. Do you see any particularly interesting ways to use the summary format for language learning?


Not in the beginner stages, these "metatasks" like summarizing, paraphrasing, mediation etc are for later stages and a preparaton for using languages in new life contexts, .. university f.ex .. think of plagiarism

The example here is a Dreaming Spanish video, which comes without any additional tasks for reinforcing and consolidating comprehension/vocabulary via additional questions on the text, summary ... etc. Pablo et al. don't do it, I'm glad it's so easy now via AI.
0 x


Return to “Practical Questions and Advice”

Who is online

Users browsing this forum: No registered users and 2 guests