- A Software Library for Creating Language Learning Materials (such as bilingual texts)
- substudy: Make Anki cards and other resources from video & bilingual subtitles (command-line)
- Rust subtitle utilities project
What's this all about?
Recently, davidzweig made a really great proposal for creating a library that works with aligned sentences and audio. I would love to have something like this, because I would love to make substudy work with many more kinds of media. But after talking with him for a while, it turned out that we had very different goals: He was mostly interested in the individual sentences, whereas I needed something more ambitious, which focused on entire video files, etc. He suggested that I might need another format for use with substudy, so I decided to go ahead and make one.
This is still very much a work in progress, but I have now created "first drafts" of many of the critical pieces:
- A specification (details here).
- Example metadata.json files for subtitles and audiobooks.
- A Rust implementation of the specification.
- A web-based validation tool that you can use in your browser (no data is uploaded)! This was ridiculously fun to build, because it was my first serious WebAssembly project.
So next, some concrete examples!
Sample 1: A video with subtitles
This file could be used an an input to the substudy-based video player that I'm working on. It specifies a video file, and a set of aligned HTML subtitles.
Code: Select all
{
"baseTrack": {
"type": "media",
"lang": "fr",
"file": "episode1.mp4"
},
"alignments": [
{
"timeSpan": [
10,
15.5
],
"tracks": [
{
"type": "html",
"lang": "fr",
"html": "<i>Jean & Luc:</i> On y va !"
},
{
"type": "html",
"lang": "en",
"html": "<i>Jean & Luc:</i> Let's go!"
}
]
}
]
}
Sample 2: The same video after being processed by substudy
But what if we didn't care about the video as a whole, and just wanted individual sentences with audio and images? Well, we could feed the above file through a tool like substudy, and it might output:
Code: Select all
{
"alignments": [
{
"timeSpan": [
10,
15.5
],
"tracks": [
{
"type": "html",
"lang": "fr",
"html": "<i>Jean & Luc:</i> On y va !"
},
{
"type": "html",
"lang": "en",
"html": "<i>Jean & Luc:</i> Let's go!"
},
{
"type": "image",
"file": "episode1_12_75.jpg"
},
{
"type": "media",
"lang": "fr",
"file": "episode1_9_00_16_50.mp3"
}
]
}
]
}
This is roughly analogous to the format that was proposed in the other thread.
Implementing support
I want to implement support for generating these files in several ways:
- Using substudy to convert a video and either 1 or 2 subtitle files.
- Using hunalign and Aeneas to produce an audiobook with fully synchronized L1 and L2 text, in a format that can be converted to Anki cards, or played directly using the new substudy media player.
Contributions and ideas welcome!
If you're the author of language-learning tools—or even if you just have some scripts lying around that you use to prepare your materials—I'd be very interested in your feedback. In particular, if you'd like to propose a change to the format, you can just submit an issue or a PR on GitHub.
Now I'm almost ready to add a "substudy export aligned" command. But first I want to improve the validator a bit, just to ensure that I actually generate valid data!