Skip to content

Creating a cross platform GUI for OpenAI's Whisper Part 3

GitHub Repository

Why use it?

The initial motivation for this repo is to have a GUI to manipulate Whisper. This is so that I can generate initial video to text files to put up transcripts on the kivyschool.com website. One good thing about this initial integration is that I can then switch models, make the GUI better, or even run this when I have no access to a terminal/cmd/powershell. Having complete control is really nice and being able to use Kivy to rapidly prototype a GUI is invaluable (this was built in 2 days discounting all the time I spent documenting it, total time was 6 days). Later on I still need to integrate multiprocessing to push blocking code onto a Python subprocess.

What can you learn from this?

In associated github repo and associated youtube playlist, you will learn:

  • How to integrate Kivy and OpenAI's Whisper automatic speech recognition system.
  • How to deal with various Poetry bugs very easily (assuming you got the setup in the PREREQUISITE SETUP video of Pyenv + python-poetry)
  • How to integrate machine learning/tensorflow Python packages with Kivy and package them with PyInstaller
  • How to create and visualize Kivy app, then easily make it.
  • How to use and manipulate Kivy's FileChooser widget on desktop.
  • How to get and set data on various Kivy widgets.
  • How to manipulate Kivy popups
  • How to manipulate filepaths with Pathlib
  • How to package a complex Kivy app with PyInstaller
  • See firsthand how to create an app from concept to completion.
  • See the app at various stages by running the various midpointX.py files

AI Transcript provided by KivyWhisper

Hello and welcome back to Kivy School. Today we're going to be going over part 3 which is connecting Whisper AI and Kivy. This is the preparation part. This is for Kivy Whisper which is a GUI for OpenAI's Whisper. All that Whisper does is it converts speech to text. So let's continue. So again we're connecting Whisper AI and Kivy.

This is the recap. We know how to change the file chooser's drive. Instead of a drop down, we've chosen a spinner so we can change the file chooser's drive. Now we have the old to do. The old to do is we still need to finish our Kivy GUI. We know how to use the file chooser and we don't need to make the text file markdown compliant.

Where we are at right now is selecting a file. We're right here. Making the file chooser actually do something. How do you select a file from file chooser? That is filechooser.selection. That's how you get your file.

So there's two obvious fail states. Number one, what if you want to transcribe and you haven't selected a file? That's a fail state. That's an obvious fail state. Number two, what if your selection is not an audio file? For this case right here, the plan is to let Whisper run. We're going to catch the error and then we'll blindly tell the user to try again with an audio file. Plus we'll show an error pop up.

That's our plan for the two obvious fail states. There might be more but I'm not covering them in this section. So how do you select a file from file chooser? We're going to look at midpoint3a.py. So right now, like this, python midpoint3a.py. I want to respond to a selection of a file. To do this under file chooser, use on selection. So when you select, that event will run. I've also changed this button right here to a label. And then I set an ID on this button. Then on selection, I will change the label's text. If you see right here, I click. I've changed the label already. I changed the label and it responds to a selection of files from the file chooser.

Okay, let's keep going. So how do you select a file from file chooser? So the first bug is sometimes the selection does not exist. Like when you're going up a folder. So we go down right here. If you go up, you see my selection has disappeared. So that's a bug and it will crash. What you need to do is you need to figure out if self.selection is empty or not. So the bug is, this is because self.selection of zero, getting the first element, will give an index error since selection is an empty list.

So if you say select a file and then empty list, but you're trying to get an index of zero from an empty list, you will crash. So we need to do a simple if statement. All we do is check for the length that self.selection is zero. So you go right here. I've selected it. If I change files, I have no more selection and it says, okay, self.selection is an empty list, so it's not greater than zero. I reset the label of selected file. So here it is. It's the one liner.

This is root ID selected text ID dot text, which is the ID of this button. I'm setting the text to selected file plus the selection. If the selection is greater than zero or else you just set to the default, which is selected file colon and then a space. So this line broken down, right? Let's go look at it. This line broken down. Root.id selected text ID dot text, right? Selected file plus self.selection.

If the length of self.selection is greater than zero, else selected file is empty. So it will first check if self.selection has a selection, aka the length is greater than zero. First check and then set the text to selected file plus self.selection of zero, the first element. Why? Because we already know that the length of this list is greater than zero, so this will not error out.

If the length of self.selection is less than zero, then that means it's an empty list and then we can just default to selected file. So to recap, we have responded to a file chooser selection with on selection. We have changed the button to reflect the selection. The selection is a list, so I naively choose the first element. We also changed the button to a label.

We set in label kv, multiline is equal to true. I'll show this right after I go over the slide. We set in the label kv, vertical align is center. I send the label kv, do wrap is equal to true. And then for a text wrap to work, you just have to add text size is equal to self.size. Let's very quickly do the bottom one first and then I'll show the top one. Another kv question I see often about using the label widget, how to have text wrap automatically.

How to have label automatically grow to accommodate this text. The answer is just set the text size to self.size. If you want to read more, you can read this. This is the wrapping text in kv's label on inclem.net. Then I will show you the other changes too on midpoint3a.py. So, respond to file chooser selection with on selection. Click, it's responding, correct. I changed the button to reflect the selection, correct. Selection is a list, I did this, I did this.

And then in label kv, multiline is equal to true. So you can see right here, change it from a button to label. I've given it the ID and have multiline to wrap vertical alignment. So let's go find something that has a long name. Try this. OK, so you see right here, multiline is true. The vertical alignment is the center.

If it was not the center, it could be at the bottom right here. But because I set it to the center, it looks nice. And then do wrap true. Yeah, because it goes like this and like this. And then I set text size to self.size. The app now looks like this. It's saved as midpoint3a.py. As you can see, there's multiline right here. And it responds to selection.

So recap from the very beginning. We have made a new virtual env with poetry. We've installed whisper with poetry. We've applied whisper to a text video. We've also made a Kivy GUI. Now we select a file and we respond to a file selection. We've used the file chooser. Now the next step is here, which is output a text file in the same current working directory. It's right here.

Basically, now we need to from Kivy use whisper and then execute whisper. How can we change your file chooser's drive? The answer is this is just something you either need to know or you can Google it and then look in the docs. Change file chooser.path. We've made a dropdown to change file chooser drive. It's not really a dropdown. It's now a spinner. In addition to the old to do, we've made the Kivy GUI. We've selected a file. We've used the file chooser. And then we're right here. We're going to actually now connect Kivy and whisper. Thank you for watching. The next part will be part four, which is connect whisper AI and Kivy for real.

Article Error Reporting

Message @BadMetrics on the Kivy Discord.