An AI That Plays WiiPlay Table Tennis

Introduction

For this project, my intention was to start posting short form content on websites such as TikTok and YouTube. I wanted to do this because I felt like I wanted to publish some sort of content in between large scale projects, such as the AI that orders you a pizza if it detects that you're sad. Doing these short videos not only allowed me to create content more consistently, but also it allowed me the opportunity to practice my coding skills and work on projects that could be done in a relatively short amount of time!

If you would like to see the video associated with this project, feel free to check it out on my TikTok, @MitchsMisadventures

For this project, our objective is pretty straightforward. We want to be able to write some code so that we can rally a table tennis ball 100 times in WiiPlay. After 100 successful rallies, the game ends and we are victorious.

Getting the Positional Data

Position

Getting the positional data can be split up into two distinct parts, both equally important. Not only did I need to try to figure out the position of the ball, but, I also had to figure out the direction that the ball was moving towards, that way I could rally the ball back. Doing this isn't too difficult. What we can do is create an HSV mask that corresponds to the orange color associated with the ball.

This wasn't too difficult to do, the only slight issue came with the fact that the wooden floor has a pretty similar color to the ball. It's at least in the same HSV range. I had two ways to go about this issue: I could either narrow the HSV range and make it more strict in hopes that I only get the ball, or, I could decrease the scope that my bot sees. I ended up doing a little bit of both. I tightened the range up a little bit in addition to changing the region that my bot can see. Instead of seeing the entire screen, it now only looked at the table itself.

With the mask and region in place, all I had to do was write some code to detect white clusters on the mask. Doing so would give me the coordinates of the ball!

Direction

For this part, what I ended up doing was continuously record the position of the ball at two distinct times. I would compare the position of the ball in the present time (Position #1) with the position of the ball from a few frames in the past (Position #2). From here, all I had to do was get the vector that goes through Position #1 starting with Position #2. This will give me a rough estimate of the direction of the ball.

I say "rough" because this game is in three dimensions. This extra dimension is a bit tricky to deal with since we are recording two dimensions worth of data. To get around this issue... I just hoped for the best. The game is lenient enough where as long as the paddle is in the general vicinity of the ball, it's going to register as a hit. Regardless, with the new vector calculated, I was able to extrapolate where on the board I should place my paddle. I only moved my paddle if the vector was pointing towards us since that would mean that the ball was going in our direction. If the vector was pointing away from us, I simply reset the paddle back to the middle of the table and waited for the rally to return.

Closing Remarks

To be honest, that's pretty much all there is to this project! It's pretty short and sweet but I had fun with it. I also feel like this is a great practice project for people that want to get familiar with ComputerVision and Python! As mentioned before, I have a few other projects and videos on my TikTok so feel free to check it out if you haven't already!