Friday, December 15, 2017

Teensytune: A Teensy-based MIDI controller/keyboard


Teensytune is a homebuilt MIDI controller/keyboard built around an old broken keyboard using the Teensy microcontroller. It features 49 keys, a programmable 16 beat drum machine, pitch bend/modulation control and two recordable loop channels with controllable tempo. It outputs MIDI signals along a USB, so you can plug it into a laptop or any other MIDI synth to generate the actual sounds. It is constructed from a wooden frame with custom sideboards and control panels made using Carvey, a programmable 3D carving machine.

This post is a continuation of previous post on rebuilding an old broken electric piano. I last touched this project about 12 months ago, where I had a Teensy reading the keyboard state and passing MIDI messages to a Raspberry Pi which was running a synthesiser and outputting sound to an amplified speaker. I ended up having a lot of troubles getting the sound output from the RPi working reliably: I could never really find an acceptable balance between getting nice stutter-free sounds with low latency, even after trying custom firmware and playing with countless settings. I got frustrated and moved on to other projects.

12 months later I decided that it's time I moved on with this: I've dropped the RPi synth for now and have just focussed on getting a workable MIDI instrument up and running, leaving space inside the case for expanding the project to include a synth, amplifier and speaker at a later stage.


Implementing the Controls:

After I got the basic circuit and code setup on the Teensy for reading the keyboard state, I started to focus on developing some cool controls. I started by adding a simple drum accompaniment function using a single on/off switch and a potentiometer for controlling the tempo. The beat is run through an interrupt using a PIT timer on the Teensy that sends a MIDI note for the current beat in a static 16 beat pattern, that is looped on repeat. Changes in the tempo pot are used to reset the interrupt interval time. I added an old two axis joystick I had sitting around and read the values using two analog inputs on the Teensy and translated these into pitch bend and modulation MIDI messages.

In order to provide a bit of feedback to the player, I decided to try and add in some Neopixels for coloured, flashy fun and a two-line character display. I plugged in a Neopixel to test everything was working fine: all good. I got this two-line character display which interfaced to the Teensy via I2C to display out data, for example, on the instrument selection and provide feedback for the programable drum machine. The display is a 5V device, so I used a logic level convertor to convert to/from the 3.3V signals on the Teensy. I2C on the Teensy 3.1 requires that both SDA and SCL data lines be connected to 4.7KOhm pull-up resistors, which I did. I used this library to control the display: tested I could display some basic text and instrument number, and all is working well. I ended up using the character display to provide visual feedback for programming the 16 beat drum pattern: one line of the display shows the instrument number assigned to each beat, and the position of the current working beat flashing on the screen. I added two extra buttons for scrolling the working beat left or right, and setup the beat to be programmed to a new drum instrument by pressing one of the bottom 10 keys on the keyboard.

In order to add a few more buttons (running short on GPIO pins by this stage) I got a I2C port expander which provides an additional 8 digital inputs and communicates on I2C. I tested this on the same bus as the I2C character display and everything seemed to be working fine with a single button, but wouldn't work with the other seven. After a bit of debugging I realised that the default I2C for the expander and the display were the same: I changed the address for the expander using the external address pins and everything was working fine.


Finally, I implemented two recordable loop channels. The idea with this was that I could play an input sequence to the keyboard over the 16 beat period of the drum machine, and the loop channel would record and playback this sequence on repeat, while the drum machine was switched on. I could also playback these sequences at a variable tempo using the drum machine's tempo knob, providing the ability to record complex patterns at slow speeds, then ramp this up to a fast speed at playback which would otherwise be impossible to play manually. Each channel is controlled by a single button: when the button is initially pressed, the recording begins and starts to playback the input sequence in a loop. Subsequent button presses turn the playback for this loop on and off, and holding the button down for 1 second deletes the sequence, making the channel open for re-recording a new sequence. To implement the record and playback, I created two arrays in memory that contain the note pressed and the sampling time over the time period of the 16 beats. During recording, these arrays are written to using the main program loop with timing provided using Teensy's "elapsedMillis" type. Playback is achieved by using two PIT timers with interrupts to ensure playback timing is smooth regardless of what is happening in the main program.

I linked one of the two channels to have additional pitch control using the bottom octave of the keyboard during playback, gaining inspiration from this project. During recording, the first note of the pattern becomes the "root" note of the sequence. During playback, keys pressed in the bottom octave of the keyboard are used to re-assign the root note of the sequence, which acts to shift the pitch of the entire sequence up or down by a fixed amount. This provides that ability to make this sequence a "baseline" and have the player manually control a chord progression in a song with a single key press.

Designing the Case:

I bought some 19 mm thick dressed pine for building a new housing for the keyboard, because the old plastic one looked ugly, and I wasn't looking forward to 3D printing new panels to fit the controls I wanted. I put together the base and back board by hand: glued two bits of timber together and screwed them in for good measure. I then designed side panels and top panels to be cut out and decorated using Carvey. I designed the panel to hold all of the controls to be carved also using Carvey. I had to use two different milling bits to get the right combination of cutouts and fine detail in the lettering on the panels, and I had to carve first on the front and then on the back to complete the design/housing for the display, joystick and electronics PCBs.

Putting it all together:

I wired up the Teensy and connections to the keyboard on a perma-protoboard, tested this was working well leaving the other components on a breadboard. I connected up two neopixels to use as part of the visual feedback to the player: I mounted these to be facing up through the top control panel adjacent to each of the loop channel control buttons. These light turn yellow for standard operation, pink when the player is selecting a new instrument, blue to indicate a loop channel has a recorded sequence available, green to indicate this channel is currently playing and red to indicate the channel is currently being recorded. I then wired up and soldered on the remaining components to the back of the control panel, connected everything up to the keyboard and screwed it into the wooden frame.



This is a video of Teensytune in action playing an interpretation of "Rainbow Road" from Mario Kart 64 (I love that game). Starts off by programming the drum machine, then recording two loops: a baseline and a little flourish. The actual song starts about 1:27. When playing with the right hand, the left hand is controlling the chord progression by shifting the baseline. Sorry about my poor piano skills :).

I'm hoping to expand the project by using the available space to install a small embedded computer to perform the synthesis, and add in an amplifier and speaker, so that the Teensytune can operate independently of a laptop. Stay tuned!

Teensytune Code:

The code that runs on the Teensy can be found at:
https://github.com/mit-mit-randomprojectlab/teensytune

There are a few required libraries for the neopixels, two-line display and port expander, listed in the readme.


Monday, November 6, 2017

PyWeek 24: HackerBot


"HackerBot" was my solo entry for PyWeek 24 (October 2017), a twice yearly video game development competition that gets competitors to build a complete game from scratch in seven days using the python programming language. This competition's theme was "They're Behind Everything" and I made a 3D stealth game in which you control a little hacking robot trying to break into a space-based installation and steal data. You can "hack" into (and take control of) more powerful NPCs in order to help complete objectives (hence you're behind everything).

 


The game character is a little bot that can crawl around and jump from asteroid to asteroid. You need to avoid being detected by much more powerful "SentryBots" and turrets that essentially one-shot you. Your only "weapon" is your ability to "hack" into things when they get close enough: you can hack into shield generators to shut them down and access new areas, you can sneak behind guarding turrets and put them out of operation and you can even hack into passing SentryBots, take control, and wreak some havoc with their on-board weapons. The game features six levels and a giant boss battle to boot!

 


I was super excited to win both the individual entry and overall competition winner with a score of 4.29 out of 5.0. This score puts the game into the top ten PyWeek entries of all time (tied in at number 7, out of more than 1000 games over more than 10 years!) so I was pretty pleased with the result :).


Playthroughs:
Here you can find a full playthrough of the game, broken up into two parts:


You can currently download the game as either a source release (works with Windows/OSX/Linux, but requires additional installations) or as a standalone Windows application (I'm still in the process of getting the standalone OSX version working):

Windows (tested on Windows 7/10):
hackerbot_pyweek24_windows.zip (41Mb)

Mac OSX: stay tuned!

Source Distribution (compatible with Windows/Mac OSX/Linux, requires python 2.7, pygame, numpy and PyOpenGL, see installation notes in README):
hackerbot_pyweek24_source_v11.zip (8.3Mb)

If you are having trouble getting the game to run, or have a bug/crash to report, please email: randomprojectlab@gmail.com


Saturday, March 18, 2017

PyWeek 23: My Enemy's Enemy is my Friend


"My Enemy's Enemy is my Friend" was my solo entry for PyWeek 23 (February 2017), a twice yearly video game development competition that gets competitors to build a complete game from scratch in seven days using the python programming language. This competition's theme was "The Lesser of Two Evils" and I made a 3D tactical space shooter in which you work for an insurgency against an oppressive colonial government after your family is killed in a government attack; your character reflects on your retaliation against government civilians and is forced to question what action should be taken when faced with two evils. This comp I went solo for the first time since my first entry back in 2014 (my usual wingman for Team Chimera, Lucid, was taking a break).


The game was inspired by 1990's space shooters such as Colony Wars and X-Wing vs. Tie Fighter. Instead of using 3D hardware acceleration via graphics libraries such as opengl (or the pyopengl python bindings), I decided to experiment with the idea of doing 3D all on the CPU, in python. I was keen on this idea because installing 3D library dependancies in python can be a bit of a pain, with slightly unpredictable results ... also, I just got stuck on the idea early in the process, and I'm stubborn :). I managed to implement a basic engine for rendering 3D flat shaded graphics by performing camera projection and lighting calculations for triangular faced models using numpy and rendering faces using "pygame.draw.polygon". I added to this a basic horizon texture mapping (using numpy and "pygame.draw.surfarray" and rendering of circular particles using "pygame.draw.circle" and circle radius computed based on camera depth using numpy. The array of resulting 3D effects was pretty simple, but I think it worked pretty well: I had something that looked at about the same level (maybe a touch better?) as Starfox did on the SNES.


I ended up ranking second in the individual entries and third overall, a result I was very happy with. I received a fairly high "production" score (4.8 out of 5.0), so I was pretty miffed that players thought the graphics were decent enough.

Playthroughs:

Here you can find a full playthrough of the game, broken up into three parts (with a fourth showing an alternative ending to the game):

I'm currently working on getting a binary release/installer for OSX/Windows, but for now you can play them game by installing the source distribution (requires python, pygame and numpy) by downloading the game from the pyweek page: My Enemy's Enemy is my Friend.



Thursday, February 2, 2017

Global Game Jam January 2017


Global Game Jam is an annual event held at approximately 700 sites simultaneously across the world in which participants join teams to create a video game in 48 hours. I participated in the event for the first time this year in January at the North Sydney Institute of TAFE's site and had a blast! This year's theme was "Waves" and joining in an enthusiastic team with three other people I met at the event, we created our game "Jet Ski Jousting".

I was a little apprehensive when I turned up to the event on a Friday afternoon, not knowing anyone, and not being sure if I'd even be able to join a team. Luckily I ran into a pair who had done the event for at least the last three years and needed some audio. Overall there were about 60-70 people there (I guess) and everyone was super friendly and inclusive. There were a range of different games types and platforms, people doing standard PC games, tablet games, VR etc.

I did the sound and music for the game (and a little bit of UI art) which was a great experience, as I've mostly been involved in programming or game design in previous projects. I used Musescore to compose the background theme and a little victory jingle, and used Audacity to chop and post-process a range of open-licenced sounds I found online, mostly through Freesound.org for sound effects. I combined these all together and integrated them into our game (which was built in Unity) using FMod, which I had never used before; it was relatively easy to pick-up with a few pointers from one of our other team members who knew what he was doing.


Jet Ski Jousting runs on Android and PC/OSX (source distribution requires Unity) and can be downloaded here: Jet Ski Jousting (Global Game Jam 2017)

I had a bit of spare time on one of the mornings so I also worked on a little side-project game. I had brought along my Wii Balance Board that I picked up from an Op Shop a few weeks ago, and a Raspberry Pi 3. I previously got the Wii Balance Board to talk to the RPi over Bluetooth using python, so I worked off this to build a game in python/pygame called "waverider". It's a sort of motion-racing game where you have to accelerate a particle along a wave function by leaning left and right to steer the particle up or down the hills in the wave function, racing another particle to get to the end of the course.


Good thing about using the RPi was that I could plug it into a ceiling mounted TV hanging above a busy thoroughfare and leave the balance board on the ground nearby, so passerbys could casually play. People seemed to enjoy the game; a bit of physical movement was probably a nice respite from intense work in front of a computer screen.


The code for the game is available here: Waverider (Global Game Jam 2017)

Tuesday, January 31, 2017

Wii Balance Board


A few weeks ago I walked into Vinnies to drop off some old clothes and knick-knacks we no longer needed. On display I found an old Wii console, complete with wiimote, nunchuk, a couple of games and a Wii Balance Board, all for $20 ... Bargain! I couldn't help myself so I went ahead and bought it, hence bringing home more junk to fill the space left by the clothes we gave away :).

The balance board is pretty cool: it has four weight sensors on the four corners of the board that allow for a measurement of the total weight of the person standing on the board and also data on their center of gravity in two dimensions. I found this cool app on osx for connecting to the balance board and providing a simple bathroom scales. Worked well on my Macbook so I decided to dig around for libraries that could connect to the board and provide the same data. I found two different solutions here and here, both of which were in python (which works for me), but unfortunately seemed to be Linux only. I decided that perhaps I would make this a Raspberry Pi project. I had trouble getting the first solution to work, as it was a patch to an existing svn repository that didn't seem to be there anymore (at least it seems like the project has switched to a different versioning system, and I wasn't so sure how to go ahead with the patch). I found I could get the second method (Gr8W8Upd8M8) working well.

I modified the code to create a daemon thread that would continuously read data from the balance board and provide it to a queue such that the sensor could be monitored continuously by a background process and interface with an application in the foreground.

The python code that provides the interface to the balance board can be found here: wii_balance_board.py

I ended up using the code in a Global Game Jam game I made using python and pygame (waverider). I'll talk more about that in my next post.