Project Story
Daniel have been involved with music and art since I was 5 years old, plays 4 different instruments, improvise, compose, has his own band in uni and a music channel with millions of views.
"Patterns exists in sketches just like they do in music; with correct processing, is it possible to turn sketches directly into songs?" This has been a question that clinged to me for years and I finally found the time to test it.
And as it turned out, the answer is yes.
Mechanism
Marker sketches are made on card machine receipt styled paper strips and fed into the MozArt machine through the set of stepper driven printer style rubber rollers. The sketch is guided by the transparent smooth acrylic sleeve and captured by the camera module. The camera takes quick snaps of the strip and patches them into a complete picture. Both the stepper and camera are linked to a raspberry pi 3 which regulates the stepper and shutter speeds.
Algorithm
The algorithm for how the glockenspiel plays was designed to be broken down into four main steps.
• Before the main algorithm runs, a calibration function is run which calculates the threshold value for the main algorithm. This value is the lowest pixel value of a picture taken of blank paper. This means that all values below this are darker than the paper hence there must be a marker line there. This stage is done on a piece of paper at the start of the strip which is left blank.
• At each time step, a picture is taken before being cropped down to the central column. The threshold function is applied to each pixel so if its value is below the threshold value, its value is set to 1 to indicate there is a line there otherwise it is changed to 0.
• It was decided that the servo motors actuating the glockenspiel pieces would be controlled by a binary string. To do this the black and white image is scaled down to an eight by one vector of Boolean values indicating which of the eight notes should be played at each time step.
• The vector is then ‘neatened’ by removing notes which are next to each other to ensure the sound is harmonious so most drawings fed into the machine produce a pleasant sounding melody. This process provides something which, considering all of the notes are in the D Major scale, does not sound necessarily like a ground breaking piece of music, but is very interesting and rather pleasurable to listen to.
Future of MozArt
Due to the overwhelming interest and support MozArt received on its public display, I have decided to take the project further by "minigameifying" it into a wechat mini-program with my friend and professional music composer Vomyrak.
The program will feature an improved musicfication algorithm and a direct wechat sharing function, scheduled to launch in March 2019.
(Minimum Viable Technical Hack)
(Final Minigame Framework, Monetization Aimed for Competitive Social Media Sharing.
Still Curious?
Additional information on this project might be private, but please feel free to contact me to learn more!