Python Hardware Sequencer Part I: PNG to CV Converter

Let's see what we can do with single board computers (SBC) and DiWhy audio. It's of course a very broad topic.  

This time I begin building a modular synthesizer sequencer/control voltage logger based on a Raspberry PI running Python.

I already experimented with Python and SBCs to create simple blinkenlites using PyGame--post is here.  

Recently I created another SBC python program that turns a PNG image into a series of Y-values along an X axis. From here, it'll should easy to turn these Y values into CVs using a I2C D to A converter, with each Y value buffered then going to the sequencer's output, when a trigger or clock is received on a GPIO pin.

 

PNG PONG? I got the "PNG to Y-values" part of the project working this week, doing some of the coding while on vacation; meanwhile, my psychiatrist girlfriend seemed to enjoy the peace and quiet. 

Python is still a bit new to me, I am embarrassed to say that for my day job I am still involved in the good old LAMP stack and C but like all good things it's time to move on. 

And as any Python programmer could will tell you: as an interpreted language (how it works is more complex than I thought--read the post here) Python is not always fast but can do damn near anything.  

And is really easy to learn and forgiving language. 

For the code below I used numpy, a data science tool for manipulating data arrays, as well as Matplotlib, a popular Python module used to create graphs, as well as Pillow ("PIL"), used for graphics manipulation.  

 

Here are the design goals for this part of the sequencer so far:

• Read a PNG file for data input (I am not sure there is another sequencer that does this?). 
• Convert the PNG file to a B&W image
• Turn the resulting data into a numpy structure.
• Flip the array its side (270 degrees).
• Y becomes (ultimately) a CV output.

This gets me into a new programming world. I didn't know Python could extract and manipulate numeric data from an image file, but indeed it can--in many ways--read more here. 

As far as the code below, this is the first thing I've ever written using matlab, PIL and numpy so I figure there are many ways to improve it.....for instance, the rotation/"flip" step isn't necessary; the code could have read each column's values of the numpy array and returned the first Y value "hit", then break, but I want to be able to flip the logged PNG by 180 degrees so might as well flip it 270 before extracting the Y values. 

Nevertheless I will probably end up writing another version of the code below using the [a:b:c,x:y:z] method of extracting columnar numpy data and see which script performs better--the latter might have less lines of code? but for now what I have works. UPDATE: Done, see bottom of post. The version without Numpy array "flips" runs much faster.

OK after the numeric extraction is done:

• Create a python list of the Y-data along w/ the Numpy data.
• (To do: Create a bar graph of the output using the amazing matplotlib, which could perhaps be displayed on the sequencer's OLED.)

For testing I used a 16 x 10 PNG file as source. The code (below) vs. this tiny file took a few seconds to run, so I figure 4K x 2K PNGs probably won't cut it, since it will take too long to run and generate more data then is needed, but a 128 x 50 pixel PNG might. 

The test graphic for my code:

..........that simple PNG file returns this bar graph (as well as a Python list and numpy data structure), which appears to be correct, based on the PNG input:

Here's the Python 3 code so far for PNG image to data conversion; first with the nifty numpy rot() methods to flip the image, then, below that, "straight up" numpy:

##########################

import matplotlib.pylab as plt

import numpy as np

from PIL import Image

###########  file locations ##############

file_loc = "./PNG2cv/tiny.png"

bw_file_loc = "./PNG2cv/tempbw.png"

#use PIL to create BW image, resave BW to HD

image_file = Image.open(file_loc) # open colour image

image_file = image_file.convert('1') # convert image to black and white

image_file.save(bw_file_loc)

#imread method is matplotlib--open graphics file >  output is numpy array

#turn it to Numpy array

im = plt.imread(bw_file_loc)

#get row and column count

output_y = []

output = []

#for this to work at all, you have to rotate the array.

im4 = np.rot90(im,3) #rotate 270 degrees  "3 is 3x90"

x1 = im4.shape

 

 

rows = (x1[0]) #number of rows

columns = (x1[1]) #number of columns

 

c = 0

r = range(columns)

for a in im4:

 

    y = 0

    for x in r:

       

       if (a[x]  < .1):            

           output.append([y])

       y = y + 1

    c = c+1

 

out5 = np.asarray(output)

list1 = out5.tolist()  #turn into python list 

strip_list = []

for tt in list1:

    xp = tt[0]

    strip_list.append(xp)

print(strip_list) #print values extracted as Python list

#bar graph require X values, matching # of items in data set, as array of string values 

#(not int--strings!).

x_axis = []

for q8 in range(rows):

    x_axis.append(str(q8))

#print (x_axis)

#show bar graph of y values

#show this as a bar graph.  This is optional.

plt.bar(x_axis,strip_list)

plt.show()

#########################

UPDATE: I am just getting started with coding this, and have posted the py files and test PNG files here.  get-photo7b.py captures the bulk of the code to date. 

UPDATE.  Here is the code again without the flipped numpy arrays. Rotating an array 90 * x degrees is a cool feature but does it make the program run too slow?  Not sure how this will end up going, but, without flip the code runs about 4x faster so flips might not make it to final build.

Update: 12-10-20 with 3K x 1000 pixel PNG files the code below still runs pretty quickly. Not so much for the code above. We might have a winner for the PNG to array conversion part of this:  

##########################

import matplotlib.pylab as plt

import numpy as np

from PIL import Image

###########  file locations ##############

file_loc = "./PNG2cv/tiny.png"

bw_file_loc = "./PNG2cv/tempbw.png"

 

image_file = Image.open(file_loc) # open colour image

image_file = image_file.convert('1') # convert image to black and white

image_file.save(bw_file_loc)

 

im = plt.imread(bw_file_loc)

x1 = im.shape

 

rows = (x1[0]) #number of rows

columns = (x1[1]) #number of columns

r = range(columns)

for p in r:

    c = 0

    for a in im[:,p]:

        if (a < .1):

            c = rows - c - 1

            output.append(c)

            break

        c = c + 1

#list below is output values we want.

print(output)

#print(out5)

#create numpy array w/ output values

out5 = np.asarray(output)

x_axis = []

for q8 in range(columns):

    x_axis.append(str(q8))

print (x_axis)

#show bar graph of y values

plt.bar(x_axis,out5)

plt.show()

#####################

Down the road, lots more to do:

• Flip the output array to x degrees (to say flip the sequencer's output values, something I always want to be able to do with a traditional sequencer--and easy with python/numpy right?)
• Reduce size of PNG to say 200 x something before processing? Not sure, and not sure if Python can do that, but I figure it probably can--Photoshop can anyway.
• Capture 16x pot analog values to a numpy data structure (and create a PNG image in the process?) and then use that to control the sequencer's output values. Adding or otherwise manipulating data in a numpy structure is easy (good vids for that are here and here) so why not?  The pots could augment/change  the PNG logged data and/or be used for 16 values at input in the mod synth sequencer traditional manner. Still thinking about this.
• Save some of the data set (perhaps all Y values?) into a sqllite DB. That way manners of sequencer data can be easily stored and retrieved....I can see sqllite being really useful for a ton of stuff we do in the audioDIY world.  
• Upload PNGs to the SBC using the web--so make the sequencer an "IOT" device--you can load new data into the sequencer without having to touch its front panel.

I can think of lots of other features....this will be fun.

PNGLEBERRIES? That's it for now.  More to come with this project in upcoming posts.

UPDATE 12-4-20: After doing manditory "windows updates" to Windows 10 v2004, for me, Python/Numpy stopped working.  Check out the info here.  To fix: open terminal and enter this:  pip uninstall numpy    then reinstall older numpy:pip install numpy==1.19.3 

UPDATE 12-10-20 here are some new functions for the project, that manipulate numpy arrays in ways that should be useful to sequencing....

#############################

def reduce_np_numbers(nparray,y):

    #reduces number of items in an np array to y

    # still doesn't always work for larger values of y

    # comes out w/ array 1 element too large, fix this.

    if nparray.size <= y:

        return nparray

    else:

        x = int(nparray.size / y)

        reducednp = nparray[0:-1:x]

        

        return reducednp

def list_to_np(list):

    #create numpy array w/ output values

    out5 = np.asarray(list)

    return out5

def add_to_array_y(nparray, x):

    #add or subtract values from np array.

    p = nparray + x

    return p

def mult_array(nparray, x):

    #multiply each array value by x

    if x == 0:

        x = 1

    p = nparray * x

    return p

def clip(nparray, min=0, max=1023):

    #set min and max values for everything in array

    x = nparray.clip(min, max)

    return x

def scale(nparray,scalevalue):

    #scales all values in array to y average. scalevalue is a percent. Output rounded to int.

    scalevaluex = scalevalue * .01

    print(scalevaluex)

    q = np.average(nparray)

    print(q)

    condlist = [nparray<q, nparray>q]

    up = 1 + scalevaluex

    down = 1 - scalevaluex

    choicelist = [nparray*up, nparray*down]

    a1 = np.select(condlist,choicelist,q)

    a2 = np.rint(a1)

    return a2