Blind student ‘hears in colour’

A blind student has developed software that turns colours into musical notes so that he can read weather maps. Victor Wong, a graduate student from Hong Kong studying at Cornell University in New York State, had to read coloured maps of the upper atmosphere as part of his research. To study “space weather” Mr Wong needed to explore minute fluctuations in order to create mathematical models. A number of solutions were tried, including having a colleague describe the maps and attempting to print them in Braille. Mr Wong eventually hit upon the idea of translating individual colours into music, and enlisted the help of a computer graphics specialist and another student to do the programming work. “The images have three dimensions and I had to find a way of reading them myself,” Mr Wong told the BBC News website. “For the sake of my own study – and for the sake of blind scientists generally – I felt it would be good to develop software that could help us to read colour images.” He tried a prototype version of the software to explore a photograph of a parrot. In order to have an exact reference to the screen, a pen and tablet device is used. The software then assigns one of 88 piano notes to individually coloured pixels – ranging from blue at the lower end of this scale to red at the upper end. Mr Wong says the application is still very much in its infancy and is only useful for reading images that have been created digitally. “If I took a random picture and scanned it and then used my software to recognise it, it wouldn’t work that well.” Mr Wong has been blind from the age of seven and he thinks that having a “colour memory” makes the software more useful than it would be to a scientist who had never had any vision. “As the notes increase in pitch I know the colour’s getting redder and redder, and in my mind’s eye a patch of red appears.” The colour to music software has not yet been made available commercially, and Mr Wong believes that several people would have to work together to make it viable. But he hopes that one day it can be developed to give blind people access to photographs and other images.