Reading a book can be a monotonous exercise at times. To beat the monotony, some people take to play songs in the background. But the discordant tempo of the music may distract the listener making him/her lose focus.
But what if there was suitable ambient music to go with the mood of the para that one isreading? Researchers at International Institute of Information Technology Hyderabad (IIIT-H) have worked on such a thought and developed an algorithm that syncs well with what one is reading in e-book format.
In their demo, the team comprising Makarand Tapaswi, Vinoo Alluri (both faculties) and BTech student Jaidev Shriram took the soundtrack from the Harry Potter movie and aptlyjuxtaposed it with the novel.
Recognition
Their research, pending patent, has won the Brave New Idea Award at the 23rd International Society for Music Information Retrieval (ISMIR 2022) held in Bengaluru early this month.
Vinoo Alluri
Jaidev Shriram
Makarand Tapaswi
In its initial stage, the research is focussed on books that have been adapted for movies. “Harry Potter series was an obvious choice for us as its movie adaptations are big hits,” Shriram said.
The researchers cut the book into parts, with each part reflecting a mood. The soundtrack from the movie too was pruned scene-wise. “We categorised the music based on how homogenous it sounded in terms of emotion,” said Shriram.
While only a part of the book matched with the movie, the researchers filled in gaps with an emotion-based retrieval system. “If a para evokes feelings of anger, gloom or fear, then the algorithm would automatically choose a bit that reflects the mood,” Alluri said.
Elements like these allows the application to play relevant music seamlessly as you scroll down or up the book on your digital device.
She, however, admits that there is a limitation in terms of the length of the music that they could use from the original sound track, factoring in the intellectual property rights.
She, however, says that the use cases can be many. “For one, we can talk to the e-book players who can consider adding another dimension to their subscribers,” she said.