Music Recommendation System by Mapping Music and Descriptive Paragraph

Xingbang Liu

4/14/2020


Music flickr photo by Jesse Kruger shared under a Creative Commons (BY-NC) license

What Music Can Do

  • Evoke emotions
  • Influence moods

Current Product

  • YouTube Music, Spotify
  • Recommendor System
    • Based on browsing history
    • Rank tracks

Limitation

  • Can't detact user emotions
  • Only digs deeper

My Idea

  • Fill up the hole
  • Music recommendor/Browser
  • User descriptive paragraph
  • Match songs (lyrics)

Mapping Workflow

Mapping workflow

Thesis Overview

  • Proposed to implement a recommendor system
  • To fill up the hole
  • Match users' text and lyrics summarization with string similarity
  • Implemented with Flask
  • User studies and Pytest
  • The outcome is insignificant

Tools Utilized

  • Web framework: Flask
  • Database: SQLAlchomy
  • Lyrics: LyricsGenius
  • Text model: Spacy-langdetect, Pytextrank, Py_stringmatching

System Design

System design

My Database

  • SQL: (user, text, lyrics) tables with relation
  • Lyrics: 593 Pop music with lyrics, author, and lyrics summarization
  • Text: Texts with the users they beloge to, text summarization, and result
  • User: username and password

Database Design

Database design

Input Types

  • Texts
  • Option 1: describe emotions or experiences on the day (e.g. diary)
  • Option 2: describe a scene (e.g. TV scripts or TV scene summarization)

Result Format

  • Store with Json: list of IDs
  • Survey links
Sample result

Sitemap

Sitemap

Survey Design

  • Google forms
  • Approved by Institutional Review Board
  • Question 1: describe the text, get precise emotion
  • Question 2: describe result, get user feedback
  • Question 3: explain their feedback

User Study Outcome

  • Only 3 participation, less than total user
  • Insignificant
    • Outcome

Program Test

  • Pytest
  • Lyrics model and text model

Program Test Outcome

  • 16 passed, 2 warnings

==============================================================
test session starts
==============================================================
platform linux -- Python 3.7.6, pytest-5.4.1, py-1.8.1,
pluggy-0.13.1
plugins: cov-2.8.1
collected 16 items

test/test_lyrics_proc.py .Searching for "Country House" by
Blur...
Done.
.Searching for "On Top Of The World" by Imagine Dragons...
Done.
.Searching for "Scared To Live" by The Weeknd...
Done.
.......
test/test_text_proc.py ......

==============================================================
warnings summary
==============================================================
FSADeprecationWarning: SQLALCHEMY_TRACK_MODIFICATIONS adds
significant overhead and will be disabled by default in the
future.  Set it to True or False to suppress this warning.
    'SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead
    and '

DeprecationWarning: Using or importing the ABCs from
'collections' instead of from 'collections.abc' is deprecated
since Python 3.3, and in 3.9 it will stop working
    if (isinstance(value, str) or isinstance(value,
    collections.Callable) or hasattr(value, 'match')

-- Docs: https://docs.pytest.org/en/latest/warnings.html
==============================================================
16 passed, 2 warnings in 11.27s
==============================================================

Threads to Validity

  • Number of participants
  • Lack of a web player
  • Experiment design: variables (text emotion, song emotion, and the perception of participants’ association between text and song)

Future Work

  • Web Player
  • Flask Project Structure and Front-End
  • Lyrics Model Automation
  • Monge-Elkan optimization
  • Experiment optimization

The End