Introduction

In this paper, we propose a novel functional representation designed as an alternative to REMI, a popular event representation that uses note pitch values and chord names to encode symbolic music.testttt!!

Fig.1 Illustration of (a) REMI and (b) the proposed functional representation, differing in note pitch and chord name events.

This new method takes musical keys into account, recognizing their significant role in shaping music’s emotional character through major-minor tonality.

Fig.2 Key histogram of high/low valence clips from EMOPIA.

Specifically, our method represents both melody notes and chords with Roman numerals relative to musical keys, a functional format considering the relationships between notes, chords and scales (major or minor).

Fig.3 Illustration of the conversion between letters and Roman numerals in the cases of C major / c minor. The solid line represents a direct one-to-one conversion, while the dotted line stands for a random conversion to either one of them.

It also allows for melodic variation with respect to keys and addresses the problem of data scarcity for better emotion modeling.

A Transformer is employed to harmonize key-adaptable melodies, allowing for keys determined in rule-based or model-based manner.

Harmonization Samples

We present a selection of samples within the same context as our user study, i.e., original pieces as well as their four variants generated with the same melody but the opposite emotion condition. Four variants include:

• REMI (trans): transpose to C major / c minor and represent in REMI; simply set the emotion event to the target emotion without changing keys
• REMI (rule): transpose to C major / c minor and represent in REMI; determine keys in rule-based method
• Ours (rule): represent in functional representation; determine keys in rule-based method
• Ours (model): represent in functional representation; determine keys in model-based method

Illustration

Fig.4 Illustration of a re-harmonization example using the method "Ours (rule)". Top: original melody and chord progression from the validation set conveying positive emotion. Bottom: melodic variation with re-harmonized chord progression to convey negative emotion. The downward arrows in the bottom figure indicate the notes shifting down by semitones due to a key change from E major to e minor. This illustration is not an exact match with the following audio for improved clarity in layout.

Original (Positive)	Generated (Negative)

Negative Variants for Positive Original Pieces

	Original	REMI (trans)	REMI (rule)	Ours (rule)	Ours (model)
Sample#1
Sample#2
Sample#3
Sample#4
Sample#5

Positive Variants for Negative Original Pieces

	Original	REMI (trans)	REMI (rule)	Ours (rule)	Ours (model)
Sample#1
Sample#2
Sample#3
Sample#4
Sample#5

Authors and Affiliations

• Jingyue Huang
  Research Assistant @ National Taiwan University / PhD student @ UC San Diego
  jih150@ucsd.edu

• Yi-Hsuan Yang
  Professor @ National Taiwan University / Joint-Appointed Researcher @ Academia Sinica
  yhyangtw@ntu.edu.tw, affige@gmail.com
  website