It was hard for me to gather a lot of VSQX, so if a much larger number of VSQX files gets collected in the future, retraining can help improve the quality of generated melodoies. The number of VSQX files used for training the current model is below 100 files, which is a relatively low number in a machine learning context. Users can choose to use make their own scripts/use 3rd party scripts if they desire though. However there were problems with this as the Youtube conversion scripts were not accurate (threfore poor quality seed input) along with one of its components from a conversion library being only supported in Python 2.7. The previous version of this kit had scripts that could convert Youtube videos to midi files, which in turn would be feed into the network to serve as a seed for note generation. As a result, the melodies produced by this version is of a higher quality at a much more consistant rate. Rapid, fluctuating melodies from a piano does not flow the same way as a more steady singing voice, so vsqx files were chosen as the main format for note and data representation instead of pure midi files. Also a majority of midi files are based off from a wide variety of instruments, not just singing. This means that midi files that had polyphonic properties (playing multiple melodies on different channels simultanous) were not suitable candidates to be used for training. The main purpose of this kit is to generate melodies that would be sung from a singer's voice (aka monophonic melody). While there are a lot more midi files than VSQX files, there were a lot of variables that affeted the quality of the data processing depending on the file. This changed in this version as the current version uses VSQX files (Vocaloid Editor file format) for dataset conversion, training, and testing. Previous version of the AI Vocaloid Kit used midi files as a main source as music data for training and testing. Python3 train.py -modelOutput music-model.pt Counts notes and syllables to ensure flow between melody and lyrics.Īllow users to train their own melody-generation model based on their VSQX dataset. Use markov models to generate Japanese lyrics and sync them to generated melodies from song. Using a Seq2Seq transformer deep learning architecture model to either randomly generate melodies from scratch or from an inital seed of midi notes Muse vocaloid vsqx manual#There is also a manual option in which you can take your generated midi fileĪnd song and import it into your vocaloid editor yourself.įor those who don't own a vocaloid editor, you can just listen to the generated midi file and read the text files containing the Japanese lyricsĬonverting a directory of VSQX files into a training and validation dataset Muse vocaloid vsqx software#This can then be automatically loaded into a Vocaloid software (tested on Vocaloid Editor 4) to have the Vocaloid sing your AI generated song. The script will generate a vsqx file that contains the melody and lyrics built into it automatically. The corpus was web-scaped from the Studio48 site, containing lyrics from groups like AKB48, Nogizaka46, Keyakizaka46, etc. It can also combine verses if one generated verse is too short So if there are 2 notes in the 1st verse and 5 notes in 2nd verse, the lyric verse combine those two verses and will generate a verse of 7 syllables (2+5=7). This means that if a melody verse contains 7 notes, the script will generate a lyric verse of 7 syllables to match it. It also uses markov models to generate its own accompanying Japanese lyrics that is in syllable sync with the melody. Developers and musicians can now have AI support to aid and inspire ideas for their next vocaloid song.Ĭan take a folder of vsqx files and use Seq2seq model (encoders and decoders) in order to study pattern of a song and generate it’s own unique music (in the form of a vsqx file or midi file). A Python kit that uses deep learning to generate vocaloid music.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |