Hmm, even saving a different working track's metadata onto it is breaking it. Maybe it is something weird with my file itself I guess.
It ended up being a cover image that was too big apparently where as soon as it got saved in, it broke the file. When I added a smaller image to my original file, it worked fine.