Hawthorne Heights' second album, If Only You Were Lonely, arrived in 2006 along with a DVD titled This Is Who We Are. Leading up to the album's release, Victory urged the band's street team members to help it chart above R&B singer Ne-Yo, whose own album was slated to be released the same day. Questionable promotional tactics included rearranging store displays to hide Ne-Yo's record and make Hawthorne Heights' more prominent. Lonely debuted at number three on the charts, though more than a few eyebrows were raised in the band and label's direction. Regardless, the album continued to sell well, eventually going gold.

However, controversy came to a head in early August when Hawthorne Heights suddenly announced that they were leaving Victory Records (a move that violated a contractual obligation calling for two more albums) and suing the company over various issues, including unpaid royalties and tarnishing their name over the aforementioned incident (the band claimed to have no knowledge of the infamous street team letter). In a statement issued online, the bandmembers compared their time at the label to being in an "abusive relationship" and directly attacked "greed driven" label head Tony Brummel. As issues were sorted out behind the scenes (and Victory countersued the band, claiming they just wanted to jump to a major's roster), Hawthorne Heights continued touring nationwide and served as a headlining act on the fall 2006 version of the popular Nintendo Fusion Tour.

Following several court dates with their label, Hawthorne Heights returned to the road in late 2007. However, guitarist Casey Calvert died in his sleep just several hours into the tour, the victim of a lethal combination of antidepressant medications and Vicodin (Calvert had reportedly undergone a root canal prior to the band's departure, hence his need for painkillers). Soldiering on as a quartet, the bandmates resolved their issues with Victory Records and issued a third album, Fragile Future, in August 2008. Rather than replace Calvert, the band decided to have guitarist Micah Carli fill in on vocals after Calvert's death. In 2010, Hawthorne Heights finally did switch labels by having Wind-Up Records issue Skeletons, their fourth album. Meanwhile, Victory wrapped up its contract with the band by releasing Midwesterners: The Hits, which featured material from their first three albums. The band split from Wind-Up in 2010 and formed their own label, Cardboard Empire. Along with touring, they began work on a series of EPs beginning with Hate in the summer of 2011, then Hope in late 2012.