The Magic will also acquire a 2017 first-round draft pick as part of the deal which delighted Raptors star Kyle Lowry.
"I think he's a great addition to any team, especially ours where we add some shot-blocking, some outside shooting, some inside toughness, some athleticism running up and down the floor," Lowry said of Ibaka.
Ibaka only joined Orlando last offseason after seven years with the Oklahoma City Thunder.
The 27-year-old averaged 15.1 points, 6.8 rebounds and 1.6 blocks in 56 games with Orlando.
With the Thunder, Ibaka twice led the league in blocked shots while averaging 11.6 points, 7.4 boards and 2.5 swats in 524 games.
He was a key part of the team which reached the NBA Finals in 2012, where they were beaten by the Miami Heat.
Ibaka's status in Orlando had been the subject of intense speculation, with the Magic anxious about their ability to hang onto the player this summer, when he will become an unrestricted free agent.
The Magic meanwhile welcomed the arrival of Ross to Florida.
"In welcoming Terrence to the Magic family, we have added an athletic, shot-making wing to our developing, young core, while also making our roster more versatile," general manager Rob Hennigan said in a news release.
"We thank Serge for his contributions and professionalism, and certainly wish him well."
The Raptors currently stand at 32-23, six games adrift of the Cleveland Cavaliers in the Eastern Conference.