Harrison's appointment was widely expected after a high-profile battle waged against the Calgary-based railway's former leadership by William Ackman, head of a New York-based investment fund.
Harrison is the former CEO of Canadian National Railway (TSX:CNR).
He is credited with turning CN into North America's most successful railway company – a track record that Ackman says is needed at Canadian Pacific.
Ackman's Pershing Square became the largest shareholder of Canadian Pacific last year, beginning months of battles that resulted in the departure of the railway's chief executive, its chairman and several directors.
Paul Haggis, who became CP chairman after the departure of John Cleghorn, said the new board appointed Harrison after a thorough selection process.
"The board welcomes Mr. Harrison's experience and leadership to CP. We look forward to benefiting from his strong track record of service reliability, efficient asset utilization and strategic capital expenditure," Haggis said in a statement.
Cleghorn had been a staunch defender of former Canadian Pacific CEO Fred Green, who resigned just hours before a shareholders meeting that pitted Ackman's nominees for the board against incumbent directors.
Green's defenders argued that he was the best person to reform Canadian Pacific's operations, a process that they said had already begun. They argued the reforms advocated by Ackman and Harrison would be too drastic or unachievable.
Ackman countered that Harrison's expertise and accomplishments were necessary to offset years of mismanagement during Green's years as leader of the country's second-largest railway company.
Harrison, who retired from Canadian National at the end of 2009, has said he welcomed the chance to return to work – especially given Canadian Pacific's illustrious history and its potential.
"CP is an incredible franchise with significant market opportunity, solid infrastructure and innovative and hard-working employees," Harrison said in a statement Friday.
"I am proud to be working with one of North America's iconic companies and I look forward to quickly getting to know the priorities of CP's customers, shareholders, employees, and the communities served by the railway."