For a complete listing of the sequence of the avodah, look in Rashi on Chumash, or your Yom Kippur Machzor. I have not seen anyone who discusses the amount of time each avodah took, but since the list of avodos is quite extensive and it all had to be done by the Kohen Gadol it doesn't seem likely he had too much time for breaks.

Tosfos (20B D.H. "Meshum") does mention one break the Kohen Gadol would take. He says the reason why on Yom Kippur "Terumas Hadashen", the removal of the ashes from the Mizbe'ach, would be done earlier than usual, at midnight, was to give the Kohen Gadol a chance to rest before starting the day's service. Furhtermore, the Brisker Rav (see Insights to Daf 52) points out that the Kohen Gadol would stand by and wait on two occassions during the Avodah: in the Kodesh ha'Kodashim, until the room filled with smoke, and after sending away the Se'ir to the desert, until he received notice that it had reached the desert.

Regarding when the Kohen Gadol would finish the Avodah, the Mishnah in 70a says that after the Kohen Gadol's final Tevilah he would dress in his own clothes, and they would escort him home where he would make a celebration for his friends and relatives. The implication of the Mishnah is that this feast took place right then. This would mean that it was no longer Yom Kipur. (Although, according to one explanation of the Meiri this party was not until the next day.)

David Schloss