Yoma 19a - Wearing out the kohein gadol

The gemara is figuring out where the Parhedrin and Avtinas rooms are. It gives the argument that a certain arrangement must be right, or the kohein gadol would have to walk one extra time across the width of the azarah. It then responds, maybe not, maybe the point is that if the kohein gadol is a Tzeduki, he'll be so overwhelmed that he'll quit. Rashi says only someone will stay with יראת שמים, who can accept on himself the effort of the Mikdash.

Can't follow any of this. Why would one extra time across the azarah (135 amos according to the earlier gemara), once a day for a week, mean anything at all?