How high is 'high'? Are we talking Inca Trial? I did this a few summers ago and I'll be returning to the area in August.
For 3-5000m trekking, days should be warm, so I will be walking in sarong or basketball shorts. I will also take powerstretch tights to add when it gets higher (therefore cooler), and for evenings, which will be much colder. The combined weight and volume of all these is lower than a pair of regular trousers. Although it's technically the dry season, you can get occasional rain higher up, so everything I'm taking is quick drying.
For higher level climbing I am taking thick Schoeller fabric trousers, as the temperature range is much wider, with more likelihood of cold days.