I was at the gym with Sarah, we were exercising when we heard Tom talking about it with John.

exercising 是动词 exercise 的现在分词形式。exercise 的意思是"锻炼、运动"，指进行体育活动或健身训练。exercising 在这里表示"正在锻炼"的动作。