This paper presents ongoing work on the evaluation of a tool for collecting feedback in large-scale software engineering and program comprehension experiments. Two most important questions one can ask about a new verbal procedure are whether it is useful and whether it is reactive. We have developed a tool, called the think-aloud tool, to support the think-aloud method in a variety of experiments. This tool instructs the subjects at regular intervals to write down their thoughts on a web based screen, as opposed to the oral expression of thoughts in conventional think-aloud method. The tool collects the subjects' feedback together with timestamps. This paper describes an experiment designed to investigate whether the think-aloud tool affects performance of the subjects and whether it collects information beyond that collected in traditional think-aloud and retrospective reports. Furthermore, we will conduct a qualitative comparison of mouse and keying rates with think-aloud and retrospective reports. In addition, the experiment aims to check for any effect from using the think-aloud tool on the subject's task-related understanding and problem solving.