The standard approach to mitigating errors made by an automatic
speech recognition system is to use confidence scores associated
with each predicted word. In the simplest case, these scores are
word posterior probabilities whilst more complex schemes utilise
bi-directional recurrent neural network (BiRNN) models. A number
of upstream and downstream applications, however, rely on confidence
scores assigned not only to 1-best hypotheses but to all words
found in confusion networks or lattices. These include but are not
limited to speaker adaptation, semi-supervised training and information
retrieval. Although word posteriors could be used in those applications
as confidence scores, they are known to have reliability issues. To make
improved confidence scores more generally available, this paper shows how
BiRNNs can be extended from 1-best sequences to confusion network and
lattice structures. Experiments are conducted using one of the Cambridge
University submissions to the IARPA OpenKWS 2016 competition. The
results show that confusion network and lattice-based BiRNNs can provide a
significant improvement in confidence estimation.