Continuous engineering (CE) practices, such as continuous integration
and continuous deployment, have become key to modern software
development.  They are characterized by short automated build and
test cycles that give developers early feedback on potential issues.
CE practices help to release software more frequently, and reduces
risk by increasing incrementality.  However, effective use of CE
practices in industrial projects requires making sense of the vast
amounts of data that results from the repeated build and test cycles.

The goal of this paper is to investigate to what extent these data
can be treated more effectively by automatically grouping logs of
runs that failed for the same underlying reasons, and what effort
reduction can be achieved.  To this end, we replicate and extend
earlier work on system log clustering to evaluate its efficacy in
the CE context, and to investigate the impact of five alternative
log vectorization techniques.

We built a prototype tool that is used to conduct an empirical case
study on continuous deployment logs provided by our industrial
collaborator.  Questions to be answered include: (1) Can we reduce
the effort needed to discover all latent issues in a set of failing
runs?  (2) How to best leverage the contrast between passing and
failing runs to increase accuracy?  (3) What trade-offs are there
between effort reduction and accuracy?  We present a quantitative
and qualitative analysis of the results of our study.  We conclude
by evaluating the trade-offs, and give recommendations for applying
this approach in practice.