Regularization-robust preconditioners for PDE-constrained optimization problems have been successfully developed. These methods, however, typically assume that observation data is available throughout the entire domain of the state equation. For many inverse problems, this is an unrealistic assumption. We propose and analyze preconditioners for PDE-constrained optimization problems with limited observation data, e.g. when observations are only available at the boundary of the computational domain. Our methods are robust with respect to both the regularization parameter and the mesh size. That is, the number of required MINRES iterations is bounded uniformly, regardless of the size of the two parameters. The theoretical findings are illuminated by several numerical results.