There are now several works reported in the literature which attempt to estimate surface colour when the same surface is viewed under two or more lights. There are many practical situations where such information is available including at shadow edges or in surveillance applications where the same scene is viewed over time. Crucially, because typical lights are highly constrained, they fall on or close to the Planckian locus, varying illumination algorithms for surface estimation can, plausibly, estimate surface colour even for scenes with little colour diversity.One of the first varying illumination algorithms made the empirical observation that the mappings, 2×2 diagonal matrices, taking spectral band ratio chromaticities (e.g. r/b and g/b) for surfaces viewed under a range of typical lights to corresponding values under a reference canonical light (e.g. D65) tended to lie on a 2D line. It follows that applying this ‘linear set’ of maps to the chromaticity of an arbitrary surface under unknown light results in a line along which the D65 counterpart should lie. Viewing the same surface under a second light results in a second constraint line. The intersection of the two lines results in an estimate of the surface chromaticity under D65.While this method can work well, Kawakami et al showed that serious estimation errors can result in the presence of even small amounts of image noise. While the noise tended to make only small changes in the slope and intercept of the constraint lines the intersection point could move a significant distance; indeed, the shifted intersection might correspond to a highly improbable (physically impossible) light. To solve this ‘intersetion stability’ problem they proposed limiting the set of maps not only to lie on a line but on a line segment (e.g. only allow illuminants that are physically plausible and likely). This observation, which necessitated dealing with the problem non-intersecting line segments, formed the foundation of a new algorithm which was shown to deliver a step change in surface colour estimation. In this paper we extend Kawakami's work in two ways. First, we deal with the non-intersecting line problem using a ‘total leastsquares’ approach (as oppose to assuming one or other of the line segments is in error in Kawakami's work). Second, we optimise over the position and length of the line segment map-set used. Experiments demonstrate that our new method delivers significantly improved surface colour estimation. Compared with the Kawakami method we deliver over 50% improved surface colour estimates. We also show that the Kawakami method can be improved by optimising the line segment map set but even in this case our new method still provides about a 25% decrease in estimation error.