In the same year, two teams submitted proposals to use the same telescope for galaxy surveys. One wanted to stare at a small patch of sky for months, catching the faintest light from the most distant galaxies. The other wanted to sweep wide, cataloging millions of galaxies quickly. The telescope could only host one project that season. The deciding factor was not the scientific vision, but a single sheet of paper: the mirror's polishing log, a record of surface roughness measured in nanometers.

A Polishing Log That Split a Panel

The log was a routine artifact, filed after the mirror's last recoating. It listed measurements of surface roughness across the mirror's 8.4-meter diameter, taken with an interferometer. The numbers were tiny, most below ten nanometers, well within the tolerance for the telescope's design. But the log also contained a sequence of measurements taken over several months, and that sequence told a different story depending on who read it.

One reviewer saw variance, a normal scatter of values around a stable mean. Another saw a trend, a slow drift upward in roughness, suggesting the coating was degrading. The two interpretations led to opposite conclusions about the mirror's performance in the coming year. If the surface was stable, the deep survey could proceed with confidence. If it was degrading, the wide survey was safer, because it needed less of the mirror's surface area at any given time.

The panel split along these lines. Three members argued the variance was statistical noise, common in any manufacturing process. Two argued the drift was systematic, a known failure mode for the coating material. The chair asked for a third analysis, but the log alone could not settle it. The measurements had been taken with different instruments on different days, and the calibration between them was not recorded.

This was not a case of bad science. Both sides had reasonable points. The log was a single, imperfect dataset, and the stakes were high: a year of observing time, several million dollars in operations, and the career trajectories of dozens of graduate students and postdocs whose projects depended on the outcome.

The Economics of a Nearly Perfect Mirror

Mirror polishing is a steeply nonlinear business. Each additional nanometer of smoothness costs exponentially more in labor and time. Getting a mirror from 10 nanometers to 5 nanometers might double the polishing hours. From 5 to 2, it might double again. The telescope's mirror had been polished to a surface roughness of about 3 nanometers, a level that required months of iterative grinding and polishing, with technicians measuring, adjusting, and repeating.

The infrastructure budget was fixed. The observatory had already spent the bulk of its annual allocation on the recoating and the subsequent polishing. Any additional work, such as a re-polish or a more frequent cleaning schedule, would come out of the operations budget, which was already tight. The trade-off was between maintaining the mirror at its current state and funding the survey itself.

Proposal A, the deep survey, required the mirror to be in near-perfect condition for the entire observing season. Its science case depended on pushing the detection limit to the faintest possible surface brightness. A slight degradation in the mirror's figure would blur the images, reducing the depth by a few percent, which could wipe out the faintest galaxies.

Proposal B, the wide survey, was more forgiving. It observed brighter galaxies, so a small loss in image quality would not significantly affect the number of usable objects. The team argued that even if the mirror degraded by 20% in the first six months, they could still meet 90% of their science goals. They also pointed out that the cost per resolved pixel, a metric they had computed, favored their approach.

The cost per resolved pixel became a battleground. Proposal A needed a pristine mirror for the entire run, so the cost of the mirror's maintenance was spread over a smaller number of pixels. Proposal B could tolerate some degradation, so the same maintenance cost was spread over more pixels. The numbers were close, but Proposal B's team made the case that their survey was more robust to the mirror's variability, and that robustness had value.

How the Log Was Read Differently

The log's measurements were plotted on a graph, and the two camps drew different lines through the points. One reviewer applied a linear regression and found a slope that was not statistically significant, p = 0.08. Another reviewer applied a moving average with a window of three months and found a clear upward trend, with a peak that exceeded the previous coating's roughness. The difference was not in the data, but in the smoothing method.

Statistical noise or systematic drift? The answer mattered, but the log alone could not distinguish them. The measurements were taken at irregular intervals, sometimes weekly, sometimes monthly, and the environmental conditions varied. Temperature and humidity affect interferometer readings, and the log did not record those covariates.

Publication pressure pushed both teams to make decisive claims. The deep survey team wrote a technical memo arguing that the drift was an artifact of the smoothing method, and that the underlying process was stable. The wide survey team wrote a memo arguing that the drift was real, and that the deep survey was too risky. Both memos were shared with the panel, but neither could be verified independently.

No replication was possible for a one-off mirror. Unlike a psychology experiment or a catalyst test, you cannot run the same mirror again. The log was the only evidence, and it was ambiguous. This is a common problem in astronomy, where instruments are unique and calibration data are often sparse.

The Role of Incentives in Survey Design

The dispute was not just about the mirror; it was about how the two teams were incentivized. Grant cycles favor novel, publishable results. A deep survey promises a few spectacular papers, each with a high citation potential. A wide survey promises many papers, each with a lower citation count but a higher total. Both are viable strategies, but they require different levels of risk tolerance.

Big surveys promise many papers per dollar, which appeals to funding agencies that want to maximize productivity. But if the instrument underperforms, the survey may produce nothing. The deep survey, by contrast, has a lower paper count but a higher chance of a breakthrough. Reviewers, who are themselves academics, often reward confidence over caution. A proposal that states "we will detect the first galaxies" is more compelling than one that says "we will try, but the mirror might not cooperate."

Incentives shaped which data was collected. The deep survey team had already invested in a custom filter set, and they had a pipeline for reducing ultra-deep images. The wide survey team had built a fast astrometry pipeline. Neither had planned for the possibility that the mirror would be a topic of debate. The log was an afterthought, a byproduct of maintenance, not a designed experiment.

This is a pattern familiar from other fields. A survey log from a replication study showed how a similar ambiguity in measurement could make an effect disappear, and the incentives to publish a positive result shaped how the data were analyzed. The mirror log was no different.

What the Panel Actually Decided

After two days of deliberation, the panel did not choose either proposal. They funded a compromise: the telescope would split the season, with the first six months devoted to a scaled-down version of the deep survey, and the last six months to the wide survey. The decision was explicitly framed as reversible. If the mirror's performance degraded more than expected in the first three months, the deep survey would be cut short and the wide survey would start early.

Both teams revised their scientific cases. The deep survey team agreed to reduce their field of view by a third, which reduced the risk of the mirror's edge effects. The wide survey team agreed to accept a shallower depth in the first six months, which they had originally argued against. The mirror log was still cited in the annual reviews, but it became a reference point for performance, not a source of contention.

The compromise was not ideal for anyone. The deep survey team lost the chance for a uniform, ultra-deep field. The wide survey team got a shorter season and a shallower start. But the panel's decision was pragmatic: it preserved the science while hedging against the mirror's uncertainty. The decision was also a test of whether the teams could adapt, which they did.

In the end, the mirror's log was not the sole determinant, but it was the catalyst. It forced the panel to confront the fact that the telescope was a finite, imperfect resource, and that any survey design had to be robust to that imperfection. The compromise reflected a lesson that is often learned the hard way in astronomy: the instrument is not a given, and its quirks are part of the science.

Lessons for Future Instrument-Based Disputes

The mirror log dispute offers several lessons for future instrument-based decisions. First, document measurement uncertainty explicitly. If the log had included the calibration history and the environmental conditions, the two interpretations might have been reconciled. A simple table of uncertainties, with a note on how they were estimated, would have saved days of debate.

Second, independent audits of calibration logs can help. An outside expert, with no stake in the outcome, might have been able to assess the drift more objectively. The panel could have commissioned such an audit, but they did not, partly because of time and budget constraints. In hindsight, that was a mistake.

Third, pre-register analysis plans before review. If the teams had agreed in advance on how to analyze the log, the ambiguity would have been reduced. This is a practice borrowed from psychology and economics, where pre-registration is increasingly common. For a one-off instrument, it is even more important, because there is no second chance to collect more data.

Fourth, separate scientific merit from engineering risk. The deep survey was scientifically more exciting, but the wide survey was more robust to the mirror's condition. The panel conflated the two, which led to the split. A clearer separation would have allowed them to weigh the risk explicitly, perhaps with a formal decision analysis.

Fifth, transparent criteria reduce post-hoc rationalization. The panel never stated explicit criteria for what would constitute a decisive failure or success. As a result, both teams could spin the outcome in their favor. A pre-defined threshold for the mirror's acceptable degradation would have made the decision easier.

These lessons are not unique to astronomy. A similar issue arose in a fMRI study where a scanner upgrade reordered the noise floor, forcing researchers to rethink their analysis. And in a catalyst study, 1,200 replicates were needed to distinguish a real effect from a solvent artifact. The mirror log was a similar test of inference under uncertainty.

The split decision was not a failure. It was a recognition that the evidence was ambiguous and that the cost of being wrong was high. The telescope eventually delivered data from both surveys, and both teams published papers. The mirror log became a footnote, but it remains a useful example of how a tiny number, measured in nanometers, can shape the direction of an entire field.

Beyond the Mirror: Analogous Instrument Disputes

To appreciate how unusual this outcome was, consider other fields where a single instrument's calibration data has become the crux of a funding decision. In particle physics, the alignment of a detector's silicon strips is routinely monitored, and a few micrometers of shift can alter the momentum resolution. In one case, a proposed dark matter search was delayed because the detector's background rate, measured in counts per kilogram per day, showed a slow increase over six months. The collaboration split over whether the rise was due to radon contamination or a subtle electronic gain drift, and the funding agency ultimately required a blind analysis of the calibration data before releasing the next tranche of support.

In genomics, the calibration of a sequencing machine's error rate is similarly contentious. A proposal to sequence a large cohort hinged on whether the machine's base-calling error rate, typically around 0.1%, had crept up to 0.15% over the previous quarter. The manufacturer insisted the increase was within specification, while the genomics center argued it would double the cost of achieving the required depth. The dispute was resolved by a re-analysis of the raw fluorescence traces, but only after weeks of negotiation.

These cases share a common feature: the instrument's performance is not a fixed property but a time-varying one, and the data needed to characterize it are often collected opportunistically rather than by design. The mirror log was no exception. It was a maintenance record, not a scientific dataset, yet it carried the weight of a scientific decision.

What made the mirror dispute particularly instructive was the absence of a pre-agreed protocol for interpreting the log. In fields like clinical trials, such ambiguities are mitigated by pre-specified analysis plans and independent data monitoring committees. Astronomy, with its reliance on unique, expensive instruments, could benefit from similar practices. The panel's eventual compromise was a step in that direction, but it came after the fact.

The mirror log also highlights the importance of quantifying the cost of being wrong. The deep survey's science case was more ambitious, but it carried a higher risk of failure if the mirror degraded. The wide survey was more robust but less likely to produce a transformative discovery. A formal decision analysis, weighing the expected scientific output under different mirror scenarios, might have led to a different conclusion. But such analyses are rare in astronomy, where institutional culture often favors bold claims over explicit risk assessment.

In the end, the mirror log served as a reminder that the frontier of astronomy is not just the sky but the instrument itself. The next generation of telescopes, with their segmented mirrors and adaptive optics, will generate even more complex calibration logs. Learning to interpret them wisely will be essential. The lessons from this dispute—document uncertainty, seek independent audits, pre-register analyses, separate merit from risk, and set transparent criteria—are more relevant than ever.