The field-testing program

Why desk research is not enough

Nearly everything published about off-grid equipment is either a datasheet or an opinion, and the gap between them is where people lose money. A datasheet describes a unit under laboratory conditions: a battery at 25 degrees Celsius cycled gently, a panel at standard test conditions, a generator at a stated load fraction with fresh fuel and a new air filter. A review describes how something felt to somebody for a few weeks.

What sits between them is measurement: what a specific unit actually delivered, at a specific site, over a specific season, under a load somebody wrote down. That is expensive to produce and it is the only evidence that answers the question people actually have, which is not what a product is rated at but what it will do in their conditions.

This program exists to produce that, slowly and in small quantity, and to publish it in a form that can be checked and cited rather than believed.

The standard, and why it is published first

Publishing a testing standard before there are any results is deliberate. It means the standard was not written to flatter whatever happened to be measured, and it means a reader can hold every future test to a commitment made in advance. It also means the gaps are visible: where the protocol says a measurement should exist and a published test does not have it, that is a shortfall rather than an omission nobody noticed.

The same principle runs through the rest of this site. The calculators publish their formulas and constants, the laws pages publish their sources and mark what could not be confirmed, and the spec tables carry a confidence flag on every row. A reference that cannot be checked is a reference that has to be taken on trust, and trust is not what an expensive decision should rest on.

What a single test can and cannot establish

One unit, at one site, over one season, tells you what that unit did in those conditions. It does not establish a manufacturer's reliability, because reliability is a distribution and a single sample says almost nothing about it. It does not establish how the same equipment behaves in a different climate, at a different altitude, or under a different duty cycle. It does not establish long-term degradation until the test has run long enough to see it.

Those limits will be stated on every published test rather than left for the reader to infer, because the most common failure in equipment writing is a conclusion broader than the evidence underneath it. A test showing that one battery held a given capacity through one winter is a useful data point and not a verdict on the model.

What a small number of careful tests can do is falsify specific claims, quantify the gap between rated and delivered performance in real conditions, and give the calculators on this site measured constants in place of assumed ones. Those are worth having, and they are worth being modest about.

What gets measured, and with what

Energy in and out, with instrumentation rather than inference: shunt-based measurement at the battery, production at the array, and consumption at the load. Temperatures, because nearly every off-grid performance question is a temperature question underneath. Runtime and fuel, weighed or metered rather than estimated. Capacity delivered, measured under a stated discharge rate rather than assumed from nameplate.

Conditions get recorded alongside every reading: ambient temperature, weather, and the load or duty cycle applied. Without them a number is not a measurement, it is an anecdote with a decimal point.

And the raw data gets published with the writeup, in a form somebody else can open and analyze. If a conclusion drawn here is wrong, the data should be sufficient for a reader to demonstrate that.

How results change the rest of the site

When a product has been field-tested, its entry in the gear pages and its row in the spec tables move from researched to personally tested, with a link to the test. Until then the labels stay honest, because a great deal of the equipment covered here has been researched carefully and never touched.

Measured figures also replace assumed ones in the calculators. Where that happens, the formula version is bumped and the change is recorded in the public corrections log with a date, so that a result produced last year can be traced to the assumptions it used. Silent changes to a calculator are indistinguishable from errors, which is why they do not happen here.

Contributing measurements

Readers running their own systems are a better source of data than any single test site, because they cover more climates, more equipment, and more years. Contributed measurements are welcome, and they are credited when they inform a page.

What makes a contribution usable is the same as what makes a test usable: the equipment identified precisely, the conditions recorded, the instrument stated, and the raw numbers rather than a summary. Winter battery capacity, firewood consumption against heating degree days, well drilling costs by district, and generator fuel burn under real charging loads are all measurements that would improve this site immediately.

The forum carries standing threads for exactly those, and anything substantial can be sent through the contact page. Where a contribution changes something published here, the change is logged in the corrections record with the date and the source.

The test site, and why it is named

Field testing done anonymously is not much better than a datasheet, because the conditions are half the result. Tests published here will name the location, the elevation, the climate normals for the site, and the configuration of the system the equipment was installed in, so that a reader can judge how far the result travels toward their own conditions.

The first tests will come from a southern Vermont build: a cold, cloudy-winter, heavily wooded climate with a short solar window and a long heating season. That is a demanding environment for solar and a normal one for wood heat, and it is entirely unrepresentative of the desert southwest. Saying so is part of publishing the result.

Where a reader's site differs materially, the site report tool gives their own worst-month sun hours, heating degree days, frost dates, and design low, which is the honest way to translate a result from one place to another rather than assuming it carries.

Independence, and how the equipment is obtained

Equipment tested here is bought rather than supplied, wherever that is affordable, because a review unit is selected by the manufacturer and a purchased one is not. Where something is loaned or provided, that will be stated plainly at the top of the test rather than in a footnote, along with whether the manufacturer had any opportunity to review the result before publication, which they will not.

This site carries affiliate links on some pages and discloses them on every page that has them. No test outcome is influenced by whether a product carries a commission, and a test that concludes a product is a poor buy will say so regardless. The disclosure page sets out the whole arrangement.

The failure mode to guard against is not bribery, it is selection: quietly declining to publish disappointing results. The numbering scheme is sequential and permanent for that reason, and a test that is started and abandoned will be recorded as such.

Timeline, and what to expect

This program produces a small number of careful tests rather than a stream of them, because each one requires a season, instrumentation, and a system that is actually being lived with. That pace is a limitation and it is also the point: a result from a full winter is worth more than a dozen from a fortnight.

In the meantime, everything on this site is labeled for what it is. Desk research says desk research. Manufacturer figures are marked with their confidence and their date. Calculators show their assumptions. Nothing here claims to be measured until something has been.