Methodology

This site's one promise is that its numbers are right and its hedges are honest. This page is the whole account of how that is kept.

Exact rational arithmetic

Every fraction is stored as a pair of arbitrary-precision integers — numerator and denominator — reduced by their greatest common divisor, with the sign carried on the numerator. Addition, subtraction, multiplication, division, and comparison are performed on those pairs directly. Comparison is by cross multiplication. Floating point appears nowhere in any calculation; it is used only to position marks in drawings.

A decimal you type is treated as exact: 0.375 is the rational 375/1000, reduced to 3/8. A repeating decimal is exact only when the repetition is stated — 0.(3) is 1/3 by the standard construction over nines — and a long decimal is never assumed to repeat.

Units

One conversion constant exists: 1 inch = 25.4 mm, exactly, as defined by the international yard and pound agreement of 1959. As a rational number that is 127/5, so metric–inch conversions and their error columns are exact fractions until the moment they are printed. Centimetres and metres are decimal multiples of the millimetre.

Display rounding

Exact values print with =. Rounded values print with ≈ and are rounded half away from zero at the stated number of places. A terminating decimal short enough to print is always printed in full. A repeating decimal is printed rounded, with the exact parenthesis notation alongside where space allows.

Nearest fraction

The nearest mark on a rule of denominator d is found by rounding value × d to the nearest integer, computed exactly. When a value sits exactly halfway between two marks, it rounds away from zero — the tape-measure habit. The signed error of a candidate is candidate − target: positive means oversize, negative undersize.

The tolerance matcher's ranking

The simplest-fraction-within-tolerance tool considers every mark of every allowed denominator inside the window [target − tolerance, target + tolerance], deduplicated after reduction, and ranks the survivors by exactly these rules, in order:

1. Only candidates inside the tolerance qualify. If none does, the answer is that none does; a result is never forced.
2. A candidate exactly equal to the target beats everything.
3. The smaller reduced denominator is simpler and wins: 3/8 beats 25/64 whether 3/8 arrived as 3/8 or as 24/64.
4. Equal denominators are ranked by smaller absolute error.
5. A dead tie in error — the target sits exactly between two marks of equal complexity — goes to the smaller value, undersize, because in most shop work material can be removed and not added.

There are no other criteria. Marks at or below zero are never offered. The ranking is pinned by the test suite, and changing it requires changing this page in the same commit.

Rational approximation

The approximation tool computes continued-fraction convergents by the standard recurrence. Each convergent is a best rational approximation: no fraction with a denominator at or below its own lands closer. The exact conversion of the typed digits is always stated separately from the approximations, because they are different claims.

What is deliberately absent

• No guessed units. A bare number is never assumed to be inches or millimetres where the difference matters; the tools ask for the unit instead.
• No forced fits. Tolerance and equivalence questions can answer “no.” An empty result is reported as one.
• No model-decided numbers. Every value is deterministic arithmetic, checked by tests on every build.