How Is the Precision Level of the Tool Tool Detected?
A step-by-step procedure for measuring the accuracy of a CNC machine tool before it cuts a production part. Written for engineers and buyers who need to read an inspection report and decide whether the machine is fit for the job.
In this article
- 1
- 2
- 3
- 4
- 5
- 6
- 7
What the procedure actually proves
What "tool tool detected" means on a machine tool
The phrase tool tool detected is shorthand used on older CNC documentation pages for the measurement of a machine tool's own accuracy. It is not about the cutting tool in the spindle. It is about the machine: how straight the axes travel, how true the spindle spins, and how closely the commanded position matches the real one. That distinction matters, because a worn end mill and a worn ball screw produce the same symptom on a part but need completely different fixes.
Detection splits into four families. Geometric accuracy covers straightness, squareness and parallelism of the axes. Positioning accuracy covers how close the machine gets to a commanded coordinate. Repeatability covers how consistently it returns to that coordinate. Working accuracy covers the result of an actual cut. Each family needs its own instrument and its own acceptance number. Skipping one leaves a blind spot that shows up later as an out-of-tolerance feature.
On a machine quoted at ±0.005 mm, the measurement chain itself has to be at least four times tighter than the tolerance being checked. That is the rule of four-to-one. A dial indicator with 0.01 mm graduations cannot verify a 0.005 mm claim. It can only tell you the machine is roughly in the right place. For real numbers you need a granite square, a laser interferometer, a ballbar, and a spindle analyzer, all with current calibration certificates.
- 1Geometric accuracyStraightness, squareness, parallelism, spindle axis orientation.
- 2Positioning accuracyDistance between commanded and actual position.
- 3RepeatabilitySpread of results over repeated approaches to one point.
- 4Working accuracyThe finished part measured on a CMM.
Which instrument answers which question
A granite square and a dial test indicator check squareness between two axes. You sweep the indicator along the square's reference face and read the total deviation across the travel. On a 500 mm travel, a squareness error of 0.01 mm over 300 mm is already 20 arc-seconds, which is too much for tight work. The square itself must be grade 00 or better, and it must be seated on a clean, stone-flatted table.
A laser interferometer measures linear positioning error along each axis. It reports both the total error and the lead-screw or scale compensation curve. Modern systems resolve to 0.001 μm and handle travels past 4,000 mm. The data feeds directly into the controller's pitch error compensation table, so the measurement and the correction are the same operation. Run it at the temperature the machine will actually work at, not at whatever the shop happens to be that morning.
A ballbar traces a circular path with two axes moving together and reports circularity, backlash, servo mismatch and squareness from a single setup. A 300 mm radius test at 480 mm/min is the common baseline. The polar plot shows where the error sits around the circle, which separates a reversal spike from a servo lag. Spindle runout and thermal growth need a spindle analyzer and a set of temperature sensors, because neither shows up in a static geometry check.
The test cut closes the loop. Machine a part with a known geometry, then measure it on a coordinate measuring machine at a controlled 20 °C. This is the only check that includes tool deflection, workpiece clamping, and thermal effects at the same time. Cutting forces bend even a rigid setup, so a 20 mm diameter end mill taking a 3 mm radial depth in 6061 will deflect more than the machine's own positioning error. Keep the test cut conservative: light depths, sharp tool, plenty of coolant.
When the numbers say the machine is not suitable
Not every job needs a laser interferometer. If your tightest feature is ±0.05 mm on an aluminium bracket, geometric checks and a test cut are enough. The cost of a full volumetric calibration only pays back when the tolerance is below about ±0.02 mm, or when the part is large enough that a small angular error becomes a large linear one. A 0.02 mm per 300 mm squareness error over a 1,000 mm part becomes 0.067 mm at the far end. That is the kind of math that decides whether a machine is suitable.
Repeatability is the number to trust for production. A machine with a positioning error of 0.02 mm that repeats to 0.002 mm can still hold a tight tolerance if you compensate the offset. A machine that is dead accurate on average but scatters ±0.015 mm cannot, because no offset fixes a random spread. Ask for the repeatability figure over at least 30 unidirectional approaches, not a single best-case reading.
Thermal behavior sets the practical floor. A spindle that grows 20 μm in the first hour of running will drift out of tolerance on a long cycle even if every static check passed. Warm the machine for 30 to 60 minutes at the working spindle speed before you measure anything, and log the ambient temperature. If the shop swings 8 °C between day and night, no calibration will hold a ±0.005 mm band around the clock. Control the room, or accept a wider band.
What this looks like in a real production run
At GreatLight we run 127 high-precision CNC machines across three plants, including 16 simultaneous 5-axis machining centers and 16 mill-turn centers. The largest travel is 4,000 × 400 × 150 mm. Machines that hold ±0.005 mm on a routine basis get the full check sequence on a fixed schedule, and any machine that fails repeatability is pulled from tight-tolerance work before it can spoil a batch.
The practical outcome is a qualification rate of 99.99% and 100% inspection before shipment. Raw material is checked on arrival, in-process dimensions are monitored during the run, and a final inspection closes the job. Reports go out with the parts when the customer asks. For a medical or aerospace program, that paper trail is the difference between a shipment that is accepted and one that sits in quarantine.
For buyers, the useful question is not whether a supplier owns a laser interferometer. It is whether they can show a dated report for the specific machine that will run your part, with the residual error after compensation and the repeatability spread. Ask for it. A supplier who cannot produce that document is asking you to trust a marketing claim instead of a measurement.
- 116 five-axis centersSimultaneous 5-axis work for complex geometry.
- 24,000 mm max travelLarge-format parts on the long-travel machines.
- 3±0.005 mmWorking tolerance on qualifying machines.
- 4100% inspectionRaw material check, in-process monitoring, final inspection.
How is the precision level of the tool tool detected: step by step
Run the checks in this order. Each one assumes the previous one passed.
- 1Clean and warm up the machineWipe the table and way covers, then run the spindle at working speed for 30–60 minutes. Log ambient temperature. Measuring a cold machine is the single most common source of bogus numbers.
- 2Level the bed and check geometrySet a precision level on the table in both directions. Accept no more than 0.02 mm/m. Then sweep a granite square with a 0.001 mm test indicator and record squareness per 300 mm of travel.
- 3Measure spindle runout and axis orientationPush a clean test arbor into the taper and indicate it near the gauge line and 300 mm out. Taper runout should stay under 0.005 mm; the far-end reading tells you the spindle axis is not parallel to Z. Check radial and axial faces both.
- 4Run the laser linear measurementSet up the interferometer on each axis, zero at the reference end, and step in 50 mm increments over full travel. Record forward and reverse readings, then load the pitch error compensation. Repeat until the residual is inside 0.005 mm.
- 5Run a ballbar circle testUse a 300 mm radius at 480 mm/min in XY. Read circularity, backlash and servo mismatch from the polar plot. A reversal spike points to backlash; a two-lobe shape points to squareness; a lagging plot points to servo tuning.
- 6Do 30 unidirectional repeatability approachesCommand the same point 30 times, approaching from the same direction each time. Take the max minus min spread. This figure, not the average error, is what predicts production capability.
- 7Cut and measure a test pieceMachine a step-and-bore test part from 6061-T6 with light depths and a fresh tool. Measure it on a CMM at 20 °C. Compare bores, steps, and hole-to-hole distances against the drawing. File the report with the machine serial number and date.
Checks, instruments and typical acceptance limits
Starting points for a general-purpose 3-axis machining center. Tighten or loosen to match the part tolerance.
| Check | Instrument | Typical limit | What it catches |
|---|---|---|---|
| Bed level | Precision level | ≤ 0.02 mm/m | Foundation settling, twist |
| Squareness | Granite square + indicator | ≤ 0.01 mm per 300 mm | Axis non-perpendicularity |
| Spindle radial runout | Test arbor + indicator | ≤ 0.005 mm | Bearing wear, taper damage |
| Linear positioning | Laser interferometer | ≤ 0.005 mm after comp | Ball screw pitch error |
| Circularity | Ballbar, 300 mm radius | ≤ 0.015 mm | Backlash, servo mismatch |
| Repeatability | Laser, 30 approaches | ≤ 0.003 mm spread | Random scatter, sticking |
| Test-cut accuracy | CMM at 20 °C | Per drawing, ±0.005 mm | Total process error |
Questions engineers ask next
How often should the precision level be re-detected?
A full check on a tightly toleranced machine is usually annual, with a ballbar check every three to six months. After a crash, a move between buildings, or any foundation work nearby, run it again regardless of the calendar.
High-utilization machines on lights-out shifts drift faster. If the machine runs unattended overnight, the thermal cycle alone justifies a shorter interval.
Can a machine be too accurate to measure with a dial indicator?
Yes. A 0.01 mm graduation indicator has a resolution coarser than the tolerance you are trying to confirm. It cannot validate a ±0.005 mm claim.
Use a 0.001 mm test indicator for geometry and a laser interferometer for linear positioning. The rule of four-to-one applies: the measurement chain should be four times tighter than the tolerance under test.
Why does a machine pass its checks but still produce out-of-tolerance parts?
Machine accuracy is only one term in the error budget. Tool deflection, clamping distortion, thermal growth of the workpiece, and chip recutting all add to it.
Run a test cut on the actual material and fixture. If the machine passes and the part fails, the remaining error is in the process, not the machine. That is where a DFM review usually pays for itself.
Does a 5-axis machine need different checks?
The three linear axes use the same procedure. The two rotary axes add their own error: center offset, angular positioning, and the alignment between the rotary table and the spindle.
A ballbar run with the rotary axes locked, followed by a test cut on a cone or a compound-angle feature, is the usual way to confirm the rotary geometry. On a Ø400 mm rotary table, a 20 μm center offset becomes a visible feature error.
What documents should come with a machine accuracy report?
The machine serial number, the ambient temperature during measurement, the instrument model with its calibration certificate number and expiry date, the raw readings, and the residual error after compensation.
A single summary figure with no raw data is not a report. It is a claim. Ask for the trace.
How does repeatability differ from accuracy?
Accuracy is how close you get to the commanded point. Repeatability is how tightly the results cluster on repeated attempts.
For production, repeatability matters more. A consistent offset can be compensated in the controller. A random spread of ±0.015 mm cannot be compensated at all, and it will show up as scrap.
Send a drawing and get a machinability answer
Upload your part and we will return a quotation with a free DFM analysis within 12 hours. No minimum order quantity, from one prototype to 10,000+ parts. Uploads are secure and confidential, and an NDA is available on request.
12-hour quote±0.005 mm tolerance100% inspectionNo MOQ
More from the shop floor
We publish setup notes, tooling trials and inspection data from the factory floor.
