How to Determine the Machining Center Precision Level: Just Four Steps
A machine's brochure number tells you very little about the parts it will actually cut. This guide shows how to determine the machining center precision level yourself: run four checks, read the numbers against your drawing tolerance, and decide whether the machine belongs on the job. Written for engineers and buyers who have to sign off on a process, not a spec sheet.

Key takeaways
What the X, Y, Z Axes Actually Contribute
When an engineer asks how to determine the machining center precision level, most start with positioning accuracy. That is the right place, but only if you understand what the number covers. Positioning accuracy is the difference between where the control commanded an axis to go and where the axis actually arrived, measured along the full travel. On a 4,000 mm bed, small angular errors at the screw and guideway get multiplied by distance, so the error at the far end of travel is usually larger than near the home position.
Repeatability is the more useful figure for production. Send the axis to the same nominal point 30 times and measure the spread. A machine can have modest positioning accuracy and still hold a tight tolerance if it repeats, because the operator or the CAM post can compensate a consistent offset. It cannot compensate a random one. When we qualify a machine for a ±0.005 mm job, repeatability is the number we look at first and the number we re-check most often.
Backlash and reversal error sit underneath both figures. Approach a point from +X, then from –X, and compare. If the two land 0.010 mm apart, every contour reversal in your program inherits that error. Ball screw preload, thrust bearing condition and the coupling between motor and screw all show up here. A machine that has cut hard steel for years often loses repeatability through the screw and bearing before the control or the scales show any fault.
Squareness and parallelism matter as soon as you have more than one feature. A machine can be accurate on each axis alone and still cut a bore and a face that are not perpendicular. For a gearbox housing or a manifold with cross-drilled ports, check squareness between X, Y and Z at the same time you check travel. That is usually a laser interferometer job, and it should be done after the machine has been leveled and settled on its foundation.
One more practical point. Positioning numbers are quoted at a given feed rate, usually rapid or a standard test feed. Run the same check at your finishing feed and the result can change. Servo tuning that is soft enough to avoid vibration at 200 mm/min may lag at 4,000 mm/min. If your part has both long rapids and tight finishing passes, ask for the test at both ends of the range.
- 1Check travel, not just centerError grows with distance on long beds.
- 2Repeatability beats absolute accuracyA consistent offset can be compensated; scatter cannot.
- 3Test both directionsReversal error hides in single-direction checks.
- 4Add squarenessMulti-feature parts fail on geometry, not on single-axis error.
Spindle Error Motion and Tool Runout
The spindle is where the machine meets the cut, and it is the hardest error source to separate from everything else. Measure it in two parts. First, static runout: put a test bar in the taper, indicate near the gage line and again 300 mm out. A spindle that reads 0.002 mm close in and 0.015 mm far out has a taper or alignment problem, not a bearing problem. Second, dynamic error motion: run the spindle at 8,000 rpm, 12,000 rpm and its rated maximum, and measure the same points. The reading should not grow much with speed on a healthy spindle.
Thermal growth and centrifugal force both push the reading up at speed. A spindle that is fine cold and shows 0.020 mm at 15,000 rpm will produce tapered bores and inconsistent surface finish across a batch. On aluminum at 300 m/min cutting speed, that taper is often the first symptom an operator notices, long before any alarm appears.
Toolholder condition is part of the same chain. A CAT40 or HSK-A63 holder with chips on the taper, a worn retention knob or a damaged collet adds runout that no machine spec can absorb. Measure at the tool tip, not at the holder. A 12 mm end mill with 0.010 mm runout cuts one flute deeper than the others and wears unevenly, which then shows up as a dimensional drift over the run.
Drawbar force belongs in this step too. Low clamping force lets the holder move under load, and the error appears only during heavy cuts. If a machine holds tolerance in light finishing and drifts in roughing, check drawbar force before you blame the control. This is a quick check with the right gage and it saves a lot of guessing.
Balance the assembly before you measure anything. An unbalanced holder at 12,000 rpm excites the spindle and the readings become noise. Balance the tool and holder together at the speed you intend to run, then repeat the runout check. The numbers usually settle down immediately.
- 1Indicate at two lengthsShort and 300 mm out separates taper error from bearing error.
- 2Measure hot and coldError motion should stay stable from 8,000 rpm to max speed.
- 3Check at the tool tipHolder, collet and retention knob all add to the total.
- 4Verify drawbar forceLow clamp force shows up only under heavy cutting load.
Thermal Drift Over a Full Shift
A machine that measures well at 8:00 a.m. and poorly at 2:00 p.m. has a thermal problem, and no amount of laser calibration on a cold machine will fix it. Ball screws, spindle bearings, linear guides and the bed casting all heat up at different rates. The result is a slow, directional shift that can reach 0.020–0.050 mm over several hours on an older machine without compensation.
To measure it, run a warm-up cycle that matches real production, then cut or probe a reference feature every 30 minutes for 4–6 hours. Plot the values. You are looking for the shape of the curve, not one number. A machine that drifts 0.015 mm and then plateaus is workable with a warm-up routine. One that keeps climbing all shift is not, at least not for tight work.
Coolant temperature is a common culprit in shops that run aluminum all day. A chiller set too warm, or a tank that has been topped up with cold water, changes the machine's thermal environment faster than the structure can respond. Keep coolant within a few degrees of ambient and check it at the same time you check the machine.
Compensation helps if the machine has it and the control is set up correctly. Screw pitch compensation and thermal compensation are only as good as the map behind them. Ask when the map was last updated and whether it was built from measurements on this machine or copied from a sister machine. Copied maps on a different foundation are close to useless.
Environment matters more than most buyers expect. A machine in a bay with a big roll-up door facing afternoon sun sees real temperature swings. If your tolerance is ±0.005 mm and the shop swings 8 °C through the day, the machine is not the only variable. Sometimes the cheapest fix is a screen, a curtain or moving the machine away from the door.
- 1Warm up before you measureRun 60–90 minutes of representative cutting first.
- 2Sample every 30 minutesFour to six hours shows the drift curve, not a snapshot.
- 3Watch coolant temperatureRapid changes move the structure before it can settle.
- 4Ask when compensation was mappedA stale or copied map does more harm than none.
Cut a Test Part and Read the Report
Geometry tests tell you what the machine can do in theory. A test part tells you what it does with a real setup, a real tool and real material. Cut a part with features that match your production work: a bored hole with a tight diameter and position tolerance, a face that has to be flat, a slot with a width tolerance, and if possible a feature on two sides that tests squareness. Measure everything on a CMM, not with calipers.
Run the test part at the feeds and speeds you plan to use, with the same workholding. This is where fixture-induced error shows up. Three-jaw chucks distort thin rings. Vises lift thin plates. Magnetic chucks change the part's stiffness. If the test part is held differently from production, the numbers do not transfer.
Cut several parts, not one. Five to ten parts from the same program shows you drift across the run: tool wear, thermal growth, chip buildup and fixture relaxation. If part one is 0.003 mm over and part ten is 0.008 mm under, the machine is drifting, and you need to know that before you quote the job.
Compare the CMM results against the machine's own probe or scale readings where you can. A gap between the two tells you something about the machine's measurement chain versus its motion. On machines with linear scales, this gap is usually small. On machines running on encoder feedback only, it can be larger and it changes with temperature.
Finally, write down what you found. Which features held, which did not, what the ambient temperature was, what tool and holder you used. A one-page record per machine turns a vague feeling about a machine into data you can use when the next tight job comes in. That record is also what you send to a supplier when you ask them to prove their capability.
- 1Match production conditionsSame tool, holder, fixture, feeds and material.
- 2Cut five to ten partsOne good part does not prove a process.
- 3Measure on a CMMCalipers cannot resolve ±0.005 mm reliably.
- 4Keep a one-page recordAmbient temperature, tool, holder, results, date.
Four Steps to Determine the Machining Center Precision Level
Run in this order and do not skip the warm-up.
- 11. Warm the machine and set the baselineRun a warm-up cycle that matches production for 60–90 minutes. Record ambient and coolant temperature. Level and clean the machine before any measurement. A cold machine gives numbers you cannot use.
- 22. Measure axis positioning and repeatabilityUse a laser interferometer or a ballbar. Check full travel on X, Y and Z, both directions, at rapid and at finishing feed. Send each axis to one point 30 times for repeatability. Expect reversal error; note it. If repeatability is worse than one quarter of your part tolerance, stop here.
- 33. Measure spindle error motion and tool runoutIndicate a test bar at the gage line and 300 mm out, hot and cold. Run at 8,000 rpm, 12,000 rpm and rated maximum. Measure runout at the tool tip with the production holder. Check drawbar force with the correct gage.
- 44. Cut a test part and measure it on a CMMUse production material, tool, holder and fixture. Cut five to ten parts with a bored hole, a flat face, a slot and a two-sided squareness feature. Measure on a CMM. Compare part one to part ten to see drift across the run.
- 55. Compare results against your drawing toleranceAdd up the error sources that apply to your part. If the total budget exceeds about one quarter of the drawing tolerance, the machine is not suitable for that feature as set up. Re-check fixture, tool and thermal routine before rejecting the machine.
- 66. Repeat the short checks monthlyRepeatability and tool-tip runout are the two numbers that move first. A monthly 30-minute check catches a worn screw or a damaged holder before it produces a rejected batch.
What Each Result Means for Your Part
Rule of thumb: the machine should be at least 4× tighter than the tolerance it must hold.
| Check | Good result | Warning sign | What to do |
|---|---|---|---|
| Positioning accuracy | Within 1/4 of part tolerance over full travel | Error grows sharply at the far end of travel | Re-level, re-map screw compensation |
| Repeatability | Spread under 0.005 mm at 30 hits | Scatter worse than 0.010 mm | Inspect screw, thrust bearings, coupling |
| Reversal error | Under 0.005 mm between directions | Above 0.010 mm on any axis | Adjust preload or replace bearing set |
| Spindle runout at tool tip | Under 0.005 mm with production holder | Grows past 0.015 mm at max speed | Balance tool assembly, inspect taper |
| Thermal drift | Plateaus within 0.015 mm per shift | Keeps climbing all shift | Add warm-up cycle, control coolant temp |
| Test part, part 1 vs part 10 | Within 0.005 mm of each other | Drifts more than 0.010 mm over the run | Check tool wear and fixture relaxation |
| Cost check | No minimum order quantity | Unclear who owns inspection data | Ask for reports with the shipment |
Questions Engineers Ask Next
Can a machine with ±0.005 mm positioning accuracy hold a ±0.005 mm part tolerance?
Not reliably. Positioning accuracy is only one error source. Spindle runout, thermal drift, fixture distortion and tool wear all add to it. A practical rule is that the machine's combined error budget should be about one quarter of the part tolerance, which means a ±0.005 mm part usually needs a machine that measures around ±0.001–0.002 mm on the critical axis.
If the tolerance is on a single feature and the machine repeats well, you can sometimes run tighter than the rule of thumb with in-process probing and a warm-up routine. That is a process decision, not a machine specification.
How often should we re-check a machine's precision?
Run repeatability and tool-tip runout monthly. They are fast checks and they move first when a screw, bearing or holder starts to fail.
Do a full laser or ballbar check every 6–12 months, after any crash, and after the machine is moved or re-leveled. If the machine runs hard materials or two shifts a day, shorten that interval.
Does a linear scale make the machine accurate?
A linear scale measures the table position directly, so it removes screw pitch error and most thermal growth of the screw from the positioning loop. That is a real gain on long travels.
It does not fix squareness, spindle error motion, fixture distortion or vibration. A machine with scales and a worn spindle still cuts bad parts. Scales improve the axis, not the whole process.
What ambient temperature should a precision machining area hold?
For work near ±0.005 mm, hold the shop within about ±2 °C of a set point through the day. The absolute temperature matters less than the stability. A shop at 22 °C that stays at 22 °C all day is easier to work in than one that swings from 18 °C to 28 °C.
Watch doors, skylights and afternoon sun on the machine bed. Local swings near the machine are worse than a slow change across the whole shop.
Should we ask a supplier for machine test reports?
Yes, and ask for the method with the number. Which standard, which instrument, hot or cold, when it was done. A capability study on your actual part is more useful than a machine brochure figure, because it includes the fixture, the tool and the operator.
At GreatLight, inspection reports are available on request, and every order goes through raw material check, in-process monitoring and 100% inspection before shipment.
When is a machine not the right choice for a tight job?
If the required tolerance is tighter than the machine's repeatability after a proper warm-up, no amount of programming will fix it. Move the feature to a grinder, a jig borer or a machine with better thermal control.
The same applies when the part is thin, flexible or held in a fixture that distorts it. In that case the setup is the limiting factor, and changing machines will not help.
Send Us the Drawing and We Will Read the Tolerance
Upload your part and we will return a quotation with a free DFM analysis within 12 hours, including a review of which features need which machine and how we plan to hold them.
12-hour quote100% inspectionNo minimum order quantityNDA on request