What Do You Mean by Resolution of a CNC Machine?

• 1
  The resolution of CNC machine control: three numbers inside one box
• 2
  Why resolution of CNC machine axes is not accuracy
• 3
  Angular resolution of CNC machine rotary axes
• 4
  When a finer resolution of CNC machine will not help
• 5
  How we match the resolution of CNC machine axes to your drawing
• 6
  Resolution of CNC machine compared with accuracy and repeatability

Ask five machinists what the resolution of a CNC machine means and you will get five answers. The word gets used for the smallest step the control can output, the smallest step the scale can read, and the smallest error the servo loop can correct. Those are three different numbers, and each one sets a different ceiling on your part.

Command resolution is the smallest move the interpolator can issue. On a common control this is 1 μm or 0.1 μm. It is a software limit. The control rounds every toolpath point to that grid before it sends a position to the drive.

Encoder resolution is what the feedback device reports. A 20-bit absolute encoder on a 10 mm pitch ball screw resolves about 0.01 μm per count. A 2,500 line incremental encoder with 4× quadrature and a 10 mm pitch resolves 1 μm per count. Same machine frame, very different feedback.

Servo resolution is what the loop can actually hold. It depends on gain, friction, screw pitch error and thermal growth. This is the number that decides whether your ±0.005 mm callout is realistic on a Tuesday afternoon after six hours of cutting.

• 1
  Command resolutionSmallest programmed increment, set by the control
• 2
  Encoder resolutionSmallest feedback step, set by the scale or encoder
• 3
  Servo resolutionSmallest stable position the loop can hold under load

A machine can have 0.1 μm encoder counts and still miss a bore position by 20 μm. Resolution tells you the size of the steps. Accuracy tells you how far the average position sits from the commanded position. Repeatability tells you how tightly the machine returns to the same spot.

The gap between resolution and accuracy comes from the mechanical chain. Ball screw pitch error on a rolled screw can run 50 μm over 300 mm. Thermal growth on a 500 mm aluminium part can move 0.012 mm per 1 °C. Backlash on a worn thrust bearing adds a few microns that no encoder count can see.

This is why a quote that promises ±0.001 mm because the machine has a 0.001 mm resolution is a red flag. The resolution number is real. The tolerance claim built on top of it is not.

• 1
  Pitch errorRolled screws drift; ground screws cost more and drift less
• 2
  Thermal growthAluminium moves about 23 μm per metre per 1 °C
• 3
  Backlash and stick-slipShow up as dwell marks and oval bores, not as position error

Linear resolution gets most of the attention. Rotary resolution decides whether a five-axis toolpath leaves facets on a curved surface. On a Ø400 mm rotary table with a 1 arc-second encoder, the surface error at the rim is about 2 μm. With a 10 arc-second encoder it is about 19 μm.

For a part 100 mm from the center of rotation, the same two encoders give 0.5 μm and 4.8 μm. Distance from the axis is the whole story. This is why a small medical implant can hold a fine blend on a mid-range table, while a 600 mm aerospace housing needs a better encoder to hit the same surface finish.

The practical check: multiply arc-second resolution by the distance from the rotary axis in metres, then by 4.85. That gives you surface error in microns. If the number is bigger than one third of your surface tolerance, the rotary axis is your limit, not the tool.

Finer resolution costs money in encoders, drives, screws and temperature control. It only pays back when the rest of the process can use it. Three cases where it does not.

First, when the feature is smaller than the tool. A 0.5 mm cutter deflects 10–30 μm under normal finishing loads. No axis resolution fixes tool deflection. Change the toolpath or the tool, not the machine.

Second, when the setup dominates. A part held in a vise and re-clamped between operations will see 10–20 μm of setup shift. That swamps a 1 μm axis. If your drawing has a 0.01 mm true position between two faces machined in different setups, the fix is one-setup five-axis work, not a better encoder.

Third, when the material moves after cutting. Thin aluminium walls, long stainless shafts and any heat-treated steel part can move more in the hours after machining than the machine's resolution. Stress relief and part design come first.

• 1
  Tool deflectionDominates below 1 mm cutter diameter
• 2
  Setup shift10–20 μm per re-clamp on typical fixtures
• 3
  Post-machining movementWorst on thin walls and stressed parts

At GreatLight we run 127 high-precision CNC machines across three plants, including 16 simultaneous five-axis machining centers, 12 four-axis mills and 16 mill-turn centers. Maximum processing size is 4,000 mm. Held tolerance is ±0.005 mm, and fine finishes run Ra 0.2–0.8 μm.

We do not pick a machine by its resolution spec sheet. We pick it by the tightest feature on your drawing, the number of setups, and the material. A 6061 bracket with a ±0.05 mm profile goes on a three-axis mill. A 17-4PH medical component with a 0.01 mm true position across five faces goes on a five-axis center, in one setup.

Every job gets a raw material check, in-process monitoring and a final inspection. Reports are available on request. If a callout cannot be met by any machine in the shop, we say so in the DFM analysis instead of quoting it.

Quotation and free DFM analysis come back within 12 hours. Production can start within 24 hours. Parts ship in 3–5 days.