The productivity of high speed milling operations is limited by the onset of self-excited vibrations known as chatter. Unless avoided, chatter vibrations may cause large dynamic loads damaging the machine spindle, cutting tool, or workpiece and leave behind a poor surface finish. The cutting force magnitude is proportional to the thickness of the chip removed from the workpiece. Many researchers focused on the development of analytical and numerical methods for the prediction of chatter. However, the applicability of these methods in industrial conditions is limited, since they require accurate modelling of machining system dynamics and of cutting forces. In this study, chatter prediction was investigated for orthogonal cutting in turning operations. Therefore, the linear analysis of the single degree of freedom (SDOF) model was performed by applying oriented transfer function (OTF) and \tau decomposition form to Nyquist criteria. Machine chatter frequency predictions obtained from both forms were compared with modal analysis and cutting tests. © 2010 Elsevier Ltd. All rights reserved.