The normative values of the Special Judo Fitness Test (SJFT) are used to evaluate judo athletes, and the question arises of which performance tests from crucial motor abilities best define the SJFT classification in elite judo athletes. This study aimed to investigate the relationship between elite judo athletes’ physical performance and the evaluation using SJFT index norms. Nineteen judo athletes (11 females) (22.8 ± 2.5 years old) from the senior judo national team voluntarily participated in this study. Body composition, reaction time, balance, flexibility, agility, hand grip strength, 20 m sprint, vertical jump, SJFT, and Wingate tests were performed by athletes on four separate days at one-day intervals. Athletes were classified as regular and above (≥regular) or poor and below (≤poor) according to their SJFT index scores. Simple logistic regression analysis was used to evaluate this classification’s consistency with performance test results. Odds ratios (OR) and 95% confidence intervals related to each possible factor and Wald test statistics were calculated. The SJFT index classification was associated with Wingate peak and mean power, vertical jump, and sprint performance results (p < 0.05), whereas it was not associated with body fat percentage, agility, reaction time, hand grip strength, flexibility, and balance performances (p > 0.05). SJFT index classificatory norms are mainly related to athletes’ anaerobic power. Higher anaerobic power increases athletes’ possibility of being classified as ≥regular.