Almond (Prunus dulcis) is a nutritious food with a rich content. In addition to consuming as food, it is also used for various purposes in sectors such as medicine, cosmetics and bioenergy. With all these usages, almond has become a globally demanded product. Accurately determining almond variety is crucial for quality assessment and market value. Convolutional Neural Network (CNN) has a great performance in image classification. In this study, a public dataset containing images of four different almond varieties was created. Five well-known and light-weight CNN models (DenseNet121, EfficientNetB0, MobileNet, MobileNet V2, NASNetMobile) were used to classify almond images. Additionally, a model called 'Genetic CNN', which has its hyperparameters determined by Genetic Algorithm, was proposed. Among the well-known and light-weight CNN models, NASNetMobile achieved the most successful result with an accuracy rate of 99.20%, precision of 99.21%, recall of 99.20% and f1-score of 99.19%. Genetic CNN outperformed well-known models with an accuracy rate of 99.55%, precision of 99.56%, recall of 99.55% and f1-score of 99.55%. Furthermore, the Genetic CNN model has a relatively small size and low test time in comparison to other models, with a parameter count of only 1.1 million. Genetic CNN is suitable for embedded and mobile systems and can be used in real-life solutions.