Background and objective: The effective performance of deep networks has provided the solution to various state-of-the-art problems. Convolutional Neural Network (CNN) is accepted as an accurate, effective, and reliable practice in image-based applications. However, there is a need to use pre-trained models in case of insufficient data in CNN. This study aims to present an alternative solution to this problem with the proposed 3D image-based filter generation approach with simpler CNNs for the classification of small datasets. Methods: In this study, a novel 3D image filters-based CNN (Hist3DCNN) is proposed. The proposed filter generation approach is based on 3D object images taken from different perspectives. The efficiency of Hist3DCNN is shown on a novel histological dataset that contains blood, connective, epithelium, muscle, and nerve tissue images. Various case studies are carried out with generated filters assigned as the initial value to AlexNet and the designed Hist3DCNN model that is simpler than AlexNet. Results: Based on results, the classification accuracy of AlexNet with proposed filters used in convolution layers were 84.65% and 85.34%. The accuracy was increased to 85.47% by Hist3DCNN on the histological image classification. Moreover, four different benchmark datasets were tested to demonstrate the robustness of Hist3DCNN on various datasets. Conclusions: This study provides a new aspect to literature due to 3D image-based filter generation approach to initialize convolution filters. Experimental results validate that Hist3DCNN can be used as a filter value initialization method with simple CNN models that contain less learnable parameters for the classification task of small datasets.