As our skin is exposed to ultraviolet rays or dangerous chemicals, aberrant growth of skin cells happens which brings up undesirable conditions such as premature skin aging, transposition in skin texture, and the worst-case scenario skin cancer. In the struggle to combat deadly skin cancer, machine learning can be a useful weapon to help dermatologists make better and clearer decisions while diagnosing patients. Despite promising results with numerous machine learning techniques, this field faces data inadequacy, more so the universally available datasets are subjected to data imbalances. In order to tackle the significant class imbalance present in datasets like the HAM10000 skin cancer dataset, this research introduces a class-weighted reward mechanism within the Deep Q-Learning framework that dynamically allocates higher positive rewards for the accurate classification of rare classes and imposes more substantial penalties for the incorrect classification of common classes. This strategy encourages the DQN agent to focus on underrepresented categories during the training process, thereby reducing bias towards majority classes. Quantitative assessment metrics such as Accuracy, Precision, F1-score, Specificity, and Sensitivity were used to evaluate the model. The results showed an accuracy of 97.97 %, sensitivity of 97.74 %, precision of 97.81 %, F1-Score of 97.70 %, and specificity of 97.83 % on a non-augmented dataset of HAM10000. Finally, the model performance was compared to that of already existing research work, and it had an upper hand with considerable differences over the existing ones.