Most automated tools for skin diagnosis look at a photograph and stop there. Symptom data — how long the rash has been present, whether it itches, where on the body it appears — is routinely discarded, even though no practising dermatologist would ignore it. On top of that, AI-generated outputs typically reach patients without any medical review at all. This paper describes a system built to correct both problems. Patients upload a lesion photograph and fill out a twenty-item symptom questionnaire. The image is processed by an EfficientNet-B3 convolutional network [16], which produces a 512-dimensional feature vector. A Restricted Boltzmann Machine condenses the symptom responses into a 100-dimensional latent vector. The two feature vectors are fused into a 612-dimensional representation and passed to a softmax classifier. A dermatologist, after consulting the Doctor Portal, is the only one able to share the output with the patient — this step cannot be skipped. The platform was built using React for the front-end, Flask for the server, and MongoDB for data storage. The system authenticates and encrypts data using JWT tokens, bcrypt, and AES-256. Our CNN+RBM model outperforms four baseline models — CNN-only, Matrix Factorisation (MF), SVD, and Weighted SVD (WSVD) — across every metric measured. By epoch 14, the model achieves 72.58% accuracy, 74.1% precision, 71.8% recall, and a 72.9% F1 score, with a false positive rate of only 4.2%.