Big Difference Between Val-acc And Prediction Accuracy In Keras Neural Network

I have a dataset that I used for making NN model in Keras, i took 2000 rows from that dataset to have them as validation data, those 2000 rows should be added in .predict function.

Solution 1:

The training data you posted gives high validation accuracy, so I'm a bit confused as to where you get that 65% from, but in general when your model performs much better on training data than on unseen data, that means you're over fitting. This is a big and recurring problem in machine learning, and there is no method guaranteed to prevent this, but there are a couple of things you can try:

• regularizing the weights of your network, e.g. using l2 regularization
• using stochastic regularization techniques such as drop-out during training
• early stopping
• reducing model complexity (but you say you've already tried this)

Solution 2:

I will list the problems/recommendations that I see on your model.

1. What are you trying to predict? You are using sigmoid activation function in the last layer which seems it is a binary classification but in your loss fuction you used mse which seems strange. You can try binary_crossentropy instead of mse loss function for your model.
2. Your model seems suffer from overfitting so you can increase the prob. of Dropout and also add new Dropout between other hidden layers or you can remove one of the hidden layers because it seem your model is too complex.
3. You can change your neuron numbers in layers like a narrower => 64 -> 32 -> 16 -> 1 or try different NN architectures.
4. Try adam optimizer instead of sgd.
5. If you have 57849 sample you can use 47000 samples in training+validation and rest of will be your test set.
6. Don't use the same sets for your evaluation and validation. First split your data into train and test set. Then when you are fitting your model give validation_split_ratio then it will automatically give validation set from your training set.