Pandas Map Function Returning 'nan'

Relevant DataFrame: http://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.data I have manually added a 'sex' column onto the DataFrame, and I am trying to replace 'M

Solution 1:

@ayhan is correct, the white space is causing the problem. A more proper fix to that could be to add skipinitialspace which is set to False by default as you're reading the data with read_csv.

df = pd.read_csv(io.StringIO(data), delimiter=',', skipinitialspace=True, header=None )
df[9] = df[9].map({'Male': 0, 'Female': 1})

Will give us (column 9 being the "gender" column):

   0                 1       2          3   4                   5   \
0  39         State-gov   77516  Bachelors  13       Never-married   
1  50  Self-emp-not-inc   83311  Bachelors  13  Married-civ-spouse   
2  38           Private  215646    HS-grad   9            Divorced   

                  6              7      8   9     10  11  12             13  \
0       Adm-clerical  Not-in-family  White   0  2174   0  40  United-States   
1    Exec-managerial        Husband  White   0     0   0  13  United-States   
2  Handlers-cleaners  Not-in-family  White   0     0   0  40  United-States   
      14  
0  <=50K  
1  <=50K  
2  <=50K  

Solution 2:

Similar to what @Leb has mentioned, this could also happen with pandas read_table. By default skipinitialspace is set to False in read_table as well. Hence, using skipinitialspace=True, will let you solve this problem when using read_table.

df = pd.read_table('smsspamcollection/SMSSpamCollection','\t',names = ['label', 'sms_message'],skipinitialspace=True)

Solution 3:

d1 = {'UK': 0, 'USA': 1, 'N': 2}
df['Nationality'] = df['Nationality'].map(d1)
d2 = {'YES': 1, 'NO': 0}
df['Go'] = df['Go'].map(d2)

Decision Tree practice from W3Schools... That originally didn't work for me until I restarted my kernels and ran all. Give it another shot.

Solution 4:

I suggest to try run

df['success'] = df['success'].str.replace(r'[\xa0]',"")  

before map function.