How To Remove Entire Column If A Particular Row Has Duplicate Values In A Dataframe In Python

I have a dataframe like this, df, Name City 0 sri chennai 1 pedhci pune 2 bahra pune there is a duplicate in City column. I tried: df['City'

Solution 1:

You can use:

df2 = df.drop_duplicates(subset='City')

if you want to store the result in a new dataframe, or:

df.drop_duplicates(subset='City',inplace=True)

if you want to update df.

This produces:

>>>df
      City    Name
0  chennai     sri
1     pune  pedhci
2     pune   bahra
>>>df.drop_duplicates(subset='City')
      City    Name
0  chennai     sri
1     pune  pedhci

This will thus only take duplicates for City into account, duplicates in Name are ignored.