Unpacking A Pandas Dataframe Column That Contains A List Of Dictionaries Into New Columns

I have a dataframe new_df that has one column, which contains a list of dictionaries, with some rows NaN. new_df 0 0

Solution 1:

The comment made by @Ben.T made me think of what I was trying to accomplish.

I was creating a dataframe from a series that is a list of dictionaries. Why was I peeling off this new dataframe column by column, when I could apply the new dataframe to the existing dataframe on the column axis?

My solution:

# Creates df but removes the NaN elementsnew_df = pd.DataFrame(list(orig_df[0]).dropna())   

# Get the orig_df indexes of non-NaN rows to apply to the new dfnew_ndx = new_df.index[orig_df[0].notna()]

# Reset index and give new indexes that will line upnew_df = new_df.reset_index(drop=True)
new_df = new_df.set_index(new_ndx)

# Now apply the new_df to the orig_dforig_df= pd.concat([orig_df, new_df ], axis=1)

Is there maybe a more pythonic way to accomplish this...?