InvalidIndexError: Reindexing Only Valid With Uniquely Valued Index Objects

Using Python, I am struggling to merge 208 CSV files into one dataframe. (My files names are Customer_1.csv, Customer_2.csv,,, and Customer_208.csv) Following are my codes, %matplo

Solution 1:

Your code works on a small sample of five files that I used for testing (each file containing two columns and three row). ONLY FOR DEBUGGING, try to write this in a for loop. First, before the loop, read all of the files into the list. Then loop again and append each one using a try/except block to catch the errors. Finally, print the problem files and investigate.

# First, read all the files into a list.
files_in = [pd.read_csv('data_TVV1/Customer_{0}.csv'.format(i), 
                        names = ['Time', 'Energy_{0}'.format(i)], 
                        parse_dates=['Time'], 
                        index_col=['Time'], 
                        skiprows=1) 
            for i in range(1, 209)]

df = pd.DataFrame()
errors = []

# Try to append each file to the dataframe.
for i i range(1, 209):
    try:
        df = pd.concat([df, files_in[i - 1]], axis=1)
    except:
        errors.append(i)

# Print files containing errors.
for error in errors:
    print(files_in[error])