Lstm Project Not Compatible With Csv Format

I am trying to replicate Chevalier's LSTM Human Activity Recognition algorithm and came across a problem when I was trying to implement my own data in a CSV format. The format used

Solution 1:

The code in the trace differs from the code you actually posted in the question -- the working code is operating on a bare file handle, not a Pandas data frame.

For reference, here is the code from the project you are referring to again:

defload_X(X_signals_paths):
    X_signals = []

    for signal_type_path in X_signals_paths:
        file = open(signal_type_path, 'r')
        # ^ the error comes where you have file = pandas.read_csv(...)# Read dataset from disk, dealing with text files' syntax
        X_signals.append(
            [np.array(serie, dtype=np.float32) for serie in [
                row.replace('  ', ' ').strip().split(' ') for row in file
            ]]
        )
        file.close()

file is just an iterator which returns a raw line (a sequence of characters) ending with a newline; on this input, it makes sense to strip newlines and squeeze spaces. But your code already opens, parses, and reformats the contents of the file into a Pandas data frame, which doesn't have newlines or spaces, just the numbers already parsed. Maybe fall back to the upstream code; or if there is something you want to change in there, figure out how to ask about that change. There's nothing wrong with the CSV as such.

Python has a quite capable csv module so maybe simply use that instead of manually parsing out the individual fields from the CSV.

    for signal_type_path in X_signals_paths:
        with open(signal_type_path, 'r') as csvfile:
            reader = csv.reader(csvfile)
            X_signals.append([np.array(row[0:2], dtype=np.float32) for row in reader])

Or as a minimal change, split on commas instead of spaces. (Your data looks like you don't actually need to remove spaces then.)

Also, tangentially, your code hardcodes the file it reads. It's probably better to keep the DATASET_PATH and TRAIN parameters entirely in the calling code, and have load_X simply accept a list of full file paths, which it accepts without modifying them in any way.