Upload Pandas Dataframe To Gcp Bucket For Dataproc

I have been working on Spark Cluster using Data Proc google cloud services for Machine Learning Modelling. I have been successful to load the data from the Google Storage bucket. H

Solution 1:

The easiest should be to convert Pandas DataFrame to Spark DataFrame and write it to GCS.

Here's instructions on how to do this: https://stackoverflow.com/a/45495969/3227693