Blog posts on Data Science, Machine Learning, Data Mining, Artificial Intelligence, Spark Machine Learning

Saturday, February 23, 2019

How to import data into Google Colab Jupyter Noteook

Accesing data is one of the first step that we need when performing any data analysis. In this tutorial, we will see two ways of loading data into the google colab environment.


  • Uploading csv from local machine and loading into colab
  • Loading data from google drive to colab

  • Uploading CSV from local machine using IMPORT functionality.



  • Load import files library from google colab
  • upload file using the upload button control

  • Running below commands will allow us to upload data files into the colab environment. Once the Choose Files button is visible, after executing the below listed python commands, we can easily upload files from local directory.
    from google.colab import files
    uploaded = files.upload()
    
    Saving DOLPHIN.csv to DOLPHIN.csv

    To view the uploaded files

    Below command allows us to verify if the file is uploaded correctly.
    for fn in uploaded.keys():
      print('User uploaded file "{name}" with length {length} bytes'.format(name=fn, length=len(uploaded[fn])))
    
    User uploaded file "DOLPHIN.csv" with length 117269 bytes



    Reading the uploaded from into pandas dataframe and displaying results

    After the data file is uploaded to the colab, we can use pandas functions to load data into python environment and continue our further analysis.
    import pandas as pd
    import io
    df = pd.read_csv(io.StringIO(uploaded['DOLPHIN.csv'].decode('utf-8')))
    
    print(df.head(2))
    
    Load data from google drive:

    Sometimes we may require to load data from google drive. Below commands will be useful in reading data from google drive. Here we assume that the data file to be loaded into python environemnt is already uploaded to Google Drive.



  • Step 1, we need to mount the google drive
  • Step 2, After mounting we need to provide authorization
  • Step 3, we can view the current list of files available at the mounted location
  • Step 4, Load data using pandas read_csv function

  • Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&scope=email%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdocs.test%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive.photos.readonly%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fpeopleapi.readonly&response_type=code
    Enter your authorization code:
    ··········
    Mounted at /content/gdrive


    Note: My data files are located at MY DRIVE\COLAB NOTEBOOKS folder of my google drive account. Please change the code accordingly
    After clicking the link and entering the authorization code, you can access your drive as follows:

    !ls -la /content/gdrive/My\ Drive/Colab\ Notebooks/
    
    total 230
    -rw------- 1 root root 117269 Feb 23 06:43 DOLPHINOFFALLN.csv
    -rw------- 1 root root 12104 Feb 23 06:46 ImportDatatoColab.ipynb
    -rw------- 1 root root 8935 Sep 8 17:23 'Running first neural network model on google colaboratory'
    -rw------- 1 root root 13498 Aug 25 17:19 SettingupDrive_GSPGC.ipynb
    -rw------- 1 root root 81691 Nov 26 06:47 'Upload data to colab from google drive'

    df2 = pd.read_csv('/content/gdrive/My Drive/Colab Notebooks/DOLPHINOFFALLN.csv')
    
    print(df2.head(2))
    
    
    
    TIP: We can Install new libraries in python environment inline using below command
    Note that appending ! before pip command
    !pip install matplotlib

    Requirement already satisfied: matplotlib in /usr/local/lib/python3.6/dist-packages (3.0.2)
    Requirement already satisfied: numpy>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (1.14.6)
    Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (0.10.0)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (2.3.1)
    Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (1.0.1)
    Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib) (2.5.3)
    Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from cycler>=0.10->matplotlib) (1.11.0)
    Requirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from kiwisolver>=1.0.1->matplotlib) (40.8.0)


    References:



  • https://www.kdnuggets.com/2019/01/more-google-colab-environment-management-tips.html
  • https://www.kdnuggets.com/2018/02/essential-google-colaboratory-tips-tricks.html

  • Full code implementation:

    20 comments:

    1. I like the valuable information you provide in your articles. I will bookmark your blog and check again here frequently. I am quite sure I’ll learn many new stuff right here! Best of luck for the next! sem ppc

      ReplyDelete
    2. I don’t even know how I ended up here, but I thought this post was great. I don't know who you are but certainly you are going to a famous blogger if you are not already ;) Cheers! business loan singapore

      ReplyDelete
    3. I was suggested this website by my cousin. I am not sure whether this post is written by him as nobody else know such detailed about my problem. You are incredible! Thanks! online marketing campaign

      ReplyDelete
    4. What i don't realize is actually how you're not really much more smartly-favored than you might be now. You are so intelligent. You realize therefore considerably in terms of this topic, produced me in my view imagine it from numerous various angles. Its like women and men aren't involved except it is something to accomplish with Woman gaga! Your individual stuffs excellent. All the time care for it up! marketing company singapore

      ReplyDelete
    5. I am extremely impressed with your writing skills as well as with the layout on your weblog. Is this a paid theme or did you customize it yourself? Anyway keep up the nice quality writing, it’s rare to see a great blog like this one today.. UV Adhesive for Glass to glass stick

      ReplyDelete
    6. Can I simply say what a reduction to seek out someone who really is aware of what theyre talking about on the internet. You definitely know methods to deliver a difficulty to mild and make it important. Extra folks have to read this and understand this side of the story. I cant imagine youre not more standard because you positively have the gift.
      ecommerce singapore

      ReplyDelete
    7. I am really happy to read through these details. You will have to find out the right path to connect to your potential customers and internet. Most of the new business owners like me are going with the Adwords Marketing and social media platforms because they are great ways to get instant results.

      ReplyDelete
    8. Hello I am so delighted I found your webpage, I really found you by error, while I was looking on Aol for something else, Anyways I am here now and would just like to say thank you for a fantastic post and a all round exciting blog (I also love the theme/design), I don’t have time to read through it all at the moment but I have bookmarked it and also included your RSS feeds, so when I have time I will be back to read much more, Please do keep up the awesome work. marketing short courses in singapore

      ReplyDelete
    9. I'm not sure exactly why but this web site is loading very slow for me. Is anyone else having this problem or is it a problem on my end? I'll check back later and see if the problem still exists.
      Marketing specialists

      ReplyDelete
    10. I’m no longer sure the place you're getting your information, however great topic. I needs to spend a while learning more or understanding more. Thanks for wonderful info I used to be on the lookout for this info for my mission.
      The Sneakers Agency

      ReplyDelete
    11. Very well written story. It will be beneficial to anyone who usess it, as well as yours truly :). Keep doing what you are doing - i will definitely read more posts.seo agency website

      ReplyDelete
    12. Howdy this is kind of of off topic but I was wondering if blogs use WYSIWYG editors or if you have to manually code with HTML. I'm starting a blog soon but have no coding knowledge so I wanted to get advice from someone with experience. Any help would be enormously appreciated! coding for kids

      ReplyDelete
    13. ese exercises will make coordinated effort and be sharing a genuine action that licenses Data researchers group to expand on each one expertise and to create the entire best outcome. data science course in pune

      ReplyDelete
    14. Well, The information which you posted here is very helpful & it is very useful for the needy like me.., Wonderful information you posted here. Thank you so much for helping me out to find the Data analytics course in Mumbai Organisations and introducing reputed stalwarts in the industry dealing with data analyzing & assorting it in a structured and precise manner. Keep up the good work. Looking forward to view more from you.

      ReplyDelete

    15. Attend The Digital Marketing Courses in Bangalore From ExcelR. Practical Digital Marketing Courses in Bangalore Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Digital Marketing Courses in Bangalore.
      Digital Marketing training in Bangalore

      ReplyDelete
    16. Attend The Data Analytics Courses From ExcelR. Practical Data Analytics Courses Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Data Analytics Courses.
      ExcelR Data Analytics Courses

      ReplyDelete
    17. Such a very useful article. I have learn some new information.thanks for sharing.
      data scientist course in mumbai

      ReplyDelete
    18. Thank you so much for helping me out to find the Data science course in mumbaiOrganisations and introducing reputed stalwarts in the industry dealing with data analyzing & assorting it in a structured and precise manner. Keep up the good work. Looking forward to view more from you.

      ReplyDelete
    19. Thanks for sharing this info,it is very helpful.
      data science

      ReplyDelete

    20. Excelr is providing emerging & trending technology training, such as for data science, Machine learning, Artificial Intelligence, AWS, Tableau, Digital Marketing. Excelr is standing as a leader in providing quality training on top demanding technologies in 2019. Excelr`s versatile training is making a huge difference all across the globe. Enable ?business analytics? skills in you, and the trainers who were delivering training on these are industry stalwarts. Get certification on "
      best institute for data science in hyderabad"
      and get trained with Excelr.

      ReplyDelete