Setting up the Kaggle API on your Paperspace machine

Sourced from: https://www.pexels.com/photo/battle-black-blur-board-game-260024/
Play is our brain’s favorite way of learning ~ Diane Ackerman

Hello there ?? Welcome to the way of the machine ? What is Kaggle? It’s an incredible platform that lets you practice your machine learning skills via competitions ? Although I’m just getting started, it’s highly regarded as one of the best ways to accelerate your skills in the machine learning community ??

Where does Paperspace come into this? For those without their own heavy-duty hardware, Paperspace allows you to rent the computing power you need for machine learning ? It’s the ideal way for students taking the fast.ai courses to set up a machine learning environment.

This guide shows you how to get the Kaggle API working on a Paperspace machine ? that’s been set up using the fast.ai template ? All text in the guide that’s in italics is a command for you to run on your Paperspace machine terminal e.g. cd .kaggle

Prerequisites

For this to work, you’ll need to:

  • Setup Paperspace using this awesome guide 
  • Sign up for a Kaggle account
  • Verify your Kaggle account: On your profile, click account then use the phone verification option

Kaggle API setup

Once the above steps are done, we can begin ? You’ll need to:

  1. Get the kaggle API package by running: pip install kaggle this will install kaggle to /home/paperspace/.local/bin/kaggle
  2. Get your API credentials from kaggle:
    1. Go to the account section of your kaggle account, your web address will look like: https://www.kaggle.com/<username>/account when you’re on the right page.
    2. Click the Create New API token button which will activate the download of a file called kaggle.json (this file has your API credentials)
    3. Open the kaggle.json file you just downloaded in a text editor, select all the text and copy it
  3. On your Paperspace machine, go to /home/paperspace by running cd ~
  4. Create a .kaggle folder by running: mkdir .kaggle 
  5. Move into the folder by running: cd .kaggle 
  6. With the credentials you copied earlier (step 2, part 3 above), replacye “your_username” and “your_key” with your details and run the command on terminal: ‘{“username”:”your_username”,”key”:”your_key”}’ >> kaggle.json e.g. ‘{“username”:”ml_wizard123″,”key”:”ab43ask563aec”}’ >> kaggle.json
  7. Set the kaggle environment variable by running: export KAGGLE_CONFIG_DIR=/home/paperspace/.kaggle/ 
  8. You may get the following error “Your Kaggle API key is readable by other users on this system!”. To fix this and secure your credentials on your system, run: chmod 600 /home/paperspace/.kaggle/kaggle.json

The Kaggle API should now be setup ? Take a short break e.g. grab a coffee ☕️, dance ?? to celebrate ?

Kaggle API examples

To download data from a specific competition e.g. Blue Book for Bulldozers, you will need to:

  1. Go to the competition page
  2. Click the “join competition” option
  3. Accept the competition rules

You can now use the API commands to interact with the Kaggle platform ? For example, if I wanted to work with the Blue Book for Bulldozers competition, I could:

  • Search for the specific competition reference by running: kaggle competitions list -s bluebook, this gives the reference bluebook-for-bulldozers  
  • Find the list of data files for the blue book competition by running: kaggle competitions files -c bluebook-for-bulldozers 
  • Download the Train.zip file by running: kaggle competitions download bluebook-for-bulldozers -f Train.zip 

Further resources


Here are some links I used to make this guide:

We have now set up our competition environment ? This means we can now get the all-important practice in our machine learning wizard journey ??‍♂️ Thanks for reading ?

Bookmark the permalink.

4 Comments

  1. Long time reader, first time commenter — so, thought I’d drop a comment..
    — and at the same time ask for a favor.

    Your wordpress site is very simplistic – hope you don’t mind me asking what theme you’re using?
    (and don’t mind if I steal it? :P)

    I just launched my small businesses site –also built in wordpress like yours–
    but the theme slows (!) the site down quite a bit.

    In case you have a minute, you can find it by searching for
    “royal cbd” on Google (would appreciate any feedback)

    Keep up the good work– and take care of yourself during
    the coronavirus scare!

    ~Justin

  2. I was able to find good information from your articles.

Leave a Reply

Your email address will not be published.