How To Upload A Folder To Colab

In this tip we will see how to upload files from a public drive to Google Colab, without access key and westithout connecting to your drive.

When nosotros do Machine Learning on Jupyter Notebook or Google Colab it'southward usual to upload files on our session. 1 can import them locally when working alone simply as presently as 1 works with several people, the question of loading the data arises.

These data tin be very large especially when it comes to pre-trained models or checkpoints.

Many people store these files on Google Drive or other storage platforms. This saves time and infinite on their local drive.

Python gives us the possibility to upload these kind of files stored on the cloud directly on our session whether they are private, within a squad, or public, attainable to all.

No need to download templates yourself, Python does it by itself !

We will see in this article how to do information technology !

[smartslider3 slider="9″]

Photograph by Michael Niessl on Unsplash

Upload files from your Google Bulldoze binder

Google Colab has a library that allows yous to import your own Drive :

          from google.colab import bulldoze drive.mount('/content/bulldoze')

After executing the code, Python gives us a link to find our access cardinal.

Nosotros just take to enter the key in the corresponding field to have admission to all the files of our Drive on our session. Field should look like this :

This is an like shooting fish in a barrel solution if you lot want to admission the files on your bulldoze, simply what about the files in a public directory that you want to admission without having the access key ?

This can exist a problem, peculiarly if nosotros piece of work in collaboration with others who don't have access to our bulldoze'southward countersign or if we want to make public our freshly baked Machine Learning algorithm !

Don't worry, there is a solution to upload public drive files ! And it fifty-fifty works for big files !

Upload files from a public Google Drive binder

Google Colab allows to utilize Beat out commands like pip, ls or wget… it'due south the last one nosotros are interested in 😉

We are going to employ wget to load the file from a public directory of Google Drive that we want but earlier that nosotros demand to get the iD of this file.

For that, nosotros need to become to the Drive link where the file is located for instance this 1.

Then make a correct click on the file we are interested in and click on Get link, hither :

Once done you should get a link like this: 'https://drive.google.com/file/d/1ML-Rpeaftox3z7b1GThUZtl1Ql_lGUrb/view?usp=sharing'

The iD you need to retrieve is between 'https://drive.google.com/file/d/' and '/view?usp=sharing'.

In our case the iD is : 1ML-Rpeaftox3z7b1GThUZtl1Ql_lGUrb

And so, depending on the size of the file, nosotros accept two options:

With pocket-size files

Utilise the command :

          !wget -q --show-progress --no-check-document 'https://docs.google.com/uc?consign=download&id=iD' -O Nom_fichier_sortie

With :

iD : the iD of the file
Output_file_name : the name nosotros desire to give the file we downloaded with the associated extension (e.g.: .txt, .png, .pdf, .zip, .tar, … )

In our example:

          !wget q --prove-progress --no-cheque-document 'https://docs.google.com/uc?export=download&id=1ML-Rpeaftox3z7b1GThUZtl1Ql_lGUrb' -O fichier.tar

With large files

Use the command :

          !wget -q --show-progress --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?consign=download&confirm=$(wget --repose --salve-cookies /tmp/cookies.txt --keep-session-cookies --no-cheque-document 'https://docs.google.com/uc?consign=download&id=iD' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\north/p')&id=iD" -O Nom_fichier_sortie && rm -rf /tmp/cookies.txt

With :

iD: the iD of the file
Output_file_name : the proper noun we want to give the file we downloaded with the associated extension (e.g.: .txt, .png, .pdf, .zip, .tar, … )

In our case :

          !wget -q --show-progress --load-cookies /tmp/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies /tmp/cookies.txt --go on-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=1ML-Rpeaftox3z7b1GThUZtl1Ql_lGUrb' -O- | sed -rn 's/.*ostend=([0-9A-Za-z_]+).*/\1\n/p')&id=1ML-Rpeaftox3z7b1GThUZtl1Ql_lGUrb" -O fichier.tar && rm -rf /tmp/cookies.txt

sources :