How I organized 60,000 photos in less than 60 seconds

Jump Directly to the important stuff

Introduction

Thanks to the amazing smartphone cameras, we are constantly taking pictures of our kids, pets, or meals (or sometimes kids eating pet’s meal or vice versa). So much so that we are projected to take 1.4 trillion pictures in 2020 alone.

How many pictures did you click this week already? Do you think you will be able to find an image from two years ago if you ever needed to? All this information needs to be organized in order to be useful. Whether you are taking pictures of your family for memories, or a subject for your blog or business, it is important to organize your pictures. Without a discipline to manage photos, you will just end up with digital clutter. Instead of saving memories that you can revisit in a few years, you are just collecting this noise. That is why I did a lot of research and found this free method of auto organizing photos and sorting all my images according to date and the device that was used to capture it. You do not need any coding experience to use this and it does not cost a dime.

How do I have so many images?

Currently, I have a smartphone, an old DSLR, and a GoPro. As expected, I use my smartphone to capture most of my photos when with friends and family. I have an old Canon which I am learning to use and is mostly reserved for taking food photos that I post here with recipes or on my Instagram page. I use GoPro mainly for travel and I used it quite extensively on my 2016 Road Trip across the United States. I love taking time-lapses of beautiful open roads in the US.

Combine all these photos from going back to 2005, I ended up with roughly 60,0000 image files. Mainly time-lapses are to be blamed for such a large number.

Problem with so many unsorted images

My image archive is something that I have neglected for years. I bought a 1T external drive almost a decade ago but I never really formed a good backup routine or strategy for my images. Every now and then, when either my phone, camera, or GoPro yelled at me because of low memory, I would dump the images onto my external drive. End result was a bunch of folders randomly named “2019_January_Canon”, “2017_Hawaii_Trip_GoPro”, “Phone_image_backup_2014”. You can see how terrible that naming “scheme” is, with no real pattern and not much useful information. If this is something you see yourself doing, I would love to hear from you in the comments.

With a lack of proper filing system for photos:

• Images are not really searchable without a proper naming scheme
• It is difficult to go back and retrieve a picture from a particular event
• Causes decision fatigue if you ever decide to sort or backup a few of your images
• Excessive time wastage in locating any particular photos

Not rely on free cloud services like google photos

Some people may argue that they use “free” services likes Google Photos to back up their photos so they do not really need a good filing system of their own. While Google Photos does a great job of storing the images and cataloging them using facial recognition, I am not 100% sold on it as my only photo filing system because of the following reasons:

1. Good Photos is not Free anymore: Starting June 1st, 2021, Google photos will not be a free service like they used to be. The new images and videos uploaded to Google cloud after June 1st will get counted towards the 15Gb storage that Google users get split between Gmail, Google Drive, etc. Any additional storage can be purchased and it starts at $1.99 / month for 100GB and has tiers going through 200GB ($2.99 / month), 2TB ($9.99 / month), and all the way up to 30TB ($149.99 / month).
2. Google Photos was NEVER really FREE: You are paying google with your private information and agreeing to potentially be used to show you targeted ads based on your location of images, maybe even the type of clothing you wear? (I am not making any claims, just my speculation and not backed by any research)
3. Only backup up your phone: Cloud-based services are definitely convenient if you get over the privacy issue but in my opinion, they are only convenient if you backup photos from your phone which happens automatically in the background. If you take a lot of pictures using your cameras, you have to remember to regularly connect them to your computer and back up to the cloud.
4. Loss of resolution: Google’s service is only free if you scale down your image resolution which is not acceptable if you are taking pictures and videos for your blogs or businesses.

Sorting format (Casey Neistat File Organizing System)

Due to the reasons listed above, I choose to use a combination of using my own hard drive and Google Photos for backing up my photos and videos. What I mean by a combination is I back up my phone, DSLR, and GoPro on my personal backup drive and only use Google for some photos that I take with friends because of the ease of sharing later.

In the last few months, as I am taking more and more photos and videos for my project (Across Kitchen Lines) and Instagram, I found myself spending too much time looking for images that I even took just a few weeks back. I decided to spend some time cleaning up my act and come up with a system. As a part of my research, I started looking at how some of the professional photographers and video creators storing. It has always amazed me how Casey Neistat, a popular YouTuber, is able to add clips from the footage that he shot several years ago in his daily vlogs. I googled “how to organize footage like Casey Neistat” and I was in luck as he long back he had done a video about that!

So this is how it looks like:

I noticed a few other popular photographers and content creators (including Peter McKinnon) using a similar system and without pondering too much over it, I decided to go with this organization system. I like it because:

1. Objective: Year and date of the shoot are very standard to be used as a name as opposed to naming individual folders with the purpose or location of the shoot. For example, If I took some photos in December of 2018 on a Farm to Table experience in Hawaii when I was in Hawaii for a wedding. Should I save those photos in a folder called 2018_Hawaii_Trip, Best_Farm_Hawaii, John_Wedding_Trip? All these options make sense, hence making the decision of picking a name difficult. Instead, if you call the folder 2018/December/19/iPhone8, it is very standard every time you backup files. Do not need to spend any additional time trying to think of what name to pick for the folder.
2. Easy to recall: Imagine trying to get find the above-mentioned files 10 years from now. Will your brain be able to remember the folder name to look for? If you vaguely remember the year you took the trip, it will be easier to track the images that way.
3. Easier to automate: When I was looking for a system, I was not looking for an image filing system for all my future photos. I was also looking for a system that I could apply to all my existing images so that all of my archives are organized in the same way. I sure was not planning to do this manually and organizing by Year/Month/Date/Device can be automated very easily. I will share how in a minute.

Available tools for the job

Every image file captured either with a smartphone or any camera has a lot more information stored than we think in the form of Exif data. You can read more about Exif(Exchangeable Image file format) here. This information includes the date and time of capture, device, GPS location if available, and a lot more.

There are a lot of tools(mostly paid) on the market that harness the above information and make image organization easier for you. Some of them being:

1. PhotoMove2.5
2. Adobe Lightroom
3. Apple Photos
4. DigiKam
5. PhotoMechanic

I do not use the above tools because:

While most of the above-mentioned tools are great options, I decided to not use them because they were either paid, platform-dependent (mac or windows only), did image organization inside the tool only, and not in the actual file system. For me cataloging inside the software is not the best option because it not really sorting in my opinion as you are getting this view only inside that tool. (Tool dependency makes me anxious.)

Tool I use

To sort all 60,000 images and videos on my hard drive, I used an application called ExifTool. I came across ExifTool on a Reddit forum. It is an open-source tool created by nuclear physicist Phil Harvey and has been around for more than a decade.

“ExifTool is a platform-independent Perl library plus a command-line application for reading, writing and editing meta information in a wide variety of files.” ExifTool is also available as a stand-alone Windows executable and a MacOS package.”

How I use my script(on mac)

Install ExifTool by following the instructions on this page. Open terminal and change directory to your destination drive using “cd” and use the command below:

exiftool -r -P -o . '-FileName<$FileModifyDate/${FileModifyDate#;DateFmt("%Y-%m-%d_%H%M%S")}_%f%-c.%e' '-FileName<$DateTimeOriginal/${DateTimeOriginal#;DateFmt("%Y-%m-%d_%H%M%S")}_%f%-c.%e'  '-FileName<$FileModifyDate/${model;}/${FileModifyDate#;DateFmt("%Y-%m-%d_%H%M%S")}_${model}_%f%-c.%e' '-FileName<$DateTimeOriginal/${model;}/${DateTimeOriginal#;DateFmt("%Y-%m-%d_%H%M%S")}_${model}_%f%-c.%e' -d '/Volumes/Documents/DestinationDrivePath/%Y/%m/%d' /Volumes/SourceDrivePath

Description of script

The script recursively goes through each subfolder of the source folder, reads each file, and copies the images to the destination folder. In the destination folder, it creates new subfolders based on dates the images were captured or modified. Under a folder named by date, subfolders for the different devices are also created if there are images captured using different cameras or phones on a given date.

Below I have broken down the script(from right to left) into sections with a brief description of that section:

/Volumes/SourceDrivePath – Path of the source folder with all the unorganized images and videos.

-d ‘/Volumes/Documents/DestinationDrivePath/%Y/%m/%d’ – Path of the destination folder along with the date format to use for the folder names after sorting.

Following 4 are the options for renaming the files based on available information in the EXIF data in the image. The way it works is, if the Original Date and Time and Camera Model are available, it uses option 1 for naming the image and the file name will look something like this “2020-12-27_102952_Canon EOS DIGITAL REBEL XT_IMG_8137.CR2” else it will use option 2 and name the file using the Last Modified Date and Time and so on.

Date_Time_Camera Model_Original File Name-number if a duplicate file exists.file extension

1. ‘-FileName<$DateTimeOriginal/${model;}/${DateTimeOriginal#;DateFmt(“%Y-%m-%d_%H%M%S”)}${model}%f%-c.%e’
2. ‘-FileName<$FileModifyDate/${model;}/${FileModifyDate#;DateFmt(“%Y-%m-%d_%H%M%S”)}${model}%f%-c.%e’
3. ‘-FileName<$DateTimeOriginal/${DateTimeOriginal#;DateFmt(“%Y-%m-%d_%H%M%S”)}_%f%-c.%e’
4. ‘-FileName<$FileModifyDate/${FileModifyDate#;DateFmt(“%Y-%m-%d_%H%M%S”)}_%f%-c.%e’

-o – Copy images from source to destination. Skip this if you want to Move instead of Copy. I personally always copy first and manually delete images from the source drive.

-P – This preserves the properties like creation date, edit date and last modified date of the copid or moved images.

-r – execute this command recursively on every file in the source folder and also in sub folders, sub sub folders and so on.

Workflow

Over the past few months, I have been trying to back up and catalog my photos and videos more regularly.

After every trip or taking photos and videos of the dish for my Instagram, I dump all of my photos in a “source” folder on my computer. I browse through all the photos and delete the ones that are obviously not usable(over-exposed, blurry, etc.). Then I run this script on the remaining photos that have some potential and back them up on my external drive.

I then import the photos and videos from the “source” folder into the photo editing software (i use Apple photos for mac). After editing a handful of photos, I export them and save these in folders that have more contextual names.

To summarize, all the usable raw images and videos are sorted and backed up using the script above and a handful of “chosen ones” are edited, exported, and saved into folders with contextual names. These images are then shared with family and/or posted online.

Closing words

I am sure I am not the only one who likes to have some control over their data. Using this small script, the process has become so easy that I actually use it on a regular basis. When I was not baking sourdough, looking for a good system was my obsession last year during the lockdown. I ran the script on my drive with ALL the pictures I have ever taken in the last decade and a half which was less than 60 seconds of active work on my part. It kept working in the background and this is how my final archive looked like:

If you find it helpful and plan to adopt it, I would love to hear from you. As far as organizing structure is concerned, there is no one size fits all and you may have another structure in mind that works for you. Exiftool is quite flexible and you can edit the script to create a structure of your choice. I hope this post is helpful in motivating you to sort through your digital clutter.