Logo
Instagram Facebook Youtube Envelope
Free Membership

Comparison: Duplicate Photo Removal Applications

Comparison Overview

Software for Removing Duplicate Photos

Which is the BEST De-Duping Application?

Professional Photo Organizers and Photo Managers owe it to their clients to use the best application for any task, but unless you compare software carefully, how do you know which is best?

Duplicate photo removal is usually the most time-consuming part of organizing a digital photo collection, and using the best application to remove duplicate photos can make a big difference in project accuracy and efficiency.

Applications Tested

(and the abbreviations used in the charts)

  • Cisdem Duplicate Finder (Cisdem)
  • dupeGuru (dGuru)
  • Duplicate Cleaner Pro 5 (DCP5)
  • Duplicate Photos Fixer Pro (DFFP)
  • Excire Foto (Excire)
  • Mylio (Mylio)
  • PhotoSweeper (PSweeper)

I started testing Teekesselchen (plugin for Lightroom Classic), but I abandoned that testing because it was so difficult to assess its performance since duplicates are not grouped. However, its abilities are quite limited, and its score would have been very low.

The Overall Scores

Duplicate Cleaner Pro 5 (by DigitalVolcano) was the clear winner, with 61 points. 
Other sections of this web page include details about the testing.

Duplicate removal applications results
Duplicate removal score by category

You’ll find detailed scores for each category further down on this page.

Testing Overview

Professional Photo Organizers and Photo Managers who organize digital photo collections must be able to de-dupe photos accurately and efficiently.

For all duplicate removal applications, accuracy and efficiency are directly related to the Scan Criteria available, combined with the Marking Options.

Scan Criteria and Marking Options

Definition: Scan Criteria
All settings, options, preferences, file properties, or metadata fields an application can use to determine which photos will be matched and grouped as duplicates. In the charts, “Match” means it’s available in the Scan Criteria.

Definition: Marking Options
All options for marking duplicates after they have been identified, either manually or using an ‘Auto-Mark’ feature.
Scoring for most marking options requires that the item be directly visible in results.

The more Scan Criteria are available, the more closely duplicates can be matched. When duplicates can be very closely matched, the difference(s) between the duplicates is known before the duplicates are even displayed. When the differences are known, duplicates can be marked very accurately and efficiently—sometimes without the need to look at the photos in the results.

Digital collections often include photos gathered from multiple Sources. Photos from one source may have been edited. Photos from another Source might be compressed, and photos from other Sources may have different filenames, folder names, and added metadata.

The Scan Criteria and Marking Options tested are those that I know are useful.

I also excluded relatively minor and unique Scan Criteria and Marking Options, like Duplicate Cleaner Pro 5’s option to limit results to files in a specific file size range.

I’ve seen many comparisons of applications that praise the applications that can find the most duplicates in one scan, but it’s more accurate and efficient to use very specific Scan Criteria to generate as few results as possible with a known difference or differences between duplicates in each group.

I did not include image similarity or histogram options in this testing.

All the applications tested can identify duplicate photos by comparing image similarity, and while there may be differences, those differences are difficult to measure and compare. 

For Professional Photo Organizers, the Scan Criteria and Marking Options are what determine how accurately and efficiently duplicate photos can be matched and marked.

Scoring Details

No: 0 Points. Option not available
Yes: 0.5 Points. Partial or incomplete option available.
Yes: 1 Point. Option available.

If You Disagree or Find Mistakes

Comparing complex applications is not easy. If you think there are errors, I really want to know, so please contact me.  

Contact Meg

Scores by Category

EXIF DateTimeOriginal

(EXIF DTO)

The EXIF DateTimeOriginal field is by far the most reliable indicator of the date and time the shutter was pressed on a camera. For that reason, it is the only date field that deserves to be included in this testing.

All applications tested can match duplicates based on ‘date’, but the fields ‘Capture Date’, ‘Date Taken’ and ‘Timestamp’, do not represent EXIF DTO!

Of the applications tested, all but Cisdem will read EXIF DTO if a photo has one, but if a photo does not have a date in that field, applications will read EXIF Modified Date, or EXIF CreateDate (Date Digitized). If a photo has no EXIF date metadata, the applications will read a file/system date.

Downloading, exporting, copying, and moving photos often changes the file/system Date Created, so that date field and Date Modified should be used only as a last resort for comparing the dates of photos.

Once ‘Exact’ duplicates have been removed, identifying duplicates becomes more difficult. My next step is to compare duplicates with 99% similarity and the same filename. After that, scanning for duplicates with 99% similarity will generate sequential photos and burst mode photos taken within the same second along with true duplicates (with different filenames).

Duplicate Cleaner Pro 5 is the only duplicate removal application that allows one to include sub seconds in Scan Criteria. That one feature will limit result to duplicates only and eliminates the need to visually compare the photos in results. 

Duplicate Cleaner Pro 5 is the only application that offers EXIF DTO as a Scan Criteria. If a photo has no EXIF DTO, that field will be blank in results.

Duplicate Cleaner Pro 5 and dupeGuru are the only de-duping applications that include EXIF DTO in Marking Options.

In Duplicate Cleaner Pro 5, a ‘Date Taken’ can be added in results, which will add a date to the EXIF DTO field, but the time cannot be adjusted. It will be the time of day where you are when you add it, so 0.5 points were awarded for that.

dupeGuru prioritizes marking the duplicate without an Exif DateTimeOriginal, so it was assigned 0.5 points.

Exif datetimeoriginal score

Dimensions and File Size

Pixel dimensions and file size are very important factors in duplicate photo removal. 

Having both dimensions and file size available as Scan Criteria helps to control the results.

When duplicates have the same dimensions but different file sizes—which is often the case— it’s important to be able to mark results based on file size.

Aspect Ratio is helpful for identifying duplicates have both different dimensions and file sizes and/ or limiting the number of results displayed.

This chart uses the term ‘Exact’, as used in Duplicate Cleaner Pro 5. In PhotoSweeper, and Cisdem, it’s Duplicates. For dupeGuru, it’s 100% match. Excire Foto uses Very Strict, but it’s not really very strict. In Mylio, the DeDupe tool generates ‘Exact’ duplicate results.

There is a very small file size allowance for ‘Exact’ duplicates in Duplicate Cleaner Pro 5. I’m told it’s a few kB. That’s why I gave that application a half a point on one line.

But I like that tolerance! It’s too small to make any practical difference! The tolerance allows for ‘Exact’ duplicates to have different added metadata, which means more duplicates can be identified and removed earlier.

While Mylio claims to be able to merge the metadata of ‘Exact’ duplicates, I tested that and I could not get Mylio to identify any photos with different added metadata as duplicates with the Photo DeDupe tool, though I have been assured it’s possible.

Dimensions file size score

Filename

Large digital photos collections always include photos with the same filename that are not duplicates. That’s because cameras repeat filenames.

But filename is still an extremely useful Scan Criteria and Marking Option. If duplicates have the same filename and the same EXIF DTO to the sub second, they are almost guaranteed to be duplicates!

When marking duplicates for removal, it’s important to be able to mark based on the content of filenames, not just the length of filenames. 

For example, if many duplicate groups include photos with no EXIF DTO, and one in each duplicate group has a date in the filename, I will keep all duplicates with a date in the filename by marking duplicates with non-date text, like ‘IMG’, in the filename.

And never assume photos with sequential filenames are not duplicates! Often social media apps will create a new version of a photo for posting and assign the next filename to that edited version. 

As for filename similarity options, I don’t often find them to be helpful.

Duplicate Cleaner Pro 5 has a unique and very helpful feature: There is a Scan Criteria option to ignore the ‘Copy’ part of filenames (suffixes) that are added to filenames automatically by applications to prevent files in the same folder from having identical names.

That feature allows duplicates with names like ‘IMG_1234.jpg’ and ‘IMG_1234 – Copy.jpg’ and ‘IMG_1234 (1).jpg’ to be seen as having the same filename. Because of that, duplicates can be identified much more easily.

Filename score

Folder Name and Folder Path

A photo’s parent folder name and any other folder names in the folder path may include date information or meaningful information added by the client that the client wants preserved.

Because of that, it’s essential to be able to compare folder name and folder path when removing duplicates. Being able to mark duplicates based on length of folder name only has little value.

If photos are exported from Apple Photos or Google Photos using a method that preserves Album names,  those Album names will become folder names.

When files are gathered from multiple Sources, duplicate photos from different Sources may be in folders with the same parent folder name.

It’s not uncommon for a meaningful folder name to not be the parent folder name but be deeper in the folder path.

Scoring for this category was based on being able to see the folder name or folder path in results, and not just for one photo at a time in a metadata panel.

Folder name folder path score

Added Metadata

There are many fields that may contain metadata added by clients that might need to be preserved. For this testing, I chose to include only the three most commonly used fields:

  • Keywords
  • Star Ratings
  • Caption/Description (terminology depends on the application)

Duplicate Cleaner Pro 5 is the only application tested that can match duplicates based on having the same added metadata in a specific field, or no added metadata in a specific field.

Marking duplicates based on whether a photo does or does not have added metadata makes little sense to me!

Duplicate photos from different Sources often have the same added metadata, but sometime have different Keywords, Star Ratings, or Captions/Descriptions. So, being able to mark duplicates based only on the presence of metadata in a field makes no sense to me.

With Duplicate Cleaner Pro 5, I can scan for duplicates with the same metadata in various fields (or no metadata in a field), then I scan for duplicates with any metadata in various fields. 

Photos often have useless/unneeded metadata, like a Caption/Description ‘Olympus Digital Camera’ or the Keyword ‘Fall 2010’ for a photo with a EXIF DTO in October 2010. In such cases, it’s best to mark and remove the duplicate with added metadata.

Scoring for this category was based on being able to see the added metadata in results, and not just for one photo at a time in a metadata panel.

Some metadata can be edited in Duplicate Cleaner Pro 5. However, it’s done through the Windows Properties dialog.

Some Windows Properties, like Comment, do not correspond to IPTC metadata fields, so it’s important to know which you should or should not edit.

Added metadata scoring

File Extensions

It’s very helpful to be able to compare files based on file extension when removing duplicate photos. In my workflow, I de-dupe files with the same extensions before comparing files with different extensions.

Being able to compare RAW files with other image file extensions is especially important.

A problem with Duplicate Cleaner Pro 5 is that it treats ,JPEG and .JPG as different extensions. Same with .TIFF and .TIF. Case is not important. That’s why DCP5 got 0.5 points on that line.

My solution is to use ‘Find and Replace’ in Directory Opus to change .JPEG to .JPG, and .TIFF to .TIF.  For about 100,000 photos, that task takes only about 2 minutes!

Version 6 won’t have that problem!

Scan Criteria and Marking options in PhotoSweeper for file extensions only distinguish between RAW and non-RAW, which is why I gave PhotoSweeper 0.5 points for that item.

File extensions scoring

Other Factors

This is for items that did not fit into another category. 

Scan folders against each other or against a Master folder: Being able to to compare 2 or more folders against each other and/or against a Master folder greatly increases duplicate removal efficiency. 

Find duplicate folders: dupeGuru can identify duplicate folders, but only in Standard mode, not Picture Mode, which is why I assigned 0.5 points.

Use Regular Expressions to mark or filter results: Using even basic Regular Expressions can make marking duplicates for removal much more efficient.

Filter results: Mylio can filter results, but in a way that I don’t think is at all helpful for selecting and marking duplicates for removal, so I did not award any points.

Scan and Mark by Title and User Comment: These metadata fields were not included in the ‘Added metadata’ category but are important. User Comment (or Comment) is a Windows Property that people use like a Caption/Description field. I’ve had Mac clients with lots of Comments because they used to use a PC. I use Directory Opus to move comments to the Description field. No other application compares metadata in this field, so Comments a person took the time to add can be lost.  

Scan and Mark by GPS Coordinates: When duplicates are otherwise equal, it’s important to be able to mark and remove photos without GPS metadata. GPS / Location metadata is often stripped from shared photos.

Scan and Mark by Software: When photos have software listed in that field, it usually indicates they have been edited. That’s very useful information if you’re keeping only a client’s unedited photos. 

Scan and Mark by Camera Make and Model: This is useful for many reasons, most commonly when a collection includes photos taken of the same event by different people. 

Move marked duplicates and preserve folder structure: Duplicate removal is complicated, and it’s easy to make a mistake (like, “Oops, I forget to check Star Ratings in results”). I move results from every scan to a unique folder, and I preserve the folder structure in the destination. If I make a mistake, I can drag the folder of duplicates back to the collection and re-do the scan.

Work with files in an Apple Photos Library or an Adobe Lightroom Classic Catalog: PhotoSweeper got points for these unique features. 

Auto Mark by Quality: I consider this to be a culling feature, not a duplicate removal feature, but I decided to include it anyways. I use this feature in Excire Foto if a client wants any culling done, but usually only for bursts with 99% similarity that I identified in Duplicate Cleaner Pro 5.

Other factors scoring

If You Disagree or You Found Mistakes

Comparing complex applications is not easy. If you think there are errors, I really want to know, so please contact me.  

Contact Meg