Script to find duplicate files and then rename them

troy_dowler

!/bin/bash

set the directory to search for duplicate files

directory="/path/to/directory"

find all duplicate files in the directory and save their paths to an array

duplicates=($(find "$directory" -type f -exec md5sum {} \; | sort -k 1,32 | uniq -w 32 -d --check-chars=32 | awk '{print $2}'))

iterate through the array of duplicate file paths

for file_path in "${duplicates[@]}"; do
# get the file extension
extension="${file_path##.}"
# get the basename of the file (without extension)
basename=$(basename "${file_path%.}")
# generate a new filename by appending a unique identifier to the basename
new_filename="$basename-$(date +%s).$extension"
# rename the file
mv "$file_path" "$directory/$new_filename"
echo "Renamed $file_path to $directory/$new_filename"
done

Here's how the script works:

The script sets the directory to search for duplicate files.
It uses the find command to find all files in the directory and calculate their md5sums.
It sorts the md5sums and finds the duplicates.
It saves the paths of the duplicate files to an array.
It iterates through the array of duplicate file paths and renames each file by appending a unique identifier to its basename.
It prints a message for each renamed file.
Note: The script assumes that you have the md5sum command installed on your system. If you're on a macOS, replace md5sum with md5.

coop

DO NOT USE!!

Please do not publicize a script that can be extremely destructive if used in certain ways you may not have tested. It removes or renames things which can really mess things up if not planned. I almost deleted this post, but I'll leave it for now.

If you merely want to find duplicate files there is an existing standard utility called hardlink which is extremely powerful and very configurable. It can match on all file properties, just size, modification time, name etc, and with "-n" can be run as a dry run, which always should be the case on any program that can alter things like this.

Just do hardlink --help to get what options are.