Situation: You have a lot of results in a file that you need to move through and single out and add or subtract to fields, compare entries, or many other needs
Complication: There are a TON of Linux tools to do this
Question: What can do what?
Answer: I will add to this post with specific examples consistently
Example 1: Search through a file list and return only the filename, not the full path
Let’s say you want to just look for missing files in a non-similar directory structure between multiple file lists. One way is to look through each file without the file path and get just the file name. Let’s say filelist1 has this list:
/tmp/derp/directoryone/file1 /tmp/derp/directoryone/file2 /tmp/derp/directoryone/file3 /tmp/derp/directoryone/file4
Let’s now say filelist2 has this list:
/tmp/der/directorytwo/file1 /tmp/der/directorytwo/file2 /tmp/der/directorytwo/file3
In this case you want to just figure out that you’re missing file 4, so that you could sync just that file and not all the contents of directoryone. So to do this you could use cat to show the file contents, and pipe it to cut to get just the filename
less filelist1 | cut -d/ -f4
The -d/ option is saying to use the / as the delimiter, and the -f4 is saying to return the 4th entry
This would return
file1 file2 file3 file4
You could then send it to a new file that had just the filenames with >>
less filelist1 | cut -d/ -f4 >> filelist1_clean less filelist2 | cut -d/ -f4 >> filelist2_clean
You can now easily find just the files missing by running
diff filelist1_clean filelist2_clean
You would then get
file4
Now you can make your sync off of this
