Remove duplicate lines

awk '!x[$0]++' file.txt > file_clean.txt
index
You are here quickies:bash:files