The uniq utility can be used to filter adjacent duplicate lines in files or in the text provided through standard input.
uniq
sort | uniq
sort -u
See man uniq and IEEE1003.1-2004-uniq for details.
man uniq