Remove Duplicate Lines

Remove duplicate lines from any list online. Keep the first occurrence in the original order, with options to ignore case, trim spaces and sort.

Input

Ignore case Trim whitespace before comparing Remove empty lines Sort alphabetically

Output

How to remove duplicate lines

Paste your list or text into the input box, with one item per line.
Choose the options you need: ignore case, trim whitespace before comparing, remove empty lines, or sort the result alphabetically.
Read the cleaned list in the result area, along with the number of lines that were removed.
Copy the result and paste it back into your spreadsheet, document or code.

How the duplicate remover works

The tool reads your text line by line. The first time a line appears, it is kept. Every later line that matches one already seen is dropped. The lines that remain stay in their original order, so the structure of your list is preserved.

For example, this list:

apple
banana
apple
cherry
banana

becomes:

apple
banana
cherry

and the tool reports that 2 lines were removed.

The options explained

Ignore case: treats “Apple”, “APPLE” and “apple” as the same line. The first version encountered is the one that is kept, with its original capitalization.
Trim whitespace before comparing: ignores spaces and tabs at the start and end of each line when looking for duplicates. This catches duplicates that only differ by a stray space, which is very common in data copied from spreadsheets or web pages.
Remove empty lines: drops blank lines from the result entirely, so you get a compact list.
Sort alphabetically: sorts the deduplicated result from A to Z. Leave it off if the original order matters to you.

Without any option enabled, the comparison is exact: “Apple” and “apple” are different lines, and so are “apple” and “apple ” with a trailing space.

Common uses

Cleaning up email lists, keyword lists or lists of URLs before importing them into another tool.
Deduplicating product codes, IDs or tags copied from a spreadsheet column.
Merging two lists and removing the overlap: paste one after the other and remove the duplicates.
Tidying log files or configuration lists where the same entry was added several times.

Privacy

Everything runs in your browser. Your list is never uploaded to a server, which makes this tool safe to use with customer email addresses, internal IDs or any other data you don’t want to share.

Frequently asked questions

Which occurrence of a duplicate line is kept?

The first one. The tool keeps the first time each line appears and removes every later copy, so the order of your list stays the same.

Can I remove duplicates regardless of capitalization?

Yes. Turn on the “ignore case” option and lines that only differ by uppercase and lowercase letters will be treated as duplicates.

Why are some duplicates not detected?

Usually because the lines aren’t exactly identical. Invisible differences such as a trailing space, a tab or different capitalization make two lines distinct. Turn on “trim whitespace” and “ignore case” to catch these.

Can I sort the list at the same time?

Yes. Enable the sort option to get the deduplicated lines in alphabetical order. If you leave it off, the original order is kept.

Is there a limit on the number of lines?

There’s no fixed limit. Because processing happens on your device, very large lists depend on your browser and computer, but lists of many thousands of lines are handled comfortably on most machines.

Is my data uploaded anywhere?

No. The lines are compared locally in your browser, and nothing is sent to a server.