Does it change the order of my lines?

No. The first occurrence of each line stays where it was and later copies are dropped. That is the difference from a sort-and-unique pipeline, which removes duplicates and reorders everything as a side effect.

Two lines look identical but both survived.

Almost certainly trailing whitespace, or a non-breaking space rather than an ordinary one. Switch on trimming, and if that does not resolve it, run the line through the Unicode inspector to see what is actually there.

Should I ignore case?

For email addresses, yes — the domain is case-insensitive and the local part is in practice. For identifiers, file paths and anything where capitalisation is meaningful, no.

Can it show me only the duplicates instead?

Not directly. Sorting the list first makes repeated lines adjacent and easy to see, which is the usual way to inspect them before deciding to remove anything.

Remove Duplicate Lines

Keep one of each line and see how many went.

Everything runs inside your browser. Your files never leave your device.

Input

Result

The result appears here as you type.

Ask about Remove Duplicate Lines

Questions about what this tool does, which option to pick, or what it can and cannot handle.

The question you type here is sent to an AI provider to be answered — your files and whatever you put in the tool above are not, and the assistant cannot see them. Answers are generated and can be wrong. The tool itself is not guessing: it runs deterministic code on your device.

About the Remove Duplicate Lines

Duplicated lines accumulate wherever lists get merged. Two exports of the same mailing list, a log filtered twice, a set of URLs collected from several pages. Finding them by eye is impossible past a few dozen lines.

This removes them and tells you how many went, which is often the more interesting number: it is the difference between "the merge worked" and "the merge ran twice".

Order is preserved by default, and that distinguishes this from the usual command-line approach. The classic sort-then-unique pipeline removes duplicates but also reorders everything, which destroys a chronology or a deliberate ranking. Here the first occurrence of each line stays where it was and later copies are dropped, so a log stays in time order and a priority list stays prioritised.

Case sensitivity is worth thinking about before you run it. Treating "Alice@example.com" and "alice@example.com" as the same line is usually right for email addresses, since the domain part is definitively case-insensitive. It is usually wrong for identifiers, file paths on a case-sensitive filesystem, or anything where the capitalisation carries meaning.

Trailing whitespace is the invisible reason two lines that look identical are not. A line ending in a space is a different string from one that does not, and neither is visible on screen. Trimming before comparison handles the common case where one export used a different convention from another.

For addresses and hostnames it is worth remembering the tool compares text and not meaning: two URLs that differ only by a trailing slash are two different lines here, because they are two different strings.

How to use it

1Paste the list you want cleaned.
2Decide whether case should matter — usually not for email addresses, usually yes for identifiers.
3Trim whitespace if the lines came from more than one source, since a trailing space makes two identical-looking lines different.
4Read the count of what was removed. It is often more informative than the cleaned list itself.

Questions

Does it change the order of my lines?: No. The first occurrence of each line stays where it was and later copies are dropped. That is the difference from a sort-and-unique pipeline, which removes duplicates and reorders everything as a side effect.
Two lines look identical but both survived.: Almost certainly trailing whitespace, or a non-breaking space rather than an ordinary one. Switch on trimming, and if that does not resolve it, run the line through the Unicode inspector to see what is actually there.
Should I ignore case?: For email addresses, yes — the domain is case-insensitive and the local part is in practice. For identifiers, file paths and anything where capitalisation is meaningful, no.
Can it show me only the duplicates instead?: Not directly. Sorting the list first makes repeated lines adjacent and easy to see, which is the usual way to inspect them before deciding to remove anything.