Skip to content
EVOA Toolbox

Remove Duplicate Lines

Delete duplicate lines from a list, keep the first or last occurrence, ignore case, trim spaces and sort. See how many were removed.

Processed locally in your browser. Your text is not uploaded.

Loading tool…

What this tool does

Paste a list and get back only the unique lines. It works on anything line-based: e-mail addresses, URLs, keywords, log entries, SKUs, domain lists or exported IDs. A summary tells you how many lines you started with, how many were removed and how many remain.

Control how lines are compared. Case sensitivity can be turned off so 'Apple' and 'apple' count as the same line, trimming ignores stray spaces at the ends of lines, and you can choose to keep the first or the last occurrence of each duplicate. Blank lines can be preserved exactly as they are or dropped entirely, and the result can be sorted A to Z or Z to A with natural number ordering (item2 before item10).

Order is preserved by default, which matters for ordered data. Windows (CRLF) and Unix line endings are respected, and the work is done with a set lookup so even hundreds of thousands of lines finish quickly in your browser.

How to use it

  1. 1Paste your list into the input box (or press Example).
  2. 2Set the options: case sensitivity, trimming, blank-line handling, first or last, and sorting.
  3. 3Check the summary to see how many duplicate lines were removed.
  4. 4Copy the result or download it as a .txt file.

Supported formats

Plain text, one item per line. Line endings (LF, CRLF) are detected and kept.

Privacy

This tool runs in your browser. The data you provide is processed on your device and is not sent to our servers.

Limitations

  • Lines are compared as whole strings; there is no fuzzy or partial matching.
  • Case-insensitive comparison uses simple lowercase folding, which is not perfect for every language.
  • Sorting uses your browser's locale rules, so the order of accented letters may differ between browsers.
  • In 'keep blank lines' mode blank lines are never counted as duplicates, even if there are many of them.
  • Very large inputs (tens of megabytes) may be slow because everything is held in memory.

FAQ

Does it keep the original order?

Yes, unless you choose a sort order. By default the first occurrence of each line stays where it was.

How do I keep the last occurrence instead of the first?

Choose 'Keep last' in the options. The last copy of each line stays in its place and earlier copies are removed.

What does trimming do?

Leading and trailing spaces and tabs are removed from every line, in the output as well, so ' apple' and 'apple ' count as duplicates.

Can it handle thousands of lines?

Yes. A 200,000-line list is deduplicated in well under a second on a typical computer.

Related tools