Text Dedupe

Paste a list and clean it: remove duplicate lines (keeping order), drop blanks, trim spaces, and see what changed.

Exported email lists, copied spreadsheets and pasted logs all arrive with duplicates, blanks and stray whitespace. Paste the list, check the cleanup options you need, and get a clean result with a before/after count.

What each option does

Remove duplicate lines keeps only the first occurrence of each line. Keep original order avoids reshuffling your list alphabetically. Remove blank lines drops empty or whitespace-only lines. Trim edges strips spaces and tabs from both ends of every line. Remove all spaces strips them inside the line too — useful for phone numbers and IDs.

Practical notes

Matching is exact and case-sensitive: "John@mail.com" and "john@mail.com" are different lines. If you need case-insensitive dedupe, convert the text to lowercase with the Case Converter first, dedupe, then restore. Very large lists (hundreds of thousands of lines) are still fast — the logic is a hash set.

Frequently asked questions

Does it treat full-width and half-width characters as the same?
No — "123" (full-width) and "123" (half-width) are different characters and will not dedupe against each other.
Can I dedupe by just one column?
The tool matches whole lines. To dedupe by a single column, paste just that column here, dedupe it, then match the rest in your spreadsheet.
How large can my list be?
Hundreds of thousands of lines process in under a second. Multi-million line files may need splitting.