List txt, remove the two duplicates

If you have access to a Linux distribution (this should also work on Windows, but perhaps in a different way).

file
A
B
B
C
D
D

In a terminal:

uniq -u file

A
C

To save:

uniq -u file > file2

A bit simpler than with LibreOffice! uniq is part of GNU coreutils, a set of tools that should be included by default in most distributions.

Hello acefalo. The manipulation with Notepad++ leaves me with one of the lines that was duplicated, so it doesn't work.

However, with LibreOffice Calc, it works very well. I'm amazed at the complexity of the manipulation; I wonder who comes up with stuff like that, lol.

However, when I have to drag the little black square at the bottom right of cell B1 to go all the way down, I wonder if there's a way to pull it down without dragging because if the list is long, it might get a bit complicated.

In any case, thank you very much, it helped me a lot.

I'm sorry, but this time I didn't understand well.

Where should I click to the right of Fx? I can see Fx at the far right of the window
https://www.udrop.com/cache/plugins/filepreviewer/424319/28493bf178b27944f4c0add4e46465057494de78dce7c5cac1a5d15ad4b54aea/1100x800_cropped.jpg

I can't click to the right because there's nothing there, and when I click on it, it opens a function box where I have to choose between SUM, AVERAGE, MIN, MAX, or IF.

I also don't understand what you mean by writing B1:B5000 or another? In my case, the formula I need to put in B1 is =COUNTIF(A:A, A1), is that what I should put?

I'm really sorry, but I've never really used this kind of software before.

P.S.: Sorry, I just saw about Linux, I do have Linux on a VM, so I'll try it; it's true that it's much simpler this way. You really know a lot about this.
Thank you again.

Hello,

I was doing two things at once (talking on the phone and writing). It should be to the left of Fx. At the top of the columns.

Hello,

I don't know how many lines are in your file, but I did some testing just to see.

I started by creating a file with 100,000 pseudo-random numbers from 1 to 100,000 containing 10,000 unique numbers. I suspected that LibreOffice Calc wouldn't be able to give me results with the explained method (the list of unique numbers). I think Excel would do the same. Maybe by using another method. Calculating the frequency and then sorting instead of copying the cells. I managed to calculate the frequency for 100,000 with LibreOffice, but it took a few minutes. Maybe it would be better with a macro.

The same test with the command line (I add

time

in front of the command to calculate the execution time). It takes a fraction of a second for 100,000 numbers.

time sort file | uniq -u > file.out

real 0m0.183s
user 0m0.200s
sys 0m0.004s

I tested for 5,000 numbers with LibreOffice and it works quite well, although it's much slower.

Regarding my previous comment about line endings. I tested with different line endings (Windows, Unix, Classic Mac) for the

sort

and

uniq

commands. It seems to work for files with Windows line endings, but Mac line endings don't work right from the

sort

. It needs to be converted then.

To display line endings with Geany, you need to go to the View menu/Show Line Endings. To convert = Document menu/Set Line Endings. I imagine the same option is available with Notepad++. It's always better to work with a copy of the original file.